R&D Hub

This month, we are excited to share the latest research from the NAEP R&D community, focusing on the evaluation of large language models (LLMs). “The Challenges of Evaluating LLM Applications: An Analysis of Automated, Human, and LLM-Based Approaches” will be presented at the LLM4Eval workshop at the 2024 Association for Computer Machinery Special Interest Group on Information Retrieval (ACM SIGIR) conference on July 18, 2024. This paper from AIR researchers Bhashithe Abeysinghe and Ruhan Circi explores innovative approaches to evaluating custom AI applications, addressing a crucial barrier to faster progress in generative AI.

There's just one weekend remaining to apply for the 2024 NAEP Data Training Workshop! After you've submitted your application, check out another valuable resource to stay on top of NAEP news: the official blog of the nation’s report card, NAEP Plus.

The NAEP R&D program recently reconnected with former intern Yichi Zhang at the National Council on Measurement in Education (NCME) annual meeting, where AIR was presenting a study using the ROME (Bayesian Region of Measurement Equivalence) method she developed during her internship. We followed up after the conference to get her perspective on the NAEP Doctoral Internship program, which is gearing up for its next cohort in June. Read on to learn about Yichi’s experience, including advice for prospective interns and more about her ROME method for advancing equity in educational assessment.

Applications are now open for the summer 2024 NAEP Data Training Workshop, which will take place on June 26–28, in Arlington, VA. Advanced graduate students, educational researchers at universities or other research organizations, analysts from state and local education agencies, and those interested in analyzing large-scale assessment data will want to check out this upcoming opportunity to learn how to conduct data analysis using NAEP data.

The National Council on Measurement in Education (NCME) and American Educational Research Association (AERA) Annual Meetings are taking place in Philadelphia on April 11–14, 2024. In this post, we’re highlighting a small selection of presentations, trainings, and demonstrations from NAEP researchers in the NCES Center for Process Data and the EdSurvey Team, as well as some NAEP Doctoral Internship alumni who are participating in these Annual Meetings.