R&D Hub

The NAEP R&D team is ringing in the new year by sharing a new journal article from NAEP researchers that will serve as a useful resource for those looking to learn more about analyzing NCES datasets like NAEP’s. NAEP researchers Ting Zhang, Paul Bailey, and Yuqi Liao, along with acting chief of NCES's Reporting and Dissemination branch Emmanuel Sikali, recently published their article “EdSurvey: An R package to analyze large-scale educational assessments data from NCES” in Large-scale Assessments in Education.

In October 2024, the NAEP Validity Studies (NVS) panel published An Exploration of Opportunity to Learn and Implications for NAEP, a white paper co-authored by Sheila W. Valencia, James W. Pellegrino, and Richard P. Durán. The white paper explores the relationship between opportunities to learn and the interpretation of NAEP results as well as how the concept and indicators of opportunities to learn could be expanded upon to include more recent thinking around social contexts for learning in the classroom.

This week, we’re able to share a new working paper, Do Survey Questions Have a Mood? Linking Sentiment to Student Test-Taking Behaviors, from NAEP researchers and co-authors Soo Lee, Ruhan Circi, and Adam Hearn. The working paper explores how test-taking behaviors are impacted by the sentiment of assessment questions, as well as how AI-powered sentiment analysis can help to analyze survey item responses such as those collected by NAEP survey questionnaires. The full text of the working paper is included below; subscribe to the NAEP R&D mailing list to stay up to date as this research develops and to access more opportunities to learn about NAEP data analysis.

In September 2024, the NAEP Validity Studies (NVS) panel published a report titled Study of Changes in Public School Composition During the Pandemic and Their Relationship With NAEP Performance, co-authored by American Institutes for Research (AIR) researchers Martin Hooper, Rebecca Shipan, David Bamat, and George Bohrnstedt.

This month, we are excited to share the latest research from the NAEP R&D community, focusing on the evaluation of large language models (LLMs). “The Challenges of Evaluating LLM Applications: An Analysis of Automated, Human, and LLM-Based Approaches” will be presented at the LLM4Eval workshop at the 2024 Association for Computer Machinery Special Interest Group on Information Retrieval (ACM SIGIR) conference on July 18, 2024. This paper from AIR researchers Bhashithe Abeysinghe and Ruhan Circi explores innovative approaches to evaluating custom AI applications, addressing a crucial barrier to faster progress in generative AI.