R&D Hub

This week, we’re able to share a new working paper, Do Survey Questions Have a Mood? Linking Sentiment to Student Test-Taking Behaviors, from NAEP researchers and co-authors Soo Lee, Ruhan Circi, and Adam Hearn. The working paper explores how test-taking behaviors are impacted by the sentiment of assessment questions, as well as how AI-powered sentiment analysis can help to analyze survey item responses such as those collected by NAEP survey questionnaires. The full text of the working paper is included below; subscribe to the NAEP R&D mailing list to stay up to date as this research develops and to access more opportunities to learn about NAEP data analysis.

In September 2024, the NAEP Validity Studies (NVS) panel published a report titled Study of Changes in Public School Composition During the Pandemic and Their Relationship With NAEP Performance, co-authored by American Institutes for Research (AIR) researchers Martin Hooper, Rebecca Shipan, David Bamat, and George Bohrnstedt.

This month, we are excited to share the latest research from the NAEP R&D community, focusing on the evaluation of large language models (LLMs). “The Challenges of Evaluating LLM Applications: An Analysis of Automated, Human, and LLM-Based Approaches” will be presented at the LLM4Eval workshop at the 2024 Association for Computer Machinery Special Interest Group on Information Retrieval (ACM SIGIR) conference on July 18, 2024. This paper from AIR researchers Bhashithe Abeysinghe and Ruhan Circi explores innovative approaches to evaluating custom AI applications, addressing a crucial barrier to faster progress in generative AI.

NAEP researcher David Bamat recently published a new research article, “Closing Reporting Gaps: A Comparison of Methods for Estimating Unreported Subgroup Achievement on NAEP,” in Measurement: Interdisciplinary Research and Perspectives.

Explore the latest insights in research and development from NAEP researchers in the R&D program. Given the current buzz around AI, this month we’re excited to share a working paper on the evaluation of chatbots by Ruhan Circi and Bhashithe Abeysinghe, NAEP researchers from the American Institutes for Research (AIR). The full text of the working paper is included below. Don’t miss out on these valuable perspectives – subscribe now to stay ahead in technology and innovation!