R&D Hub

NAEP Researchers and NAEP Doctoral Internship Alumni at IAEA Annual Conference

Published On Friday, September 20, 2024

The 49th International Association for Educational Assessment (IAEA) Annual Conference will be held in Philadelphia in just a few days, on September 22–25, 2024. This year’s theme explores the growing interest in artificial intelligence by asking, “How Can AI Help Improve Educational Assessments?” In this post, we highlight upcoming IAEA presentations from NAEP researchers and NAEP Doctoral Student Internship Program alumni.

Research First Look: Can Large Language Models Transform Automated Scoring Further?

Published On Friday, September 6, 2024

This week, the NAEP R&D Hub is able to offer yet another sneak peek at some upcoming research from experts in large-scale assessment. Authored by Ruhan Circi, a NAEP process data researcher from the American Institutes for Research, alongside Maggie Perkoff, one of our 2024 summer doctoral interns, this working paper was produced as part of the Summer 2024 NAEP Doctoral Student Internship Program AI focus area. The authors also wish to thank Bhashithe Abeysinghe for his review of the working paper. It presents insights from a literature review on using LLMs for automated scoring of constructed response items, briefly explores how advancements in LLMs can improve current scoring systems, and highlights the remaining challenges and areas for further research.

Research First Look: Passage Text Difficulty in the Hive of Language Models

Published On Friday, August 23, 2024

The NAEP R&D Hub is often able to give readers in the NAEP research community a sneak peek at upcoming research from experts in the field. This week, we’re happy to share some research work from NAEP researchers Ruhan Circi and Bhashithe Abeysinghe on passage text difficulty and language models, soon to become a white paper on the topic, “A Voice from Past: Passage Text Difficulty in the Hive of Language Models.” Check out the full text of the preview below and remember to subscribe to stay connected with other exciting research and opportunities to get involved!

New Research Paper on Advances and Challenges in Evaluating LLM-Based Applications

Published On Friday, June 7, 2024

This month, we are excited to share the latest research from the NAEP R&D community, focusing on the evaluation of large language models (LLMs). “The Challenges of Evaluating LLM Applications: An Analysis of Automated, Human, and LLM-Based Approaches” will be presented at the LLM4Eval workshop at the 2024 Association for Computer Machinery Special Interest Group on Information Retrieval (ACM SIGIR) conference on July 18, 2024. This paper from AIR researchers Bhashithe Abeysinghe and Ruhan Circi explores innovative approaches to evaluating custom AI applications, addressing a crucial barrier to faster progress in generative AI.

123456
«December 2024»
MonTueWedThuFriSatSun
2526272829301
2345678
9101112131415
16171819202122
23242526272829
303112345