R&D Hub

Research First Look: Can Large Language Models Transform Automated Scoring Further?

Published On Friday, September 6, 2024

This week, the NAEP R&D Hub is able to offer yet another sneak peek at some upcoming research from experts in large-scale assessment. Authored by Ruhan Circi, a NAEP process data researcher from the American Institutes for Research, alongside Maggie Perkoff, one of our 2024 summer doctoral interns, this working paper was produced as part of the Summer 2024 NAEP Doctoral Student Internship Program AI focus area. The authors also wish to thank Bhashithe Abeysinghe for his review of the working paper. It presents insights from a literature review on using LLMs for automated scoring of constructed response items, briefly explores how advancements in LLMs can improve current scoring systems, and highlights the remaining challenges and areas for further research.

Research First Look: Passage Text Difficulty in the Hive of Language Models

Published On Friday, August 23, 2024

The NAEP R&D Hub is often able to give readers in the NAEP research community a sneak peek at upcoming research from experts in the field. This week, we’re happy to share some research work from NAEP researchers Ruhan Circi and Bhashithe Abeysinghe on passage text difficulty and language models, soon to become a white paper on the topic, “A Voice from Past: Passage Text Difficulty in the Hive of Language Models.” Check out the full text of the preview below and remember to subscribe to stay connected with other exciting research and opportunities to get involved!

EdSurvey 4.0.7 Released With New Assessment Support and Rounding Features

Published On Friday, August 9, 2024

EdSurvey 4.0.7 was recently released and is now available on CRAN-R. This update enables EdSurvey to support the analysis of main NAEP data up to 2022, NAEP long-term trend data up to 2023, the 2022 Monthly School Survey (MSS) Linking Study, and the 2021 School and Teacher Questionnaire Special Study. In addition, the package now supports two international large-scale assessments: PISA 2022 and ICILS 2023. A new rounding feature has also been introduced to facilitate rounding results according to NCES statistical standards or user-defined criteria. For details, users can refer to “?es_round” within package 4.0.7. For more information on minor changes and bug fixes in this update, visit the EdSurvey news page on CRAN-R.  

Interview With NAEP Doctoral Internship Program Alum: Yichi Zhang

Published On Friday, April 26, 2024

The NAEP R&D program recently reconnected with former intern Yichi Zhang at the National Council on Measurement in Education (NCME) annual meeting, where AIR was presenting a study using the ROME (Bayesian Region of Measurement Equivalence) method she developed during her internship. We followed up after the conference to get her perspective on the NAEP Doctoral Internship program, which is gearing up for its next cohort in June. Read on to learn about Yichi’s experience, including advice for prospective interns and more about her ROME method for advancing equity in educational assessment.

12345678
title of plugged in news

Apply Now: Summer 2025 NAEP Doctoral Student Internship Program

01-09-2025

Gain hands-on experience in educational research and analysis with the NAEP Doctoral Student Internship Program. Apply now here—the application deadline is February 1, 2025!

EdSurvey 4.0.7 Released!

08-09-2024

Check out news about the latest release from the EdSurvey team here. And remember you can always access their e-book here.  

«February 2025»
MonTueWedThuFriSatSun
272829303112
3456789
10111213141516
17181920212223
242526272812
3456789