R&D Hub

With the 2024 NAEP Doctoral Student Internship Program well underway, our interns are now developing their technical skills and knowledge in the field of large-scale assessment by working collaboratively with NAEP experts on client-based projects. Recently, two more interns, Paulina Grekov and Dekyi Suolang, participated in our ongoing interviews to introduce the newest cohort to the NAEP R&D community.

Summer is in full swing, and so is the 2024 NAEP Doctoral Student Internship! Our new interns are busy analyzing NAEP data alongside researchers from the National Center for Education Statistics and the American Institutes for Research. Naomi Duran and Saugat Pandey, two members of this newest intern cohort, have taken the time to answer a few questions and introduce themselves to the NAEP R&D community.

This month, we are excited to share the latest research from the NAEP R&D community, focusing on the evaluation of large language models (LLMs). “The Challenges of Evaluating LLM Applications: An Analysis of Automated, Human, and LLM-Based Approaches” will be presented at the LLM4Eval workshop at the 2024 Association for Computer Machinery Special Interest Group on Information Retrieval (ACM SIGIR) conference on July 18, 2024. This paper from AIR researchers Bhashithe Abeysinghe and Ruhan Circi explores innovative approaches to evaluating custom AI applications, addressing a crucial barrier to faster progress in generative AI.

This week, the NAEP R&D program had the opportunity to interview former NAEP doctoral intern Mingqin Zhang. Zhang presented at the National Council on Measurement in Education (NCME) annual meeting as part of a training session titled “Examining MIRT Models From a Graphical Perspective Using RShiny and Mathematica.” Zhang, now a research scientist at Amazon Web Services, joined the 2018 cohort of NAEP Doctoral Internship Program interns from the University of Iowa, College of Education’s Educational Measurement & Statistics program. We were glad to catch up with her after the conference and ask her some questions about her NAEP Doctoral Internship program experience as the program gets ready to welcome the newest cohort of interns.

The NAEP R&D program recently reconnected with former intern Yichi Zhang at the National Council on Measurement in Education (NCME) annual meeting, where AIR was presenting a study using the ROME (Bayesian Region of Measurement Equivalence) method she developed during her internship. We followed up after the conference to get her perspective on the NAEP Doctoral Internship program, which is gearing up for its next cohort in June. Read on to learn about Yichi’s experience, including advice for prospective interns and more about her ROME method for advancing equity in educational assessment.