R&D Hub

New Research Paper on Advances and Challenges in Evaluating LLM-Based Applications

Published On Friday, June 7, 2024

This month, we are excited to share the latest research from the NAEP R&D community, focusing on the evaluation of large language models (LLMs). “The Challenges of Evaluating LLM Applications: An Analysis of Automated, Human, and LLM-Based Approaches” will be presented at the LLM4Eval workshop at the 2024 Association for Computer Machinery Special Interest Group on Information Retrieval (ACM SIGIR) conference on July 18, 2024. This paper from AIR researchers Bhashithe Abeysinghe and Ruhan Circi explores innovative approaches to evaluating custom AI applications, addressing a crucial barrier to faster progress in generative AI.

Summer 2024 NAEP Data Training Workshop: Applications Now Open

Published On Friday, April 12, 2024

Applications are now open for the summer 2024 NAEP Data Training Workshop, which will take place on June 26–28, in Arlington, VA. Advanced graduate students, educational researchers at universities or other research organizations, analysts from state and local education agencies, and those interested in analyzing large-scale assessment data will want to check out this upcoming opportunity to learn how to conduct data analysis using NAEP data.

NAEP Researchers and NAEP Doctoral Internship Alumni at NCME and AERA 2024 Annual Meetings

Published On Friday, March 29, 2024

The National Council on Measurement in Education (NCME) and American Educational Research Association (AERA) Annual Meetings are taking place in Philadelphia on April 11–14, 2024. In this post, we’re highlighting a small selection of presentations, trainings, and demonstrations from NAEP researchers in the NCES Center for Process Data and the EdSurvey Team, as well as some NAEP Doctoral Internship alumni who are participating in these Annual Meetings. 

NAEP Math Automated Scoring Challenge Results and Preprint

Published On Friday, December 8, 2023

NCES held the NAEP Math Automated Scoring Challenge in spring 2023, inviting anyone eligible for NCES secure data access to use algorithms and automated scoring techniques to predict the scores given by human raters to open-ended responses to 10 released NAEP mathematics assessment items. The results of this challenge (as well as those of the prior challenge using reading assessment items) are now available in a brief preprint.

1234
title of plugged in news

The Summer 2024 NAEP Data Training Workshop - Applications Open

04-12-2024

Applications are now open for the summer 2024 NAEP Data Training Workshop! This workshop is for quantitative researchers with strong statistical skills who are interested in conducting data analyses using NAEP data. For the first time, participants in this year's training will get an introduction to COVID data collections. Learn more here!

EdSurvey e-book now available!

02-14-2022

Analyzing NCES Data Using EdSurvey: A User's Guide is now available for input from the research community online here.  Check it out and give the team your feedback.

«June 2024»
MonTueWedThuFriSatSun
272829303112
3456789
10111213141516
17181920212223
24252627282930
1234567