R&D Hub

Published on Friday, February 17, 2023

NAEP Validity Studies Panel Publishes White Paper: NAEP Framework and Trend Considerations

NAEP Validity Studies Panel Publishes White Paper: NAEP Framework and Trend Considerations

In October 2022, NAEP Validity Studies (NVS) panelist Dr. Lorrie Shepard of the University of Colorado Boulder published a white paper making recommendations for a more “incremental and evolutionary approach” to updating assessment frameworks.

NVS is an independent panel of experts contracted by the National Center for Education Statistics (NCES) to study the validity of all aspects of the National Assessment of Educational Progress (NAEP), including assessment development, data collection and analysis, and reporting and data use. The most recent meeting was held on February 10, 2023. The panel’s work is important as technological innovations, such as digitally based assessments, push testing forward and generate new considerations for monitoring the validity of the NAEP assessment. This white paper is one of many produced by NVS panelists over more than 20 years of research on issues of NAEP validity. A complete list of reports produced by the NVS panel can be found here.

The most recent NVS white paper, Dr. Lorrie Shepard’s NAEP Framework and Trend Considerations, makes recommendations to improve NAEP assessment frameworks. Assessment frameworks are broad overview documents that serve as a “blueprint” to guide NAEP assessment development. Frameworks specify the knowledge and skills to be covered by an assessment and provide examples of the types of questions that should be included. Frameworks must be updated as subject-matter domains evolve, but this presents validity issues, because changing frameworks could threaten the comparability of trend data used to monitor progress in achievement over time. Dr. Shepard offers the central recommendation that NAGB “should develop a new policy to enable smaller and more frequent updates to existing frameworks.” NVS panel members have termed this an “evolutionary” approach. You can learn more about the details, potential benefits, and potential challenges of such an approach by accessing the white paper here.

Comments (0)Number of views (253)

More links

title of plugged in news

The Summer 2024 NAEP Data Training Workshop - Applications Open


Applications are now open for the summer 2024 NAEP Data Training Workshop! This workshop is for quantitative researchers with strong statistical skills who are interested in conducting data analyses using NAEP data. For the first time, participants in this year's training will get an introduction to COVID data collections. Learn more here!

EdSurvey e-book now available!


Analyzing NCES Data Using EdSurvey: A User's Guide is now available for input from the research community online here.  Check it out and give the team your feedback.

«April 2024»