In October 2022, NAEP Validity Studies (NVS) panelist Dr. Lorrie Shepard of the University of Colorado Boulder published a white paper making recommendations for a more “incremental and evolutionary approach” to updating assessment frameworks.

NVS is an independent panel of experts contracted by the National Center for Education Statistics (NCES) to study the validity of all aspects of the National Assessment of Educational Progress (NAEP), including assessment development, data collection and analysis, and reporting and data use. The most recent meeting was held on February 10, 2023. The panel’s work is important as technological innovations, such as digitally based assessments, push testing forward and generate new considerations for monitoring the validity of the NAEP assessment. This white paper is one of many produced by NVS panelists over more than 20 years of research on issues of NAEP validity. A complete list of reports produced by the NVS panel can be found here.

The most recent NVS white paper, Dr. Lorrie Shepard’s NAEP Framework and Trend Considerations, makes recommendations to improve NAEP assessment frameworks. Assessment frameworks are broad overview documents that serve as a “blueprint” to guide NAEP assessment development. Frameworks specify the knowledge and skills to be covered by an assessment and provide examples of the types of questions that should be included. Frameworks must be updated as subject-matter domains evolve, but this presents validity issues, because changing frameworks could threaten the comparability of trend data used to monitor progress in achievement over time. Dr. Shepard offers the central recommendation that NAGB “should develop a new policy to enable smaller and more frequent updates to existing frameworks.” NVS panel members have termed this an “evolutionary” approach. You can learn more about the details, potential benefits, and potential challenges of such an approach by accessing the white paper here.