In September 2024, the NAEP Validity Studies (NVS) panel published a report titled Study of Changes in Public School Composition During the Pandemic and Their Relationship With NAEP Performance, co-authored by American Institutes for Research (AIR) researchers Martin Hooper, Rebecca Shipan, David Bamat, and George Bohrnstedt.

This most recent research publication provides context for the major declines in eighth-grade NAEP mathematics achievement from 2019 to 2022, with the 2022 results marking the first release of Main NAEP data since the onset of the COVID-19 pandemic in 2020. Noting losses in public school enrollment during this period, the study authors focused on two primary goals: “(a) to examine changes in the demographic composition of students among eighth graders in U.S. public schools between 2019 and 2022 and (b) to estimate the extent to which the compositional changes relate to NAEP eighth-grade mathematics achievement scores in the same time period.”

The researchers found that compositional changes between 2019 and 2022 largely mirrored trends from previous years. They also found that only a “relatively small proportion” of the declines in eighth-grade NAEP mathematics achievement could be attributed to these changes. Instead, the declines are “best understood as the result of decreases in performance,” with compositional changes playing an accentuating role to varying degrees across states and districts. For more details on the study and its methodology, see the study page here.

NVS is an independent panel of experts contracted by the National Center for Education Statistics (NCES) to study the validity of all aspects of the National Assessment of Educational Progress (NAEP), including assessment development, data collection and analysis, and reporting and data use. Since its formation in 1995, the panel has identified and explored a range of technical concerns and promising techniques relevant to NAEP, especially as technological innovations—such as digitally based assessments—push testing forward and invite new research. This study is one of many research products produced by NVS over nearly 30 years of research on NAEP validity.

Additional detail on the role of the NVS panel and a complete list of research products can be found here. To stay informed about NVS activities, subscribe to their mailing list here.