NAEP process data consist of time-stamped records of student actions (e.g.,
highlighter use, calculator use, answer selection, item navigation) within the
test delivery system. These time-stamped records are collected at each part of
the NAEP assessment (i.e., tutorial, cognitive section, and student
questionnaire) across multiple subject areas (e.g., mathematics, reading,
science). Records of actions/events executed by students (e.g., clicking on an
item option) or by the system itself (e.g., a time left message) are captured
in files by the assessment system.
How this work supports NAEP
Process data research addresses
Phase 2 (Create the Assessment) and
Phase 7 (Analyze and Report the Results) of the NAEP
Assessment Process. Process data research meets the strategic vision to
Inform, Innovate, & Engage by continuing to improve “the
content analysis, and reporting of NAEP contextual variables by considering
the questions’ relevance, sensitivity, and potential to provide meaningful
context and insights for policy and practice”.
Current Work
Currently, NAEP process data are logged but they are noisy and complex,
requiring detailed and strenuous exploration due to a lack of standards in
place. NAEP has taken multiple initiatives to increase the process data’s
potential utility and to improve access to them in order to help to form a
research community centered around this emerging field:
Process data provide NAEP the opportunity to gain insight into:
- Students’ testing experiences
-
Problem-solving behaviors (e.g., guessing, skipping patterns of items)
- Misconceptions
- Metacognitive processes (e.g., item revisits)
- Motivation, persistence, and engagement
Process data also support operational decisions on:
- Item development, analysis, and selection
- Questionnaire development and validation
- Block and test assembly optimization
- Understanding test administration conditions
Other Resources
-
Learn more
about the Center for Process Data
-
Courses offered at the
American Educational Research Association
-
Apply
to the NAEP Data Training Workshop
-
Apply
for the NAEP Doctoral Internship Program
-
Join
the NCME SIGIMIE Big Data in Educational Measurement
Image selected to be featured on the cover of one of the upcoming issues
of EM:IP Educational Measurement: Issues and Practice