Process Data

NAEP process data consist of time-stamped records of student actions (e.g., highlighter use, calculator use, answer selection, item navigation) within the test delivery system. These time-stamped records are collected at each part of the NAEP assessment (i.e., tutorial, cognitive section, and student questionnaire) across multiple subject areas (e.g., mathematics, reading, science). Records of actions/events executed by students (e.g., clicking on an item option) or by the system itself (e.g., a time left message) are captured in files by the assessment system.

How this work supports NAEP

Process data research addresses Phase 2 (Create the Assessment) and Phase 7 (Analyze and Report the Results) of the NAEP Assessment Process. Process data research meets the strategic vision to Inform, Innovate, & Engage by continuing to improve “the content analysis, and reporting of NAEP contextual variables by considering the questions’ relevance, sensitivity, and potential to provide meaningful context and insights for policy and practice”.

Current Work

Currently, NAEP process data are logged but they are noisy and complex, requiring detailed and strenuous exploration due to a lack of standards in place. NAEP has taken multiple initiatives to increase the process data’s potential utility and to improve access to them in order to help to form a research community centered around this emerging field:

Process data provide NAEP the opportunity to gain insight into:

• Students’ testing experiences
• Problem-solving behaviors (e.g., guessing, skipping patterns of items)
• Misconceptions
• Metacognitive processes (e.g., item revisits)
• Motivation, persistence, and engagement

Process data also support operational decisions on:

• Item development, analysis, and selection
• Questionnaire development and validation
• Block and test assembly optimization
• Understanding test administration conditions

Other Resources

• Learn more about the Center for Process Data
• Courses offered at the American Educational Research Association
• Apply to the NAEP Data Training Workshop
• Apply for the NAEP Doctoral Internship Program
• Join the NCME SIGIMIE Big Data in Educational Measurement