As the 2022 cohort of NAEP doctoral interns begin their internships, we take a look at some of the research being conducted and presented by the 2021 cohort.
Elsie Lee-Robbins (joined by Tiago A. Calico, Juanita Hicks, and Cadelle Hemphill) conducted a virtual demonstration session titled “Re-Envisioning Quantitative Information: Communicating NAEP Data to Parents and Policymakers.” Novel NAEP visualizations were presented in order to:
demonstrate a framework for designing visualizations for distinct audiences with varying degrees of familiarity with data reporting. Reporting effectiveness of data visualizations is evaluated by leveraging learning objectives and assessments, providing empirical feedback on the effectiveness of designs which informs further development.
In conjunction with Xiaying Zheng, Young Yee Kim, and Xiaoying Feng, Haobai Zhang presented their paper “Evaluation Effects of Text-to-Speech on Mathematics Performance Using Propensity Score Models.” The abstract of their presentation was:
Text-to-Speech (TTS) is a universal design feature in digitally based assessment. This study evaluated effects of different levels of TTS use on mathematics performance among EL/SD and non-EL/SD students using NAEP mathematics assessment data for grade 4 and 8. Quantile treatment effects of TTS were also estimated across achievement distribution.
The Graduate Electronic Board Session saw Yi Yang partnering with Young Yee Kim, Xiaying Zheng, and Xiaoying Feng to present their study “Identifying Response Time Thresholds for Solution Behaviors.” Their presentation expounded upon:
Differentiating between effortful behavior (solution behavior) and non-effortful behavior (skipping, rapid guessing, etc.) can contribute to ensuring the validity of achievement test scores. This study explores the effective approach to identifying meaningful interactions for both item responses and item nonresponses by applying multiple threshold-setting methods to NAEP data.
Congratulations to Yi, Haobai, and Elsie, whose work is furthering NAEP innovation, research, and data communication!