The NAEP R&D Program stayed busy in 2022, making strides on a wide variety of fronts. As the year comes to a close, let’s look back at some of the progress and achievements made in 2022.

If you want to keep up to date with what’s going on at NAEP R&D, make sure to keep checking in and sign up for our mailing list. Keep an eye out for upcoming EdSurvey data trainings!

NAEP Doctoral Internship Program

• In December, applications opened for the 2023 NAEP Doctoral Student Internship Program. The program is a 10-week paid remote internship that provides interns with the opportunity to dive into data from the National Assessment of Educational Progress (NAEP) while working directly with researchers from the National Center for Education Statistics (NCES) and the American Institutes for Research (AIR). You can learn more about the 2023 NAEP Doctoral Student Internship Program and apply now here.

• In August, NAEP acknowledged and congratulated the outgoing 2022 NAEP Doctoral Student Internship Program alumni. This next generation of the NAEP research community spent their summer on activities that ranged from investigating air pollution and education outcomes to developing interactive NAEP resources for a public audience. Check out a video of the 2022 alumni sharing their experience and their project work here.

• In May, the NAEP R&D Hub caught up with NAEP Doctoral Internship Program alumna Melanie Gonzalez for an interview on her experience as an intern and how the program has bolstered her career in education policy research. Melanie joined the internship under the policy-relevant research topic area in the summer of 2019 as a doctoral student from the University of Texas at Austin’s developmental psychology program. Check out Melanie’s interview to learn more about her experience with the program here.

Process Data

• In November, Process Data Researcher Dr. Ruhan Circi presented a paper that she co-authored, “Using NLP-Based Methods With Process Data to Explore Student Accessibility Feature Use Behavior in Mathematics,” at the Association for Public Policy Analysis & Management (APPAM) 2022 Fall Conference. In her presentation, Dr. Circi highlighted the applicability of natural language processing (NLP) methods to analyzing action patterns for accessibility feature use across disability types.

• Process Data Researcher Dr. Juanita Hicks was invited to Norway this year to present NAEP process data at a workshop on academic resilience. The workshop was part of a grant awarded to Dr. Nani Teig, associate professor at the University of Oslo, Norway, through the Centre for Advanced Studies Fellowship Program. Over the course of the workshop, Dr. Hicks participated in presentations and discussions using process data to investigate the different types of academic resilience that may be observed in student behavior.

EdSurvey

• In September, EdSurvey 3.0 became available on CRAN-R. This update provided education researchers with the ability to utilize cutting-edge techniques, including latent regression and artificial intelligence, to leverage complex statistical analysis of large-scale assessment data.

• In July, NAEP researchers Paul Bailey and Ting Zhang, along with NCES Acting Branch Chief of Reporting and Dissemination Emmanuel Sikali, presented a one-day course on “Analyzing NAEP and TIMSS Data With Direct Estimation Using the R Packages EdSurvey and Dire,” at the 2022 International Meeting of the Psychometric Society (IMPS) in Bologna, Italy.

• Also in July, the EdSurvey product development team published a new version of Dire, an R package intended to complement its primary statistical software application, allowing researchers to conduct linear regressions with a latent outcome variable.

• In April, the first introductory manual dedicated to introducing the EdSurvey R package to the education research community was made available online. Analyzing NCES Data Using EdSurvey: A User’s Guide is a living document with scope for revisions and additions, expanding NCES’s instruction on EdSurvey beyond just workshops and national and international conferences.

• In March, NCES added modules to its Distance Learning Dataset Training (DLDT) system on how to use the EdSurvey R package to analyze various NCES datasets. The DLDT is an interactive web tool that allows data users to learn about NCES data products and to appraise their usefulness for various purposes.

Data Trainings

• In October, NAEP began accepting applications to the Winter 2023 NAEP Data Training Workshop. Applications are now closed, but NAEP is hard at work preparing for the workshop, which will take place in January. You can learn more about the workshop here.

• In April, the National Council on Measurement in Education (NCME) and the American Educational Research Association (AERA) held their annual meetings, featuring trainings, demonstrations, and symposiums relevant to NAEP R&D efforts.
• • Before the meetings, a virtual data training was held by researchers from AIR under the guidance of NCES on how to analyze NAEP process data under a process mining framework to gain a better understanding of students’ response processes during the assessment.
  • At the meetings, NAEP researchers Yan Wang and Emmanuel Sikali were joined by cooperative agreement awardee Kristin Stoeffler in a virtual symposium called “The Impact of Item, Test, and Test-Taker Characteristics on Engagement During NAEP Assessments.”
  • NCES’s Emmanuel Sikali and Researchers Juanita Hicks, Ruhan Circi, and Tiago Calico also demonstrated the “Process Data iSmart Tool” at the meetings. This tool is designed to make process data accessible and interpretable to educators.