The NAEP R&D program has recently welcomed a new cohort of summer interns who are just getting started learning about and working with NAEP data. While the program moves forward, we are taking a look back, talking to NAEP Doctoral Internship Program alumna Hailey Kuang to hear about her experience participating in the program and NAEP research.

Hailey Kuang joined the 2020 cohort of NAEP Doctoral Internship Program interns from the University of Florida College of Education’s Research and Evaluation Methodology Program. She focused on the Process Data Assessment Operations topic area, and her project, “Comparison of disengagement levels and the impact of disengagement on item parameters between PISA 2015 and PISA 2018 in the United States,” was published in Large-Scale Assessments in Education earlier this year.

Here's how Hailey describes her research now:

“Currently, I’m doing my research using the restricted-use response process data from the NAEP 2017 grade 8 mathematics assessment. Specifically, I’m working on predicting responses both at the item level (correct or incorrect) and at the assessment level, estimating students’ abilities using deep learning neural network models. I want to see to what extent the action sequence can reflect the students’ item responses. The purpose of this is to use the response procedure—the action sequence—to provide additional information for item selection in the computer-adaptive testing (CAT).”

Here's how Hailey feels about her experience with the program:

“I did my internship remotely in 2020 due to the pandemic. Although I didn’t have the chance to work directly alongside some of the AIR and NAEP mentors and other interns because of this, it was overall a wonderful experience. All the people involved were so easy to reach out to for any questions, and the communication in the program gave me the confidence and professional skills to reach out directly to organizations with large-scale assessment data with questions related to my research. I also feel like the experience working with large-scale assessment data in the internship helped me to know how to parse data to find the information I need.”

Hailey faced some challenges that the internship helped prepare her for:

“The challenges I faced with PISA and NAEP data involved finding the information I needed about the items, students, and procedures. Previously, I had mostly worked with simulated data. In the real-world data, I knew these things were findable, I just needed to reach out to the experts and data sources for some help. The internship helped me navigate not only the datasets, but also the often complicated process of applying for restricted-use data licenses. The internship mentors’ encouragement, support, and expertise in this area was very helpful.”

Here's how Hailey feels about future avenues for her research:

“There are a lot! Earlier this year, I was hired for the tenure-track assistant professor position at Florida State University, which I will start this coming fall. When I think about my career path, I see a lot of potential for exploring response process data in the NAEP and PISA datasets. We can see how good an assessment is by looking at how students interact with it. Personally, I want to help improve the design, administration, and analysis of assessments. I also want to explore how the analysis of interaction patterns can be translated across different assessments like NAEP and PISA to predict performance.”

Hailey talked a bit about how her research might impact policy and practice:

“I’m not sure at the moment how we might impact policy, but certainly practice! We’ve already been working on implementation based on the results of my research in an effort to improve item design. We’re also working on making the data accessible and interpretable for teachers. In the future, I’d like to further improve the design and understanding of student interactions with assessments.”

Hailey shared that she’d happily recommend the NAEP Doctoral Internship Program:

“Of course! I’ve already recommended not just the internship, but also the data training program to everyone in my program. I learned a lot from them, and the knowledge and experience I gained helped me both in my dissertation and in my current role. The program has also helped me create connections with talented scholars in the field. The environment is so friendly! I’ll be telling my future students to apply too! The internship is on the cutting edge of response process data, which is really a hot topic right now.”

The NAEP R&D team is thrilled to congratulate Hailey on her new job! We look forward to seeing the new and exciting paths her research takes. Stay tuned to the NAEP R&D Hub as we continue to put a spotlight on our interns, alumni, and other vital members of the NAEP R&D community. If you find Hailey’s experience and recommendation compelling, consider checking out the NAEP Doctoral Internship Program for yourself. You could be part of the next cohort of graduate students working directly with NCES researchers and engaging in methodological developments and secondary analysis using NAEP data. To be the first to apply to our next internship opportunity, keep checking in on the NAEP R&D Hub and sign up for our mailing list.