NAEP Doctoral Student Internship Program

The mission of the NAEP Doctoral Student Internship Program is to support current doctoral students by providing an opportunity to engage in methodological developments and secondary analysis using data from the National Assessment of Educational Progress (NAEP). During the 10-week internship program, interns will work directly with researchers from the National Center for Education Statistics (NCES) and the American Institutes for Research (AIR) on research in topic areas such as psychometrics and statistical methods, process data, policy-relevant research, re-envisioning quantitative information (data visualization), and artificial intelligence. Doctoral interns will collaborate with teams on client-based projects and gain technical skills and knowledge in the field of large-scale assessment.

Meet the NAEP Doctoral Student Internship Alumni here.

Topic areas

Click below to read more about each internship topic area:

The NAEP psychometric and statistical methods research team applies advanced psychometric and statistical methods to investigate a wide range of methodological and substantive issues related to measurement in large-scale assessments.

One potential topic may be implementing missing data techniques to large-scale data analysis. Applicants with knowledge and research experience in item response theory (IRT), multiple imputations, machine learning methods (e.g., random forest, LASSO), plausible value methodology, analysis of complex sample data, and latent regression are strongly encouraged to apply. Proficiency in R programming and various IRT/missing data R packages is expected. Familiarity with programming languages such as Bayesian modeling with Stan is an asset.

Selected psychometrics and statistical methods projects from prior interns include:

  • Improving proficiency estimation through marginal maximum likelihood latent regression for NAEP data
  • Comparing Mantel–Haenszel and Wald differential item functioning (DIF) detection methods under matrix sampling of items
  • Exploring gender-related differential item time functioning in digitally based assessments
  • Examining the effect of digital familiarity on writing performance using multiple-group analysis
  • Designing a novel multistage testing (MST) assembly method for a test with many subscales
  • Applying Bayesian region of measurement equivalence (ROME) method to examine the invariance of NAEP student questionnaire index variables

As part of the eNAEP digital assessment delivery system, process data is collected. Process data represent student interactions with the assessment platform and the assessment tasks; the data consist of time-stamped records of student actions or activities (e.g., highlighter use and answer selection) as well as automatically generated actions (e.g., switching to the next section due to time out). Process data have the potential to provide valuable insights into students’ response processes and behaviors as well as aid in evaluating how students interact with assessment items, system functions, and exam delivery platform. Process data can be used as the primary data source or the auxiliary data source on a wide range of projects.

One potential topic may be an exploration of response change analysis for open-ended items and whether there is any indication in process data as to how and why students change their responses on open ended items. Potential methods of analysis include traditional Natural Language Processing (NLP) and novel transformer methods. We expect the candidate to have strong statistical skills, strong programming skills in Python and/or R, and the ability to work effectively with large-scale and complex data.

Selected process data projects from prior interns include:

  • Understanding students’ problem-solving processes via action sequence analyses
  • Item profiling: Insights into characteristics and design of each item
  • A social network analysis of answer change behavior using NAEP process data
  • Insights from students’ computations using the calculator
  • A comparison of student disengagement across years
  • Exploration of whether items can be reconstructed from the process data

The policy-relevant research team uses NAEP data to provide valuable insights and evidence that can inform education policies at district, state, and national levels. We explore diverse topics through research questions informed by education, sociology, economics, school finance, psychology, public health, and any other related disciplines that have the potential to affect policy and practice in K–12 education.

Potential topics may include: (1) measuring differential grading effects among high school mathematics courses using NAEP as a common benchmark, and (2) examining the relationship between different types of education expenditures paid from the COVID-19 Federal Assistance Fund and NAEP grade 8 mathematics performance. Applicants with an interest in any of the contexts addressed above as well as experience with advanced statistical modeling methods (e.g., structural equation modeling with polytomous categorical indicators, econometrics, multi-level modeling) are strongly encouraged to apply. Familiarity with R, Stata, or Mplus is preferred.

Selected policy-relevant research projects from prior interns include:

  • Text analytics using the 2019 NIES teacher responses to the write-in questions
  • Investigating the relationship between air pollution and education outcomes using NAEP and county-level data
  • The relationship between algebra skills, NAEP performance, and postsecondary outcomes
  • Behavioral incidents, school behavioral climate, and student level mathematics achievement: Exploration of NAEP and the School Survey on Crime and Safety (SSOCS)
  • The relationships between student experiences and NAEP technology and engineering literacy achievement
  • Examining the relationship between mathematics growth and growth in working memory in the early grades using the Early Childhood Longitudinal Study, Kindergarten Class of 2010–11 (ECLS-K:2011)

The re-envisioning quantitative information team focuses on expanding the nature and scope of reports based on NAEP data. The intern will be part of a multidisciplinary team composed of analysts, graphic designers, measurement scientists, and programmers. Projects in this topic area require interns to leverage their communication and information science skills to develop engaging and accessible reports for various audiences, such as educators, parents, policymakers, and researchers.

The intern will use NAEP and other federal datasets to explore aspects of student experiences with large-scale assessments. Potential topics may include: an examination of student experiences with the NAEP assessment, an exploration of student actions on the assessment by subgroup (e.g., region, locale, gender), and the development of a data story to display variations in student actions on the assessment.

The ideal applicant will possess an interest in communicating statistical information; experience creating advanced, interactive data visualizations; and demonstrable technical proficiency in their chosen programming framework.

Selected projects from prior interns in this area include:

  • Developing interactive data stories that accurately convey statistically valid insights drawn from combining NAEP with other federal and state data sets.
  • Developing interactive visualizations allowing the comparison across states and time of student access to high-speed internet and computer resources over the last decade.

Digitally based assessments furnish abundant opportunities for the use of AI tools and methods to advance students’ test-taking experiences. They also hold the potential to support and transform various elements of the assessment cycle, including item generation, item scoring, reporting advancement, and more.

This new topic area is looking for applicants with a solid foundation in machine learning, data analysis, and ethical AI principles to explore the innovative use of artificial intelligence in NAEP assessments within this digital landscape. As part of our dynamic team, you will be at the forefront of harnessing AI’s power to not only enhance learning outcomes but also contribute to educational measurement. You will have the unique chance to work on cutting-edge projects that leverage AI to significantly improve the accuracy, efficiency, and fairness of educational assessments.

A potential topic may involve developing AI algorithms and natural language processing (NLP) models to assist in the creation of high-quality assessment items and content augmentation.

Qualifications

  • Strong organizational and interpersonal skills
  • Current doctoral student in educational measurement, statistics, information sciences, sociology, psychology, economics, computer science, or a related field
  • Current doctoral student that has completed two years of coursework in a Ph.D. program OR current doctoral student with a previously completed master’s degree in educational measurement, statistics, information science, sociology, psychology, economics, computer science, or a related field
  • Experience with applying advanced statistical and/or psychometric methods
  • Experience analyzing data from large-scale surveys and assessment data with complex sampling design is an asset
  • Software skills requirements:
    • Sophisticated experience with statistical software such as, but not limited to, Mplus, Python, R, and Stata
    • Additional qualifications for re-envisioning quantitative information applicants: demonstrable advanced experience with D3.js, Nivo, React-Vis, Shiny, Tableau, etc.
    • Additional qualifications for artificial intelligence applicants: familiarity with generative models like transformers (e.g., GPT-4, Llama 2) and cloud services, good understanding of ethical AI principles, and experience with NLP and computer vision (preferred but not required)

Additional application requirements

  • A cover letter should be included with the initial application to express the applicant’s interest and qualifications for a primary and secondary area.
  • Only candidates selected for an interview will be asked to provide a writing sample and a letter of recommendation at a later date.
  • Candidates should expect to demonstrate their stated coding and analytical skills during the selection process.

Responsibilities

The doctoral student interns will contribute to the development of statistical methods and substantive knowledge about education that inform the discussion, debate, and planning of decision-makers at national, state, and local levels. Specific research topics and project assignments will be based on a consideration of applicants’ skills and interests as well as NCES’s priorities. Example opportunities include presentations to the NCES client and proposal submissions to research conferences.

Internship details

  • The 2024 NAEP Doctoral Student Internship Program will take place over 10 weeks starting early June.
  • The internship will be a remote program with one travel experience to Arlington, VA within the first week of the internship for onboarding and training. Additional travel may occur towards the end of the summer. All travel and lodging expenses will be fully covered by the program.
  • Interns are allowed to work remotely within the United States.
  • Interns will receive an AIR-issued laptop to complete project work.
  • This is a paid internship.

Application process

title of plugged in news

EdSurvey 4.0.7 Released!

08-09-2024

Check out news about the latest release from the EdSurvey team here. And remember you can always access their e-book here.