The NAEP Doctoral Student Internship Program

Overview

The mission of the NAEP doctoral student internship program is to advance and encourage secondary analysis and methodological developments using data from the National Assessment of Educational Progress (NAEP). During a 10-week internship program, interns will work directly with researchers from the National Center for Education Statistics (NCES) and American Institutes for Research (AIR) on research in one of three topic areas: Psychometrics & statistical methods, Process data, and Data analysis on policy-relevant topics.

Research Areas

Psychometrics & statistical methods

The NAEP statistical and psychometric research team applies advanced statistical and psychometric methods to investigate a wide range of methodological and substantive issues related to large scale assessments. Potential topics for the internship program include missing data, analysis of complex sample data, causal inference (e.g., propensity scores methods), issues in multistage testing, measurement invariance, student writing process, and test-taking behaviors (e.g., speededness and rapid guessing). Applicants with knowledge and research experience in structural equation modeling (e.g., factor analysis, path analysis, and finite mixture models), item response theory (e.g., calibration, scoring, linking, plausible value generation, and multidimensional/multigroup IRT), and response time modeling are strongly encouraged to apply. Familiarity with flexMIRT, Mplus and/or various SEM/IRT R packages is preferred.

Research projects in psychometrics & statistical methods from prior interns include:

• Applying CUSUM based person fit statistics to detect speededness
• Comparing Mantel-Haenszel and Wald DIF detection methods under matrix sampling of items
• Can a multiple-choice-items-only routing block reduce routing stage time in MST?
• Construction of an IRT-based SES index for NAEP 2003 –2017

Process data

The introduction of NAEP digitally based assessments furnishes a rich empirical data source reflecting the actions of students as they interact with the digital platform on which the assessment is presented. With its potential of providing valuable insights into students’ responding processes, process data serves a variety of diagnostic purposes and leads to a wide range of research projects. Potential topics for the internship program include data quality projects (developing a systematic approach to create process datasets and assure information quality), implementing methods for the extraction of useful information from process data (e.g., clustering techniques), exploration of new methods for analyzing process data. Strong statistical skills and the ability to work effectively with complex data as well as good programming skills (R and/or Python) are required. Psychometrics skills and experience in advanced statistical modeling/machine learning methods is preferred.

Research projects in process data from prior interns include:

• Effects of the extended time accommodation on performance in NAEP mathematics
• Item profiling: A case study of NAEP grade 4 mathematics process data

Data analysis on policy-relevant topics

Projects in this area use NAEP data to explore research questions informed by education research, psychology, sociology, or any related disciplines that have the potential to affect policy and practice in schools. Potential topics for the internship program include examining how science motivation is related to NAEP performance and NAEP achievement gaps taking both student level and school level into account (HLM approach); studying the effects of high-school mathematics and science motivation, course-taking, parent and peer influences and NAEP grade 12 mathematics performance on student entrance into college STEM fields; and validating a proposed index of socio-economic status for NAEP and conduct analyses with such an SES index. Applicants with an interest in these and other topics in this area and experience with multi-level analysis, structural equation modeling, factor analysis, survival analysis and other statistical modeling methods are encouraged to apply. Familiarity with STATA and/or R is preferred.

Research projects in data analysis on policy-relevant topics from prior interns include:

• Reading growth and summer loss: Helping those potentially at-risk. A study using the overlap sample of NAEP and the Early Childhood Longitudinal Study (ECLS-K:2011)
• The Role of mathematics motivation, self-other discrepancies and STEM course-taking in grade 12 mathematics achievement (Using the overlap sample of NAEP and the High School Longitudinal Study – HSLS:09)

Qualifications

• Doctoral students in educational measurement, statistics, sociology, computer science, or other related fields.
• Two years of coursework in a Ph.D. program or a master’s degree completed in a relevant field.
• Must possess strong organizational and interpersonal skills.
• Experience with applying advanced statistical methods
• Sophisticated experience with statistical software packages such as SAS, Stata, R, or Mplus.
• Knowledge of Item Response Theory and sampling theory is a plus.
• Experience analyzing data from large-scale surveys and assessment data with complex sampling design is a plus.

Responsibilities

The Doctoral Student Intern would contribute to the development of statistics about education that inform the discussion, debate and planning of decision-makers at national, state, and local levels through a contract with the National Center for Education Statistics. The goal of the internship program is to advance and encourage secondary analysis and methodological developments using data from the National Assessment of Educational Progress (NAEP).

Specific research topics and project assignments will be based on the consideration of applicants’ skills and interests, and NCES priorities. Example opportunities include presentations to the NCES client and proposal submissions to research conferences such as the American Educational Research Association or the National Council on Measurement in Education.

Internship details

• 10 weeks starting May 28th, 2019
• Based in AIR’s office in Crystal City, VA
• Paid Internship (Up to $12,000 for 10 weeks)

Application Process

• Apply online: Here
• Deadline: February 15, 2019
• Notification date: mid-March