How do you want to get involved?

INTERNSHIP

PROGRAM

The NAEP Doctoral Student Internship Program

Overview

The mission of the NAEP Doctoral Student Internship Program is to advance and encourage secondary analysis and methodological developments using data from the National Assessment of Educational Progress (NAEP). During the10-week internship program, interns will work directly with researchers from the National Center for Education Statistics (NCES) and American Institutes for Research (AIR) on research in one of four topic areas: Psychometrics & Statistical Methods, Process Data, Policy-Relevant Research, and for the first time, Envisioning Quantitative Information for Digital Media (Data Visualization). Read more about each topic area in the descriptions listed below.

Meet the NAEP Doctoral Student Internship Alumni here.

Topic Areas

Click below to read more about each internship topic area:

The NAEP psychometric and statistical methods research team applies advanced psychometric and statistical methods to investigate a wide range of methodological and substantive issues related to measurement in large-scale assessments. Potential topics for the internship program include missing data, analysis of complex sample data, causal inference methods (e.g., propensity score methods), issues in multistage testing, measurement invariance, and test-taking behaviors (e.g., speededness and rapid guessing). Applicants with knowledge and research experience in structural equation modeling (e.g., factor analysis, path analysis, and finite mixture models), item response theory (e.g., calibration, scoring, linking, plausible value generation, and multidimensional/multigroup IRT), and response time modeling are strongly encouraged to apply. Familiarity with flexMIRT, Mplus, and/or various SEM/IRT R packages is preferred.

Selected Psychometrics & Statistical Methods projects from prior interns:

Applying CUSUM-based person fit statistics to detect speededness
Comparing Mantel-Haenszel and Wald DIF detection methods under matrix item sampling
Exploring gender-related differential item time functioning in digitally based assessments
Designing a novel MST assembly method for a test with many subscales

The introduction of NAEP digitally based assessments provides a large and rich data source logging the actions of students as they interact with the NAEP digital platform. Process data have the potential to provide valuable insights into students’ responding processes and behaviors as well as into how the assessment items, functions, and system are performing, and offer a wide range of projects. Interns who work in the process data area will be working on research in either of two priority areas: (1) general process data research or (2) assessment item and delivery platform development process data research.

(1) General process data research.

Potential projects include data privacy and quality projects (developing a systematic approach to creating process datasets and ensuring information quality), implementing methods for the extraction of useful information from process data (e.g., pattern recognition, uncertainty modeling), and exploration of new methods for analyzing process data. Strong statistical skills and the ability to work effectively with large and complex data, as well as good programming skills (R and/or Python), are required. Psychometrics skills and experience in advanced statistical modeling/machine learning methods are preferred.

Selected Process Data research projects from prior interns:

Revisiting the omitted and not-reached scoring rule using NAEP process data
Exploring item visits in process data and modeling students’ visit behaviors and intentions
Understanding students’ problem-solving processes via action sequence analyses
Item profiling: Insights into the characteristics and design of each item

(2) Assessment item and delivery platform development process data research.

Potential projects include investigating student interactions with digital tools, such as calculators and equation editors; investigating if interactive item components are being used as intended; and determining if students at different grade levels interact and use digital tools differently. The intern should have an interest in large-scale K-12 mathematics assessments and in learning how process data can be used to inform item and platform development along with other operational decisions.

Selected process data operation projects from prior interns:

Insights from students' calculator use

The policy-relevant research team uses NAEP data to explore research questions informed by education, economics, psychology, sociology, or any related disciplines that have the potential to affect policy and practice in schools. Potential topics for the internship program include using multilevel modeling to examine how science motivation is related to NAEP performance and NAEP achievement gaps; studying the predictive relationship between high school mathematics and science motivation, coursetaking, parent and peer influences, and NAEP grade 12 mathematics performance and student entrance into college STEM fields; understanding the mechanism behind students who are academically resilient and identifying the support they need; and validating a proposed index of socioeconomic status for NAEP and conducting analyses with such an SES index. Applicants with an interest in these and other topics in this area and experience with multi-level analysis, structural equation modeling, factor analysis, survival analysis, propensity scores, econometric techniques, and other advanced statistical modeling methods are encouraged to apply. Familiarity with Stata and/or R is preferred.

Selected Policy-Relevant Research projects from prior interns:

The role of mathematics motivation, self-other discrepancies, and STEM coursetaking in grade 12 mathematics achievement (using the overlap sample of NAEP and the High School Longitudinal Study of 2009)
The relationships between student experiences and NAEP technology and engineering achievement
College preparedness research using NAEP data or informing NAEP

Combining skills from communications and information science to explore new ways to envision quantitative information, a project in this area will focus on understanding/learning how non-experts use NAEP data. The goal of the 10-week internship is to explore innovative multimedia approaches to reporting NAEP data targeted to non-expert consumers. The intern will work with NCES and AIR staff to develop and conduct a research project of clear, compelling, and accurate data engagements for non-expert consumers of NAEP data (e.g., policymakers, parents, and teachers). Applicants with an interest in communication and visualization of statistical information are encouraged to apply.

Potential projects in Envisioning Quantitative Information for Digital Media:

Develop data visualizations that transform process data from the 4th-grade NAEP reading assessment to a diagnostic display to support reading instruction
Document best practices and innovative uses of NAEP reports at the state, district, and school levels with a focus on cross-team/varied-expertise stakeholder/cross-discipline use of NAEP reports for local decisionmaking

Qualifications

Doctoral students in educational measurement, statistics, information science, sociology, psychology, economics, computer science, or other related fields.
Two years of coursework completed in a doctoral program OR a master’s degree completed with at least one (1) year of coursework completed in a doctoral program.
Must possess strong organizational and interpersonal skills.
Experience with applying advanced statistical and/or psychometric methods
Sophisticated experience with statistical software packages such as Stata, R, or Mplus.
Knowledge of Item Response Theory and/or sampling theory.
Experience analyzing data from large-scale surveys and assessment data with complex sampling design is a plus.

Cover Letter Requirements

Please include in your cover letter information that help us understand why you would be a good fit for the internship in general and your preference and fit for one of the four topic areas: Psychometrics & Statistical Methods, Process Data, Policy-Relevant Research, and Envisioning Quantitative Information for Digital Media. Not submitting a cover letter will result in non-consideration for this position.

Additional Requirements

A transcript (unofficial copy), writing sample as first author, and a letter of recommendation may be requested at a later date.

Responsibilities

The Doctoral Student Interns will contribute to the development of statistical methods and substantive knowledge about education that inform the discussion, debate, and planning of decisionmakers at national, state, and local levels who are contracted with the National Center for Education Statistics. The goal of the internship program is to advance and encourage secondary analysis and methodological developments using NAEP data.

Specific research topics and project assignments in a primary and secondary area will be based on a consideration of applicants’ skills and interests and NCES’s priorities. Example opportunities include presentations to the NCES client and proposal submissions to research conferences.

Internship Details

10 weeks starting May 26, 2020
Based in AIR’s office in Crystal City, VA
Paid internship (up to $12,000 for 10 weeks)

Application Process

Apply online: Here
Deadline: February 18, 2020
Notification date: mid-March
For internship questions email: naepinternship@air.org

NAEP DATA

TRAINING WORKSHOP

COOPERATIVE

AGREEMENTS