The mission of the NAEP Doctoral Student Internship Program is to support current doctoral students by providing an opportunity to engage in methodological developments and secondary analysis and using data from the National Assessment of Educational Progress (NAEP). During the 10-week internship program, interns will work directly with researchers from the National Center for Education Statistics (NCES) and the American Institutes for Research (AIR) on research in topic areas such as psychometrics & statistical methods, process data, policy-relevant research, and re-envisioning quantitative information (data visualization). Doctoral interns will collaborate with teams on client-based projects and gain technical skills and knowledge in the field of large-scale assessment.
Meet the NAEP Doctoral Student Internship Alumni here.
Topic Areas
Click below to read more about each internship topic area:
Psychometrics & statistical methods
The NAEP psychometric and statistical methods research team applies advanced psychometric and statistical methods to investigate a wide range of methodological and substantive issues related to measurement in large-scale assessments. Potential topics for the internship program include: (1) improving proficiency estimation through marginal maximum likelihood latent regression for NAEP data and (2) implementing missing data techniques to large-scale data analysis. Applicants with knowledge and research experience in item response theory (IRT), latent regression, plausible value methodology, multiple imputations, and analysis of complex sample data are strongly encouraged to apply. Proficiency in R programming and various IRT/missing data R packages is expected. Familiarity with programming languages such as Bayesian modeling with Stan is an asset.
Selected psychometrics & statistical methods projects from prior interns include:
- Comparing Mantel–Haenszel and Wald differential item functioning (DIF) detection methods under matrix sampling of items
- Exploring gender-related differential item time functioning in digitally based assessments
- Examining the effect of digital familiarity on writing performance using multiple-group analysis
- Designing a novel multistage testing (MST) assembly method for a test with many subscales
- Applying Bayesian region of measurement equivalence (ROME) method to examine the invariance of NAEP student questionnaire index variables
As part of the eNAEP digital assessment delivery system, process data is collected. Process data represent student interactions with the assessment platform and the assessment tasks; the data consist of time-stamped records of student actions or activities (e.g., highlighter use and answer selection) as well as automatically generated actions (e.g., switching to the next section due to time out). Process data have the potential to provide valuable insights into students’ response processes and behaviors as well as aid in evaluating how students interact with assessment items, system functions, and exam delivery platform. Process data can be used as the primary data source or the auxiliary data source on a wide range of projects.
Potential projects in foundational process data research include data privacy and quality (e.g., developing a systematic approach to create process data sets and ensure information quality), implementing methods for the extraction of useful information from process data (e.g., pattern recognition, uncertainty modeling), and exploration of new methods for analyzing process data (e.g., sequence mining and process mining). Strong statistical skills and the ability to work effectively with large and complex data as well as good programming skills (R and/or Python) are required. Experience in advanced statistical modeling/machine-learning methods are preferred.
Selected process data projects from prior interns include:
- Understanding Students’ Problem-Solving Processes via Action Sequence Analyses
- Item profiling: Insights into characteristics and design of each item
- A Social Network Analysis of Answer Change Behavior Using NAEP Process Data
- Insights from students’ computations using the calculator
- A comparison of student disengagement across years
- Whether items can be reconstructed from the process data
The policy-relevant research team uses NAEP data to explore research questions informed by education, sociology, economics, psychology, public health, and any other related disciplines that have the potential to affect policy and practice in K–12 education. Topics for the internship program will focus on: (1) text analytics of teacher responses to the write-in questions in the 2019 National Indian Education Study (NIES) to identify effective teaching and learning strategies for American Indian and Alaska Native students as well as critical sociocultural contexts of their school and community, (2) investigating green space around NAEP Trial Urban District Assessment (TUDA) schools and its relationship with student performance, and (3) examining state-level variations in course offerings and their relationship to student achievement. Applicants with an interest in any of the contexts addressed above as well as experience with advanced statistical modeling methods (e.g., topic modeling, text categorization, geospatial analysis, multi-level modeling) are strongly encouraged to apply. Familiarity with R, desktop GIS, Stata, or Mplus is preferred.
Selected projects in policy-relevant research projects from prior interns include:
- Investigating the relationship between air pollution and education outcomes using NAEP and county-level data
- The relationship between algebra skills, NAEP performance, and postsecondary outcomes
- Behavioral incidents, school behavioral climate, and student level mathematics achievement: Exploration of NAEP and the School Survey on Crime and Safety (SSOCS)
- The relationships between student experiences and NAEP technology and engineering literacy achievement
- Examining the relationship between mathematics growth and growth in working memory in the early grades using the Early Childhood Longitudinal Study, Kindergarten Class of 2010–11 (ECLS-K:2011)
Re-envisioning Quantitative Information (Data Visualization)
The project in this topic area will focus on expanding the nature and scope of reports based on NAEP data by combining communication and information science skills for highly-engaged and technically sophisticated constituencies, such as NAEP’s National Assessment Governing Board (NAGB) members, assessment experts, and other stakeholders. The intern will be part of a multidisciplinary team composed of NCES and AIR staff, including analysts, graphic designers, measurement scientists, and programmers.
The intern will use demographic and performance data from NAEP and other federal datasets to explore aspects of student learning experiences. Topics for the internship program will include: (1) an examination of instruction modes (e.g., remote, hybrid), NAEP performance by student groups (e.g., region, location), and the impact of COVID-19 (e.g., cases, deaths); and (2) an exploration of different aspects of school climate (e.g., school disruptions like school violence), learning experiences (e.g., hours of live teacher instruction), and variation in NAEP performance. The ideal applicant will possess an interest in communicating statistical information; experience creating advanced, interactive data visualizations; and demonstrable technical proficiency in their chosen programming framework.
Selected projects from prior interns in this area include:
- Developing interactive data visualizations that accurately convey statistically valid insights drawn from combining NAEP with other federal and state data sets.
- Developing interactive visualizations allowing the comparison across states and time of student access to high-speed internet and computer resources over the last decade.
Qualifications
- Strong organizational and interpersonal skills.
- Current doctoral student in educational measurement, statistics, information sciences, sociology, psychology, economics, computer science, or a related field.
- Current doctoral student that has completed two years of coursework in a Ph.D. program OR current doctoral student with a previously completed master’s degree in educational measurement, statistics, information science, sociology, psychology, economics, computer science, or a related field.
- Experience with applying advanced statistical and/or psychometric methods
- Knowledge of item response theory (IRT) and/or sampling theory.
- Experience analyzing data from large-scale surveys and assessment data with complex sampling design is a plus.
- Software skills requirements:
- For applicants to the Policy, Process Data, and Psychometrics topic areas, demonstrable advanced experience with statistical software such as, but not limited to, Mplus, Python, R, and Stata.
- For applicants to the Re-envisioning Quantitative Information topic area, demonstrable advanced experience with statistical software such as, but not limited to, D3.js, Nivo, Python, R, React-Vis, Shiny, and Tableau.
Additional application requirements
- A cover letter to express applicant’s interest and qualifications for a primary and secondary area.
- A transcript (unofficial copy), writing sample as first author, and a letter of recommendation may be requested at a later date.
- Candidates should expect to demonstrate their stated coding and analytical skills during the selection process.
Responsibilities
The doctoral student interns will contribute to the development of statistical methods and substantive knowledge about education that inform the discussion, debate, and planning of decision-makers at national, state, and local levels. Specific research topics and project assignments will be based on a consideration of applicants’ skills and interests as well as NCES’s priorities. Example opportunities include presentations to the NCES client and proposal submissions to research conferences.
Internship Details
- Ten weeks starting June 1, 2023.
- The 2023 NAEP Doctoral Student Internship Program will be a virtual program. Interns are allowed to work remotely within the U.S.
- Interns will receive an AIR-issued laptop to complete project work.
- This is a paid internship.
Application Process
- Apply online: Here
- Deadline: January 27, 2023
- Notification date: Late February 2023
- For internship questions email: naepinternship@air.org