Last month, NAEP researchers Ruhan Circi and Juanita Hicks, along with Emmanuel Sikali, acting branch chief of reporting and dissemination in the Assessment Division at the National Center for Education Statistics (NCES), published their mini-review article titled “Automatic Item Generation: Foundations and Machine Learning-Based Approaches for Assessments.” The article was published in the open-access journal Frontiers in Education.
With artificial intelligence and natural language processing at the forefront of the public consciousness, it’s never been more vital to delve into the existing body of research on these topics as they relate to education and assessment. To this end, Drs. Circi, Hicks, and Sikali have created a valuable resource for the NAEP research community and the broader education research community. Their new mini review covers more than 40 papers, two multimedia sources, and one systematic literature review, all focusing on automatic item generation (AIG) in educational assessment.
The mini review defines AIG as “the process of generating items/questions from various inputs, including models, templates, or schemas” and identifies common promises that AIG will reduce the time and cost involved in item generation, ease the burden of maintaining large item pools, and support learning with items custom-tailored to specific needs. The authors explore whether these promises bear out in the current research and work in AIG, focusing on four key points of inquiry:
- purpose of AIG in the reviewed material,
- type of items generated,
- input type and approaches to generate items, and
- methods used to evaluate generated items.
One key conclusion was of the need for more research in this rapidly expanding field, which highlights the importance of broad topic overviews like this mini review. We hope the NAEP research community will take the opportunity to check it out so we can all acquaint or further familiarize ourselves with the current body of AIG research. Engaging in initial exploration is the first step toward generating the necessary new research!
If you want to further your ability to research and work with large-scale assessment data, consider joining the NAEP R&D Hub mailing list to stay up to date on internship and workshop opportunities. You could have the opportunity to work with and learn from NAEP researchers like the authors of this mini review!