“Explain it in your own words.” Open-ended prompts are a meaningful opportunity to ask students how well they understand a concept.
Evaluating responses, on the other hand, can be challenging and time-consuming—and often unfeasible in large-enrollment undergraduate STEM courses, which typically rely on multiple-choice exams.
Throughout his career, Associate Professor Emeritus Mark Urban-Lurain has explored how science, technology, engineering and math (STEM) education can use more authentic computerized assessments to move beyond multiple choice.
This work—along with a dedication to mentoring the next generation of researchers—earned Urban-Lurain the prestigious distinction of American Association for the Advancement of Science (AAAS) Fellow in 2021. The lifetime honor is one of science’s top accolades, awarded by the world’s largest general scientific society.
Before he retired in 2018, Urban-Lurain was the director of the Automated Analysis of Constructed Responses (AACR) research group, a collaboration of researchers from multiple universities and a part of MSU’s CREATE for STEM Institute.
Founded in the early 2000s, the group identified assessment challenges in large-enrollment introductory science courses as a source of undergraduate students dropping out of STEM.
“When students write what they know, it’s much more revealing of their understanding,” explained Urban-Lurain. “You see a bunch of ideas that are mixed together, and that’s important for faculty who wish to help them understand a concept.”
AACR developed computer models that use machine learning to evaluate student writing like an expert would. Their online tool allows STEM professors to select from a list of open-ended questions and have student responses categorized through computerized text analysis. The software is informed by hundreds of previously expert-scored responses of student writing.
These techniques provide important insight into students’ understanding and have the potential to improve assessment practices across STEM disciplines.
AACR’s work has influenced related research projects, including PASTA (or “Potential of Automatically Scored Three-dimensional Assessment”). Also a part of CREATE for STEM, PASTA will develop tools to examine how science teachers use automatically scored science assessment tasks and create related instructional supports.
Urban-Lurain was one nine MSU researchers elected as AAAS fellows in 2021, joining dozens of Spartan fellows elected in previous years.
Read more about AACR: beyondmultiplechoice.org
*Last updated 12/14/22 to reflect Urban-Lurain was elected an AAAS fellow in 2021, not 2022 as previously written.