Exam Analytics: Response Frequencies

Exam analytics: What are response frequencies? Some people refer to this as a distractor analysis or a foil analysis. Response frequencies show the number of exam takers who selected each response option. Incorrect response options are also called distractors or foils. You want somewhere around 2.5-5% of exam takers to select each distractor. In a group of 300 students, that would mean around 10 students chose each wrong answer. If you have a lot of response frequencies of 0 or single digits, you might want to re-explore how believable your distractors were. However, remember if this was a mastery item, meaning an item about a concept we’ve taught so many times that we expect everyone to get it correct, we would expect to see zeroes for the distractors and that would be okay. A good way to develop distractors is to consider the differentials for a condition discussed in an item or to think about answers that would be very close to correct. In this summary, our middle two items have multiple zeroes for distractors. This could mean the items were too easy, because the other answer choices weren’t believable to students. To investigate our suspicions, we can look at the percent correct and see that these both scored above 90%, which also suggests they were easier items. When we look at the upper 27% and lower 27%, it suggests both high and low exam performers did really well on these items. The point biserial and discrimination indices look low on the first item, though not too bad on the second. But still, our advice would be to review the full text and the response options to edit them for plausibility. Don’t hesitate to contact the Instructional Design and Assessment team for more information on exam tagging, item statistics, and other assessment or instructional design topics. And don’t forget to visit bit.ly/msucomexams to check out our job aids on exams and other topics.