Exam Analytics: How to tell if an item is good.

Exam analytics: Is this a “good” item? It can be difficult to determine what makes a “good” exam or a “good” item. It seems like if students perform well, it must be good, right? Well…not necessarily. A successful exam or item is one that’s both valid and reliable, meaning it tests what we want it to test, and it can reproduce similar results each time it’s used. Exam platforms like ExamSoft capture a variety of psychometrics to help gauge reliability and validity. Psychometrics are statistics that help us explore our exam and item performance. We need to consider exam psychometrics like the KR-20, which appears at the top of an ExamSoft summary report. We also need to consider item psychometrics like the % correct, the performance of the upper and lower 27%, the point biserial, the discrimination index, and the response frequencies. This can seem overwhelming at first, but we’re happy to guide you through it. If most learners respond correctly, that doesn’t necessarily mean an item is good. And if most learners respond incorrectly, that doesn’t necessarily mean an item is bad. A good exam or exam item is one that helps us effectively sort higher performers from lower performers while providing feedback to learners. The Kuder-Richardson Formula 20, or the KR-20 at the top of each ExamSoft summary report, gauges the performance of the exam as a whole. It estimates the chance that the same group of students would score the same if they took the exam again. Scores range from 0 to 1. 1 suggests a stronger relationship between items on the exam, meaning it may be more reliable and consistent. For integrated exams, KR-20 can be less reliable, because the exam includes items from different courses. The KR-20 is generally more reliable on exams with more items and bigger groups taking it. Don’t hesitate to contact the Instructional Design and Assessment team for more information on exam tagging, item statistics, and other assessment or instructional design topics. And don’t forget to visit bit.ly/msucomexams to check out our job aids on exams and other topics.