Exam Analytics: Rescoring "bad" items.

Exam rescoring: What do I do with “bad” items or poor performance? By looking at item analytics, you should have a sense of how exam items are performing. We only recommend re-scoring an item if the item has errors, or it’s poorly written. Poorly written might mean it doesn’t give enough information for students to respond correctly, it doesn’t have convincing distractors, or for some other reason it’s not discriminating well between high and low performers. How do we know the discrimination is poor? You might see things like the upper 27% and lower 27% of students performing similarly on the item or the point biserial and discrimination index coming in low, which suggests success on the item and the exam don’t correlate. However, if the item is free of errors, well-written, and seems to be discriminating well even though the % correct was low, it may just be an opportunity to explore how fully you covered that content with students. Maybe it wasn’t taught at all, or maybe it wasn’t taught in a way that helped learners understand or retain it, which provides formative feedback for a teaching follow-up. It doesn’t mean the item itself was poor. Here’s an example: In this summary, the upper and lower 27% statistics show high performers on the exam overall didn’t respond to this item correctly. The low performers on the exam scored much higher than and the discrimination index and point biserial are very low. Based on this, the item probably wasn’t well-written or it had a flaw. We would recommend rescoring the exam so we don’t penalize students for a poorly-written item. We’d also suggest revisiting that item before it’s used again and making a note in ExamSoft explaining why it was rescored. There are different ways to rescore or adjust items. We don’t suggest awarding credit to everybody or removing the item from scoring. Why? Well, because either of these methods could increase scores for the students who answered incorrectly, and potentially even decrease scores for those who answered correctly. That wouldn’t make sense. The preferred approaches are to accept multiple answers if, in fact, there are more than one answer that can be considered most correct based on what was taught or the way the item was written, or to treat the item as a bonus point, so it’s removed from the points possible but those responding correctly still receive credit. Here’s another example. In this summary, the percent correct or p-value is .66, or 66%, which is a little low. But when we look at other stats, we see the upper 27% of high performers on the exam answered this item correctly overall and the lower 27% of performers responded correctly at a nice margin lower than this. The discrimination index and point biserial support the idea that the item is discriminating well between high and low performers because they’re both upward of .20, so we don’t assume there are any flaws in the way this item was written. We wouldn’t suggest rescoring this item. It was probably okay. If the faculty member or students were hoping for more correct responses here, we’d recommend revisiting the content in class to enhance their understanding. If you don’t feel it was sufficiently taught, you could also treat this as a bonus point. Don’t hesitate to contact the Instructional Design and Assessment team for more information on exam tagging, item statistics, and other assessment or instructional design topics. And don’t forget to visit bit.ly/msucomexams to check out our job aids on exams and other topics.