[IPAC-List] Threatening a Penalty for Guessing

[Mark Hammer]

To respond to Chris's comments.  In fact, the original research I drew from (stumbled onto in the journal "Teaching of Psychology") noted that students did not necessarily do better when the answer justification option was offered, but they did think of the exam as significantly fairer.  My experience in using it was no different.  Of course, in the employment testing or entrance-exam situation, one is not obliged to earn and maintain the buy-in of the unsuccessful the way one is in courses that have several exams distributed over the semester.  "If I do it for you, then I would have to do it for everyone" is a tremendously unsatisfying argument to students, or any other individual on the receiving end of a less than stellar result.  The advantage it offered me was that by allowing people an opportunity to provide evidence at the time, rather than post hoc. of what they believed the question or response alternative to be, I actually *could* do it for them without having to do it for everyone since I had incontrovertible evidence of the exam that they believed they were taking.  I can imagine circumstances where use of this option imposes undue hardship on the instructor, but in my instance it worked as well as promised.

But back to penalizing for guessing.  Ruminating over Chris' remarks got me to thinking that in the evaluation of test bias, and other aspects of item performance, we actually NEED people to take honest wild stabs at items, rather than hold off cautiously, since that very response forms the datum of our item analysis, doesn't it?  Moreover, if there are new items being piloted for whether they function as intended, you need every single response to that item you can lay your hands on.  If the phrasing of the item has enough of a hint of ambiguity about it that leads people to skip it for fear of penalization, you can cheat yourself out of acquiring sufficient data to assess the item.

And finally, you're only penalized for guessing if you guess wrong.  If you happen, for reasons of dumb luck rather than deep insight, to select the correct answer while guessing. you DON'T get penalized.  Hence the use of the phrase "may be penalized", in the absence of its companion phrase "...although who knows, you might get lucky". 

Mark Hammer
Ottawa