[IPAC-List] scoring multiple answer questions

[Mark Hammer]

As an aside/tangent, back when I was teaching university, I stumbled
across some papers in the journal Teaching of Psychology about what the
authors referred to as an "answer justification option" for multiple
choice exams.  I'm sure I've mentioned it here before.
 
The option permits students who aren't sure about whether they
interpreted the question correctly to provide a 1-2 sentence explanation
(on the exam booklet itself) of why they chose the response alternative
they chose.  If the explanation provided some tangible evidence they
understood the concept that the question was probing, they would get
credit.
 
In my own case, I made that half or partial credit, contingent on the
clarity of their explanation.  It required a bit more time to score,
than simply shipping the scan sheets off to computing services, although
in most instances the student had already blackened in the "correct"
alternative, so the justification didn't need to be looked at.  The
small amount of time invested was more than made up for by avoiding
those interminable doorway discussions with students eager to squeeze a
couple more points out of the exam.  The evidence of misunderstanding or
poorly phrased questions was there on the page, and no discussion was
needed.
 
The reason I mention it here is because the possibility of salvaging
*something* from a question, as opposed to nothing, had a clear effect
on the perceived fairness of the test.  Indeed, it was the replicable
finding of higher fairness ratings in the papers I found that spurred me
on to trying it out, and nothing in my subsequent experience ever
contradicted that initial observation.  Of course the partial credit
awarded had to be fair itself, and the questions reasonable, but the
opportunity to make multiple responses and pull something out of the
fire conforms to what we know about the properties of selection tools
that have higher acceptability to candidates.
 
The point here is that allowing the testee to salvage something from a
multiple-response/multiple-choice question, by having ranked choices
(best, 2nd best, etc), is very likely to hold some benefits with respect
to acceptability, and reduce challenges to the test as a result.  I
think that's good news.
 
Mark Hammer
Ottawa


>

This e-mail message is intended for the named recipient(s) and may
contain information that is privileged, confidential and/or exempt from
disclosure under applicable law. Unauthorized disclosure, copying or
re-transmission is prohibited. If you are not a named recipient or not
authorized by the named recipient(s), or if you have received this
e-mail in error, then please notify the sender immediately and delete
the message and any copies.
 >
Ce courriel est destiné exclusivement au destinataire mentionné en titre
et peut contenir de l'information privilégiée, confidentielle ou
soustraite à la communication aux termes des lois applicables. Toute
divulgation non autorisée, toute reproduction ou réacheminement est
interdit. Si vous n'êtes pas le destinataire de ce courriel, ou n'êtes
pas autorisé par le destinataire visé, ou encore, si vous l'avez reçu
par erreur, veuillez le mentionner immédiatement à l'expéditeur et
supprimer le courriel et les copies.