[IPAC-List] scoring multiple answer questions

[Doverspike,Dennis]

Megan,

Given this is a knowledge test, there are a lot of issues. Not that other people do not use similar type of scoring, but what is a "right" or a "wrong" answer? Are some answers then more "right" than others? What is a "correct" answer? How are the instructions worded - I am assuming it is something of the nature of check all of the possible right or plausible answers? 

What is wrong with losing more points than you earn? What is wrong with a simple - one point for every selected right answer and one point off for every selected wrong answer? Just add enough points to the total to get no below zero scores.

Another approach would be to just treat this as a series of True-False items. Which in a sense it is. So, just have people pick Right or Wrong for every single alternative. Give one point for every correct answer - whether it is a right or a wrong. In a sense this is the most honest, because you basically have a series of T-F items.

All of the above assumes you do not want to give less than 1 point  (for example .50) for some right or wrong answers.

Dennis


________________________________________
From: ipac-list-bounces at ipacweb.org [ipac-list-bounces at ipacweb.org] On Behalf Of Megan Paul [mpaul2 at unlserve.unl.edu]
Sent: Monday, March 21, 2011 2:49 PM
To: IPAC-List at ipacweb.org
Subject: [IPAC-List] scoring multiple answer questions

I am reviewing an on-the-job knowledge test developed by a client for
in-house use (a pseudo certification test, I guess you could say). Many
of the items are multiple answer, with the number of right and wrong
answers varying across items. They want to give partial credit for
getting some of the right answers correct. To do that, they'll also need
to penalize for selecting wrong answers (otherwise a test taker need
only select all answers--right or wrong--to get full credit). So...I'm
trying to help them figure out a reasonable way to do this. If the wrong
answers are worth the same points as the right answers (albeit with a
negative value), there are cases where a test taker could lose more
points than they'd earn. The only practice I can think to apply is the
correction for guessing, but I've never used such formulas, and it seems
like it won't be so straightforward, given that a) we're talking about
the possibility of multiple wrong answers for each question and b) the
number of right and wrong answers varies across items (i.e., I can't
arrive at a single formula for the whole test; might have to be a
formula for each item? Is there such a thing?).

Any thoughts?

To be clear, I did not develop this test. I'm just being asked to weigh
in after the fact.

Thanks in advance,
Megan

--
Megan E. Paul, Ph.D.
Research Assistant Professor
University of Nebraska-Lincoln
Center on Children, Families, and the Law
206 S. 13th Street, Suite 1000
Lincoln, NE 68588-0227

(402) 472-9812 Office
(402) 472-8412 Fax


_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
http://www.ipacweb.org/mailman/listinfo/ipac-list