[IPAC-List] scoring multiple answer questions

[Harry.Brull at pdininthhouse.com]

I'd suggest a rational scale, something on the order of:

+2 = essential to do
+1 = nice to do
0 = no harm/no benefit
-1 = bad to do
-2 = terrible to do

Get a bunch of SMEs together and come to consensus

Hope that works for you.

Harry Brull 


------------------------------------------------
Harry Brull |  Senior Vice-President 
PDI Ninth House
Global Leadership Solutions
 
1.612.337.8233  office
1.612.414.8998  mobile
1.612.337.3695  fax
Harry.Brull at PDINinthHouse.com

33 South Sixth Street
Suite 4900
Minneapolis, MN 55402
 
www.pdininthhouse.com 

-----Original Message-----
From: ipac-list-bounces at ipacweb.org
[mailto:ipac-list-bounces at ipacweb.org] On Behalf Of Megan Paul
Sent: Monday, March 21, 2011 1:50 PM
To: IPAC-List at ipacweb.org
Subject: [IPAC-List] scoring multiple answer questions

I am reviewing an on-the-job knowledge test developed by a client for 
in-house use (a pseudo certification test, I guess you could say). Many 
of the items are multiple answer, with the number of right and wrong 
answers varying across items. They want to give partial credit for 
getting some of the right answers correct. To do that, they'll also need

to penalize for selecting wrong answers (otherwise a test taker need 
only select all answers--right or wrong--to get full credit). So...I'm 
trying to help them figure out a reasonable way to do this. If the wrong

answers are worth the same points as the right answers (albeit with a 
negative value), there are cases where a test taker could lose more 
points than they'd earn. The only practice I can think to apply is the 
correction for guessing, but I've never used such formulas, and it seems

like it won't be so straightforward, given that a) we're talking about 
the possibility of multiple wrong answers for each question and b) the 
number of right and wrong answers varies across items (i.e., I can't 
arrive at a single formula for the whole test; might have to be a 
formula for each item? Is there such a thing?).

Any thoughts?

To be clear, I did not develop this test. I'm just being asked to weigh 
in after the fact.

Thanks in advance,
Megan

-- 
Megan E. Paul, Ph.D.
Research Assistant Professor
University of Nebraska-Lincoln
Center on Children, Families, and the Law
206 S. 13th Street, Suite 1000
Lincoln, NE 68588-0227

(402) 472-9812 Office
(402) 472-8412 Fax


_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
http://www.ipacweb.org/mailman/listinfo/ipac-list

Confidentiality Notice: All information in this communication, including any files or attachments, is intended for the sole use of the individual or entity to which it is addressed and may contain information that is confidential, proprietary and/or trade secret information entitled to protection and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender by return email and delete this communication from your system. Thank you for your cooperation.