[IPAC-List] Threatening a Penalty for Guessing

Lance Seberhagen sebe at erols.com
Wed Apr 28 00:10:08 EDT 2010

Here's a quick reference that's pretty good.

Frary, Bob (1988). Formula scoring of multiple-choice tests (correction
for guessing). NCME, Instructional Topics in Educational Measurement,
Module 4. http://www.ncme.org/pubs/items/ITEMS_Mod_4.pdf

Lance Seberhagen, Ph.D.
Seberhagen& Associates
9021 Trailridge Ct
Vienna, VA 22182
Tel 703-790-0796

On 4/27/2010 10:43 PM, Winfred Arthur, Jr. wrote:

> no, Joel, sorry for being unclear but i did not mean to suggest or

> imply that it is standard practice to correct both speeded and

> non-speeded tests for guessing/accuracy. however, to echo Dennis'

> point, we do not correct our knowledge tests for guessing -- this is

> primarily b/c there is not a strong conceptual case or basis for doing

> so. [although, i can envisage some domains in which errors and lack

> of knowledge might be so dangerous that you might want to penalize

> test takers for getting an answer wrong!! :) ]


> anyway, if you are interested in some reading on this, you might want

> to look at the "scoring of information-processing tests" section (pp.

> 60-61) of:


> Arthur, W. Jr., Doverspike, D., & Bell, S. T. (2004). Information

> processing tests. In M. Hersen, & J. C. Thomas (Eds.), /Comprehensive

> Handbook of Psychological Assessment: Volume 4, Industrial and

> Organizational Assessment/ (pp. 56-74). NY: John Wiley & Sons, Inc.


> it discusses the different ways in which tests (like your clerical

> speed test?) could be scored, namely, (1) # of correct responses

> [accuracy], (2) response time [speed], or (3) some weighted composite

> of accuracy and speed using specified algorithms.


> - winfred


> On 4/27/2010 11:13 AM, Joel Wiesen wrote:

>> Thanks for your thoughtful comments, Winfred.


>> Do I understand you correctly as saying it is standard practice to

>> correct both speeded and non-speeded test for guessing, but that you

>> do not do so for your (non-speeded) knowledge tests?


>> If so, why do you not correct for guessing?


>> Thx.


>> Joel




>> - -

>> Joel P. Wiesen, Ph.D., Director

>> Applied Personnel Research

>> 62 Candlewood Road

>> Scarsdale, NY 10583-6040

>> (617) 244-8859

>> http://appliedpersonnelresearch.com





>> Note: This e-mail and any attachments may contain confidential and/or

>> legally privileged information. Please do not forward any contents

>> without permission. If you have received this message in error please

>> destroy all copies, completely remove it from your computer, and

>> notify the sender. Thank you.




>> Winfred Arthur, Jr. wrote:

>>> Joel, some general thoughts are embedded below:


>>> On 4/27/2010 8:19 AM, Joel Wiesen wrote:

>>>> If a test's instructions say "you may be penalized for questions you

>>>> mark incorrectly" and then the grading does not correct for guessing,

>>>> what might the effect be?

>>> well, my first thought is that the use of "may" makes this

>>> problematic. it shld state explicitly whether one is going to do so

>>> or not. "may" without specifying the conditions under which this

>>> will or will not be invoked seems to me to be a recipe for . . .

>>> well, problems!


>>> and whereas i have not seen any empirical rsch or data on this, the

>>> college board uses this instruction set for some sections of the SAT

>>> [they do not use "may"; they use "will"] and it is my impression

>>> that students are more likely to leave these items blank than guess

>>> when they do not know the answer.



>>>> Has anyone had practical experience with such instructions? Do test

>>>> takers pay attention to such instructions?

>>> not personally. indeed to the contrary, i use an instruction set

>>> that states that there is no penalty for guessing and so it is in

>>> one's best interest to guess if one does not know the answer.

>>> subsequently, i rarely get any non-responses. of course, these are

>>> knowledge tests.


>>>> Is there research on this type of ambiguous ("may be penalized")

>>>> test instruction?


>>> not that i am aware of; but then i have not done a lit search either.

>>>> (This particular instruction was used on a speeded (clerical speed)

>>>> portion of a longer test for a craft type job.)

>>> for what it is worth, it is common, if not stand practice to correct

>>> these types of tests for accuracy as well.


>>> hope this is somewhat useful.


>>> thanks.


>>> - winfred

> _______________________________________________________

> IPAC-List

> IPAC-List at ipacweb.org

> http://www.ipacweb.org/mailman/listinfo/ipac-list




More information about the IPAC-List mailing list