[IPAC-List] Threatening a Penalty for Guessing

[Winfred Arthur, Jr.]

no, Joel, sorry for being unclear but i did not mean to suggest or imply 
that it is standard practice to correct both speeded and non-speeded 
tests for guessing/accuracy.  however, to echo Dennis' point, we do not 
correct our knowledge tests for guessing -- this is primarily b/c there 
is not a strong conceptual case or basis for doing so.  [although, i can 
envisage some domains in which errors and lack of knowledge might be so 
dangerous that you might want to penalize test takers for getting an 
answer wrong!!  :)  ]

anyway, if you are interested in some reading on this, you might want to 
look at the "scoring of information-processing tests" section (pp. 
60-61) of:

Arthur, W. Jr., Doverspike, D., & Bell, S. T. (2004).  Information 
processing tests.  In M. Hersen, & J. C. Thomas (Eds.), /Comprehensive 
Handbook of Psychological Assessment: Volume 4, Industrial and 
Organizational Assessment/ (pp. 56-74).  NY: John Wiley & Sons, Inc.

it discusses the different ways in which tests (like your clerical speed 
test?) could be scored, namely, (1) # of correct responses [accuracy], 
(2) response time [speed], or (3) some weighted composite of accuracy 
and speed using specified algorithms.

- winfred

On 4/27/2010 11:13 AM, Joel Wiesen wrote:

> Thanks for your thoughtful comments, Winfred.

>

> Do I understand you correctly as saying it is standard practice to 

> correct both speeded and non-speeded test for guessing, but that you 

> do not do so for your (non-speeded) knowledge tests?

>

> If so, why do you not correct for guessing?

>

> Thx.

>

> Joel

>

>

>

> - -

> Joel P. Wiesen, Ph.D., Director

> Applied Personnel Research

> 62 Candlewood Road

> Scarsdale, NY 10583-6040

> (617) 244-8859

> http://appliedpersonnelresearch.com

>

>

>

>

> Note: This e-mail and any attachments may contain confidential and/or 

> legally privileged information. Please do not forward any contents 

> without permission. If you have received this message in error please 

> destroy all copies, completely remove it from your computer, and 

> notify the sender. Thank you.

>

>

>

> Winfred Arthur, Jr. wrote:

>> Joel, some general thoughts are embedded below:

>>

>> On 4/27/2010 8:19 AM, Joel Wiesen wrote:

>>> If a test's instructions say "you may be penalized for questions you

>>> mark incorrectly" and then the grading does not correct for guessing,

>>> what might the effect be?

>> well, my first thought is that the use of "may" makes this 

>> problematic.  it shld state explicitly whether one is going to do so 

>> or not.  "may" without specifying the conditions under which this 

>> will or will not be invoked seems to me to be a recipe for . . . 

>> well, problems!

>>

>> and whereas i have not seen any empirical rsch or data on this, the 

>> college board uses this instruction set for some sections of the SAT 

>> [they do not use "may"; they use "will"] and it is my impression that 

>> students are more likely to leave these items blank than guess when 

>> they do not know the answer.

>>

>>>

>>> Has anyone had practical experience with such instructions? Do test 

>>> takers pay attention to such instructions?

>> not personally.  indeed to the contrary, i use an instruction set 

>> that states that there is no penalty for guessing and so it is in 

>> one's best interest to guess if one does not know the answer.  

>> subsequently, i rarely get any non-responses.  of course, these are 

>> knowledge tests.

>>>

>>> Is there research on this type of ambiguous ("may be penalized") 

>>> test instruction?

>>>

>> not that i am aware of; but then i have not done a lit search either.

>>> (This particular instruction was used on a speeded (clerical speed)

>>> portion of a longer test for a craft type job.)

>> for what it is worth, it is common, if not stand practice to correct 

>> these types of tests for accuracy as well.

>>

>> hope this is somewhat useful.

>>

>> thanks.

>>

>> - winfred