[IPAC-List] Threatening a Penalty for Guessing

Winfred Arthur, Jr. w-arthur at neo.tamu.edu
Tue Apr 27 22:43:17 EDT 2010

no, Joel, sorry for being unclear but i did not mean to suggest or imply
that it is standard practice to correct both speeded and non-speeded
tests for guessing/accuracy. however, to echo Dennis' point, we do not
correct our knowledge tests for guessing -- this is primarily b/c there
is not a strong conceptual case or basis for doing so. [although, i can
envisage some domains in which errors and lack of knowledge might be so
dangerous that you might want to penalize test takers for getting an
answer wrong!! :) ]

anyway, if you are interested in some reading on this, you might want to
look at the "scoring of information-processing tests" section (pp.
60-61) of:

Arthur, W. Jr., Doverspike, D., & Bell, S. T. (2004). Information
processing tests. In M. Hersen, & J. C. Thomas (Eds.), /Comprehensive
Handbook of Psychological Assessment: Volume 4, Industrial and
Organizational Assessment/ (pp. 56-74). NY: John Wiley & Sons, Inc.

it discusses the different ways in which tests (like your clerical speed
test?) could be scored, namely, (1) # of correct responses [accuracy],
(2) response time [speed], or (3) some weighted composite of accuracy
and speed using specified algorithms.

- winfred

On 4/27/2010 11:13 AM, Joel Wiesen wrote:

> Thanks for your thoughtful comments, Winfred.


> Do I understand you correctly as saying it is standard practice to

> correct both speeded and non-speeded test for guessing, but that you

> do not do so for your (non-speeded) knowledge tests?


> If so, why do you not correct for guessing?


> Thx.


> Joel




> - -

> Joel P. Wiesen, Ph.D., Director

> Applied Personnel Research

> 62 Candlewood Road

> Scarsdale, NY 10583-6040

> (617) 244-8859

> http://appliedpersonnelresearch.com





> Note: This e-mail and any attachments may contain confidential and/or

> legally privileged information. Please do not forward any contents

> without permission. If you have received this message in error please

> destroy all copies, completely remove it from your computer, and

> notify the sender. Thank you.




> Winfred Arthur, Jr. wrote:

>> Joel, some general thoughts are embedded below:


>> On 4/27/2010 8:19 AM, Joel Wiesen wrote:

>>> If a test's instructions say "you may be penalized for questions you

>>> mark incorrectly" and then the grading does not correct for guessing,

>>> what might the effect be?

>> well, my first thought is that the use of "may" makes this

>> problematic. it shld state explicitly whether one is going to do so

>> or not. "may" without specifying the conditions under which this

>> will or will not be invoked seems to me to be a recipe for . . .

>> well, problems!


>> and whereas i have not seen any empirical rsch or data on this, the

>> college board uses this instruction set for some sections of the SAT

>> [they do not use "may"; they use "will"] and it is my impression that

>> students are more likely to leave these items blank than guess when

>> they do not know the answer.



>>> Has anyone had practical experience with such instructions? Do test

>>> takers pay attention to such instructions?

>> not personally. indeed to the contrary, i use an instruction set

>> that states that there is no penalty for guessing and so it is in

>> one's best interest to guess if one does not know the answer.

>> subsequently, i rarely get any non-responses. of course, these are

>> knowledge tests.


>>> Is there research on this type of ambiguous ("may be penalized")

>>> test instruction?


>> not that i am aware of; but then i have not done a lit search either.

>>> (This particular instruction was used on a speeded (clerical speed)

>>> portion of a longer test for a craft type job.)

>> for what it is worth, it is common, if not stand practice to correct

>> these types of tests for accuracy as well.


>> hope this is somewhat useful.


>> thanks.


>> - winfred

More information about the IPAC-List mailing list