[IPAC-List] Norms

[Dennis Doverspike]

Patrick,

You really have two or three different issues here.

1. First with regard to the general issue of retest effects, there is quite
a bit of literature, I might start with a study by Paul Sackett

Lievens, F., Buyse, T., & Sackett, P. R. (2005). Retest effects in
operational selection settings: Development and test of a framework. *Personnel
Psychology*, *58*(4), 981-1007.


>From the abstract


This study proposes a framework for examining the effects of retaking tests
in operational selection settings. A central feature of this framework is
the distinction between within-person and between-person retest effects.
This framework is used to develop hypotheses about retest effects for
exemplars of 3 types of tests (knowledge tests, cognitive ability tests,
and situational judgment tests) and to test these hypotheses in a high
stakes selection setting (admission to medical studies in Belgium).
Analyses of within-person retest effects showed that mean scores of repeat
test takers were one-third of a standard deviation higher for the knowledge
test and situational judgment test and one-half of a standard deviation
higher for the cognitive ability test.

I would probably disagree with Fred on one thing he said, in that again I
believe research by Sackett and colleagues shows that there are substantial
gains on the retest with personality test scores, especially when the
person believes or knows that is what they failed the first time. Not sure
if all that research is published.

So, your assumption that the results for a test and a retest would be
similar seems unlikely.

2. Second,

I guess this is the question of what are you doing - are  you really
norming or are you looking at the results for something like EEOC purposes.
If for EEOC purposes, then I believe you have received a number of good
answers. If for norming, then I think that is a different issue. There is a
science of norming, similar to one for survey sampling. If you are really
generating norms, then you would want some sort of sampling strategy. It
would be a strange sampling strategy that would include people more than
once.

Dennis


On Wed, Jan 8, 2014 at 12:49 PM, Patrick McCoy
<Patrick.McCoy at psc-cfp.gc.ca>wrote:


>  I am currently running off some normative data on a test (means and

> frequency distributions) to see how people are doing on it and to see what

> proportion pass at various cutoff scores.

>

> My question concerns how to handle the issue of candidates taking the same

> test more than once.  One option would be to use only the first test

> score.  Another would be to use all scores.  A third option would be to use

> only the most recent score.

>

> My tendency would be to use the most recent score or all scores, and I

> suspect that the results from these two approaches will be quite similar in

> most instances.  Does that make sense or am I missing something?

>

> Your thinking on this would be appreciated!

>

>

> Pat McCoy

> Ottawa Canada

>

>

>

>

> >

> This e-mail message is intended for the named recipient(s) and may contain

> information that is privileged, confidential and/or exempt from disclosure

> under applicable law. Unauthorized disclosure, copying or re-transmission

> is prohibited. If you are not a named recipient or not authorized by the

> named recipient(s), or if you have received this e-mail in error, then

> please notify the sender immediately and delete the message and any copies.

> >

> Ce courriel est destiné exclusivement au destinataire mentionné en titre

> et peut contenir de l'information privilégiée, confidentielle ou soustraite

> à la communication aux termes des lois applicables. Toute divulgation non

> autorisée, toute reproduction ou réacheminement est interdit. Si vous

> n'êtes pas le destinataire de ce courriel, ou n'êtes pas autorisé par le

> destinataire visé, ou encore, si vous l'avez reçu par erreur, veuillez le

> mentionner immédiatement à l'expéditeur et supprimer le courriel et les

> copies.

>

> _______________________________________________________

> IPAC-List

> IPAC-List at ipacweb.org

> http://nine.pairlist.net/mailman/listinfo/ipac-list

>

>



-- 
Dennis Doverspike, PhD., ABPP
Licensed Psychologist, #3539 (OHIO)
Independent Consultant
Professor of Psychology, University of Akron
dennisdoverspike at gmail.com

The information is intended only for the person or entity to which it is
addressed and may contain confidential, privileged and/or a work product
for the sole use of the intended recipient. No confidentiality or privilege
is waived or lost by any errant transmission. If you receive this message
in error, please destroy all copies of it and notify the sender. If the
reader of this message is not the intended recipient, you are hereby
notified that any dissemination, distribution or copying of this
communication is strictly prohibited. In the case of E-mail or electronic
transmission, immediately delete it and all copies of it from your system
and notify the sender. E-mail and fax transmission cannot be guaranteed to
be secure or error-free as information could be intercepted, corrupted,
lost, destroyed, arrive late or incomplete, or contain viruses.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://nine.pairlist.net/pipermail/ipac-list/attachments/20140108/00d08d54/attachment.htm