[IPAC-List] Paradox in the 1999 APA Standards

[Dennis Doverspike]

Pat

Both your comments and Joel comments are true - although some recent
research questions the general finding of a "fair regression" line. So how
can both your and Joel's comments be true? That is why it is a paradox.

Basically, there are many theories of fairness. But two predominant ones are
Cleary and Thorndike. Cleary comes to the conclusions you describe,
Thorndike comes to the conclusions that Joel describes. So how can that be -
well to make a long story short - it is because we define a best solution as
the one that minimizes least squared errors, and it is that definition of
best that leads to the paradox.

This is not a new finding, or unique to cognitive ability testing, it played
a big role in the pay equity debate. The paradox is often described as
Birnbaum's or even Galton's, showing how far back it goes - basically to the
discovery of the correlation coefficient and of associated prediction. 

One place to look for a discussion is the National Academy of Sciences study
related to the GATB controversy. I am sure someone can fill in the correct
reference for me here as I am swamped today. 

So Pat, basically your conclusion, while logical, is of course wrong. And
thus the paradox. In part it is because when we are talking about the number
of false positives, we are really talking about reverse regression. Thus,
another name, the reverse regression paradox. 

So minorities can have lower performance scores than predicted, but also be
overrepresented among those who would have been successful on the job but
were not selected. Women can receive less pay than predicted for the same
merit, but for the same pay have less merit. And we could go on with many
examples. 

Dennis Doverspike, Ph.D., ABPP
Professor of Psychology
Director, Center for Organizational Research
Senior Fellow of the Institute for Life-Span Development and Gerontology
Psychology Department
University of Akron
Akron, Ohio  44325-4301
330-972-8372 (Office)
330-972-5174 (Office Fax)
ddoverspike at uakron.edu

The information is intended only for the person or entity to which it is
addressed and may contain confidential, privileged and/or a work product for
the sole use of the intended recipient.  No confidentiality or privilege is
waived or lost by any errant transmission. If you receive this message in
error, please destroy all copies of it and notify the sender. If the reader
of this message is not the intended recipient, you are hereby notified that
any dissemination, distribution or copying of this communication is strictly
prohibited. In the case of E-mail or electronic transmission, immediately
delete it and all copies of it from your system and notify the sender.
E-mail and fax transmission cannot be guaranteed to be secure or error-free
as information could be intercepted, corrupted, lost, destroyed, arrive late
or incomplete, or contain viruses. 

-----Original Message-----
From: ipac-list-bounces at ipacweb.org [mailto:ipac-list-bounces at ipacweb.org]
On Behalf Of Patrick McCoy
Sent: Friday, May 28, 2010 11:13 AM
To: Joel Wiesen; IPAC-List at ipacweb.org
Subject: Re: [IPAC-List] Paradox in the 1999 APA Standards

Hello Joel:

That quote is a bit surprising.  Quite a bit of research used to show that
with one regression line the performance of minorities is actually often
"over predicted" rather than under predicted.  That is, minorities perform
worse in school or on the job than the equation would suggest they would
(which would imply more false positives for minorities; right?).

Two relevant references are:

Zwick, R. (2002). Fair game?: The use of standardized admission tests in
higher education. NY: Routledge Falmer; Chapter 5 might be most useful

Sackett,P.R., & Wilk,S.L. (1994). Within-group norming and other forms of
score adjustment in preemployment testing. American Psychologist, 49(11),
929-954

Good luck!  This is likely a difficult area.

Pat McCoy,Ph.D.
Ottawa, Canada





>>> Joel Wiesen <jwiesen at appliedpersonnelresearch.com> 5/27/2010 7:09 pm >>>

A paradox that is seen in selection applications is described in the APA 
Standards as follows:

"... a given selection score and criterion threshold will often result 
in proportionately more false negative decisions in groups with lower 
mean test scores. In other words, a lower-scoring group will usually 
have a higher proportion of examinees who are rejected on the basis of 
their test scores even though they would have performed successfully if 
they had been selected. This seeming paradox is a statistical 
consequence of the imperfect correlation between test and criterion."
(AERA, APA, NCME, 1999, page 79, col 2.)

The Standards do not provide a literature citation for this statement.
Can anyone point me to published literature on this topic?

Thx.

Joel


-- 
Joel P. Wiesen, Ph.D., Director
Applied Personnel Research
62 Candlewood Road
Scarsdale, NY 10583-6040
(617) 244-8859
http://appliedpersonnelresearch.com 




Note: This e-mail and any attachments may contain confidential and/or 
legally privileged information. Please do not forward any contents 
without permission. If you have received this message in error please 
destroy all copies, completely remove it from your computer, and notify 
the sender. Thank you.

_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org 
http://www.ipacweb.org/mailman/listinfo/ipac-list

_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
http://www.ipacweb.org/mailman/listinfo/ipac-list