[IPAC-List] Ricci Update Prompts a Question

[Dennis Doverspike]

John,

Your scenario is excellent because it points out that adverse impact is both
situationally and sample (applicant group characteristics) specific. Adverse
impact is not wholly a result of the test and in many cases with small
sample sizes may have little to do with the underlying characteristics of
the test.

Of course, this then results in a situation, as occurred recently, where the
almost identically test can have adverse impact against Blacks as compared
to Whites and also against Whites as compared to Blacks in slightly
different promotional situations. So, adverse impact has no necessary
linkage to any property of the test. In addition, it is very difficult, if
not impossible, to predict ahead of time if a test will have adverse impact,
unless we know that so many people will be hired or promoted that it will
not be an issue.

One could argue that is because adverse impact is basically a legal
gatekeeper and has very little to do with assessment science. 

Dennis Doverspike, Ph.D., ABPP
Professor of Psychology
Director, Center for Organizational Research
Senior Fellow of the Institute for Life-Span Development and Gerontology
Psychology Department
University of Akron
Akron, Ohio  44325-4301
330-972-8372 (Office)
330-972-5174 (Office Fax)
ddoverspike at uakron.edu



-----Original Message-----
From: ipac-list-bounces at ipacweb.org [mailto:ipac-list-bounces at ipacweb.org]
On Behalf Of John Ford
Sent: Wednesday, December 09, 2009 7:32 PM
To: IPAC-List at ipacweb.org
Subject: [IPAC-List] Ricci Update Prompts a Question

I appreciate the perspectives on the Ricci case and on best practices with
respect to adverse impact.  They raise a question in my mind that I would
appreciate perspective on from experienced selection folks.

Suppose that you adopt targeted recruiting procedures with respect to an
underrepresented minority group.  Could the following happen?  And if so,
how should it be dealt with?

--SCENARIO--

Government agency X announces that it is concerned about underrepresentation
of minority group Y in its workforce.  They adopt a number of measures to
reach out to Y applicants, including placing something in their job
announcements like "Qualified Y applicants are encouraged to apply."  

This has a subtle effect on the applicant pool.  Before the targeted
recruitment, self-selection among applicants resulted in an ability
distribution around the assessment cut score that is equivalent for all
subgroups.  After the targeted recruitment policy is announced, this
changes.  Nonminority applicants who perceive themselves as barely qualified
self-select out in greater numbers because they believe the policy reduces
their chances.  Minority Y applicants who are marginally qualified apply in
greater numbers because they believe the policy increases their chances.  An
equivalent number of well-qualified applicants from all groups still apply,
giving the agency a good, diverse pool from which to select one or two top
applicants.  But the assessment seems to have adverse impact because it
passes fewer minority Y applicants overall.  It is seen as a biased and
inappropriate assessment.

--END SCENARIO--

My concern is that this can happen even with a valid assessment that under
reasonable circumstances would not have adverse impact.  It appears to
because awareness of the policy by the applicant pool, and their
understandable response to the policy, can create an applicant pool with
different ability distributions among nonminority and Y applicants.  Thus
will likely be seen as a fault in the assessment procedure rather than as a
result of applicant response to the recruitment policy.

So, do other assessment practitioners agree that this can happen?  If so,
how could we reasonably discriminate this situation from one in which there
is a biased assessment?  Or is this not a distinction we would care to make
because we hold to a definition of bias that sees it as present whenever
there is differential impact on demographic subgroups?

Your responses are appreciated.

John Ford
Research Psychologist
U.S. Merit Systems Protection Board

_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
http://www.ipacweb.org/mailman/listinfo/ipac-list