[IPAC-List] Ricci Update Prompts a Question

[John Ford]

I appreciate the perspectives on the Ricci case and on best practices with
respect to adverse impact.  They raise a question in my mind that I would
appreciate perspective on from experienced selection folks.

Suppose that you adopt targeted recruiting procedures with respect to an
underrepresented minority group.  Could the following happen?  And if so,
how should it be dealt with?

--SCENARIO--

Government agency X announces that it is concerned about underrepresentation
of minority group Y in its workforce.  They adopt a number of measures to
reach out to Y applicants, including placing something in their job
announcements like "Qualified Y applicants are encouraged to apply."  

This has a subtle effect on the applicant pool.  Before the targeted
recruitment, self-selection among applicants resulted in an ability
distribution around the assessment cut score that is equivalent for all
subgroups.  After the targeted recruitment policy is announced, this
changes.  Nonminority applicants who perceive themselves as barely qualified
self-select out in greater numbers because they believe the policy reduces
their chances.  Minority Y applicants who are marginally qualified apply in
greater numbers because they believe the policy increases their chances.  An
equivalent number of well-qualified applicants from all groups still apply,
giving the agency a good, diverse pool from which to select one or two top
applicants.  But the assessment seems to have adverse impact because it
passes fewer minority Y applicants overall.  It is seen as a biased and
inappropriate assessment.

--END SCENARIO--

My concern is that this can happen even with a valid assessment that under
reasonable circumstances would not have adverse impact.  It appears to
because awareness of the policy by the applicant pool, and their
understandable response to the policy, can create an applicant pool with
different ability distributions among nonminority and Y applicants.  Thus
will likely be seen as a fault in the assessment procedure rather than as a
result of applicant response to the recruitment policy.

So, do other assessment practitioners agree that this can happen?  If so,
how could we reasonably discriminate this situation from one in which there
is a biased assessment?  Or is this not a distinction we would care to make
because we hold to a definition of bias that sees it as present whenever
there is differential impact on demographic subgroups?

Your responses are appreciated.

John Ford
Research Psychologist
U.S. Merit Systems Protection Board