[IPAC-List] Cert rule

[Mark Hammer]

1) While I have no doubt that multiple-choice tests can introduce enough adverse impact into the mix to cause difficulties when used as an initial screen, there are reasons to rethink HOW one uses it rather than WHETHER to use it.  A decade ago, I assisted on a recruitment activity, and the manager of the program was very reticent to use a cognitive ability test as a screen, precisely because of apprehension about adverse impact.  Instead, they elected to use a simulation exercise (our K-Mart version of an assessment center) for all 417 people screened in on T&E.  The simulation exercise is normally used when one has arrived at a short list, simply because it is very labour intensive.  Because the candidates were essentially unfiltered, the assessors got to see many third-rate candidates that they would not normally see using that tool.  In the end, we saw *more* adverse impact with it than we normally saw with a multiple-choice CAT.  Partly, I suspect, because of contrast effects.  In retrospect, they would have had less headaches using the CAT with a lower pass mark, if only to make the pool of people to be assessed with the more labour-intensive tool just a little more select.

2) In the Canadian federal context, our current legislation shifted the definition of merit from relative to individual.  Although managers can still use top-down, they don't have to.  The appointee, however, MUST always be qualified, and demonstrably a "good fit".  That "fit" can be achieved when the hiring manager includes what we define as "asset qualifications" and "organizational needs".  Asset qualifications might include experience in specific areas or with specific processes/materials/methods, or second languages, or EE membership.  Organizational needs might include things like willingness to travel, or do shift work.  So, in effect, the competition can result in a sort of banding exercise with two resulting bands (pass, not pass).  The manager has laid out documented a priori considerations (in the poster) on top of the essential qualifications.  Again, the legislation still permits managers to use top-down; it just does not require it, and provides a framework for other approaches.  It does, however, impose upon the manager to have a clear idea of what "qualified" looks like and consists of, since anyone who makes the cut (not just those who are defined as "best" by test scores) *could* ultimately be the hire.

Of course, identifying a "good fit" is probably easier to do when the process is aimed at one or two hires, and substantially harder to do when it is geared towards recruiting 80.  My gut sense is that when the list is small, and there are more qualified people than positions, managers weigh off the various asset and organizational considerations against the essential qualifications of the candidates, and make reasoned judgments.  For instance, they may think about work-team composition.  When it's a cattle call, my sense is that they probably let the tests do the heavy lifting.  Now, where the dividing line is, or should be, between the two approaches, is anybody's guess.

In retrospect, it seems that the New Haven decision is one borne of the **expectation** that top-down WAS the basis for decision, would remain so, and was THE legitimate basis for selection.  The approach we adopted at the federal level in Canada, despite its warts and initial hiccups, permits the employer to ignore the impact (up to a point) of adverse impact on tests, and make a selection based on total fit (and that *could* include things like developing more diversity in the internal feeder groups for subsequent leadership positions).  Key to this is the fact that this approach is declared UP FRONT, and is not a post hoc "adjustment" like the one that New Haven made.  Their approach to staffing is one that would have  worked seamlessly within our current legislation here.  Their sin was not that they wanted to increase diversity; rather it was that they changed the rules after the game was already in play.  And that, sayeth regular folks and the Supreme Court, is a no-no.

Mark Hammer
Ottawa


>>> "Pluta, Paul" <ppluta at hr.lacounty.gov> 2009/07/27 10:12 am >>>

Dennis,

I don't believe I indicated that the City of New Haven decision says anything about top-down versus banding per se.  I simply noted that those two cases (i.e., New York and New Haven) both used top-down selection (i.e., rule of three or rule of five, etc.) and both had a disparate impact on minorities.  There is a huge literature to support the fact that a test of cognitive abilities will almost assuredly result in adverse impact for blacks and Hispanics.

"Had the City reviewed the exam results and then [emphasis added] adopted banding to make the minority test scores appear higher, it would have violated Title VII's prohibition of adjusting test results on the basis of race."  This statement appears to pertain to post hoc banding to achieve some race-based objective.  Our banding process is routine and has nothing to do with race per se.  We simply acknowledge measurement error and feel it is best in high stakes testing to error on the side of the individual.

The real bottom-line is, if you don't want adverse impact, don't use a written test of cognitive abilities (especially not as a first hurdle in a multiple-hurdle process).  If you must use a test of cognitive abilities because that is what the job analysis study results indicate is important for the performance of the work, be sure that gets thoroughly documented.  Based on the New York case, that goes double for documenting the linkages between the KSAs and the tasks performed on the job.  At least then you can defend the job-relatedness and business necessity of the test.  IMHO

Paul E. Pluta, ABD, SPHR

Human Resources Analyst III

Los Angeles County Department of Human Resources

Workforce Planning, Test Research, & Appeals Division