[IPAC-List] Pass points for employment tests and the Standardsforeducati...
jkuthy at biddle.com
Thu Jul 29 17:30:07 EDT 2010
By suggesting that the highest scoring persons on the list should be chosen first, you assume that you have sufficient validity and/or reliability to "rank order" your job candidates.
Some have suggested a reliability coefficient of 0.85 to 0.90 for using test results in strict rank order (e.g., Aiken, L. R. . Psychological Testing and Assessment (2nd ed.; p. 100)). If the test is not sufficiently reliable, you should not separate candidates based on very small score differences.
For content-related validity, Section 14C(9) of the Uniform Guidelines specifies, "If a user can show, by a job analysis or otherwise, that a higher score on a content valid selection procedure is likely to result in better job performance, the results may be used to rank persons who score above minimum levels. Where a selection procedure supported solely or primarily by content validity is used to rank job candidates, the selection procedure should measure those aspects of performance which differentiate among levels of job performance." So, there are standards for ranking job candidates for tests that are content valid.
Section 15C(7) states, " If the selection procedure is used with a cutoff score, the user should describe the way in which normal expectations of proficiency within the work force were determined and the way in which the cutoff score was determined (essential). In addition, if the selection procedure is to be used for ranking, the user should specify the evidence showing that a higher score on the selection procedure is likely to result in better job performance."
For criterion-related validity, several court cases have indicated that validity coefficients of 0.30 or higher are required if the list you wish to rank from has adverse impact (e.g., Brunet v. City of Columbus, 1993; Boston Chapter, NAACP Inc. v. Beecher, 1974; Clady v. County of Los Angeles, 1985; and Zamlen v. City of Cleveland, 1988).
Also, Section 15B(10) of the Uniform Guidelines states, "If the selection procedure is used with a cutoff score, the user should describe the way in which normal expectations of proficiency within the work force were determined and the way in which the cutoff score was determined (essential)." So, there are standards for setting cutoffs when using criterion-related validity also.
Jim Kuthy, Ph.D.
Biddle Consulting Group, Inc.
800-999-0438 x 239
jkuthy at biddle.com
From: RPClare at aol.com [mailto:RPClare at aol.com]
Sent: Wednesday, July 28, 2010 2:01 PM
To: Bryan.Baldwin at doj.ca.gov; ipac-list at ipacweb.org; Rene.Shekerjian at cs.state.ny.us
Subject: Re: [IPAC-List] Pass points for employment tests and the Standardsforeducati...
This has always been a thorny issue. In the absence of any union issues or
other binding "list use" rules, there is no reason to set a pass point. If
1000 folks take a "test" and there are 14 vacancies the last 986 folks
don't have a chance of getting a job. Other factors such as "rule of...", "life of the list" and unavailability of one or more of the "reachable" folks shape what that final "reachable" number is. The issue of a
non-discriminatory list (test result) rings hollow if the protected groups are in the bottom half of the list and have no realistic chance of getting a job. Regardless of what we call it, the group who have a reasonable chance of getting a position is the "true" list no matter how it may be identified. There is no more "job related" factor than the number of people having a reasonable chance of appointment.
In a message dated 7/28/2010 3:41:49 P.M. Eastern Daylight Time,
Bryan.Baldwin at doj.ca.gov writes:
The key to your question (or is it the answer) lies with "in the absence
of factors that would suggest discriminatory effect or intent." To some
extent then this is the age-old "if a tree falls in the forest..."
question, but looking at cases that have been brought against employers
(e.g., SEPTA, City of Chicago), courts have not looked particularly
fondly upon employers that have set cut-scores based purely on
Can you set pass points based on number of positions to fill? Sure.
You could also set it based on the fact that 85 is your favorite number.
Is this a problem? Probably, from purely a utility perspective if it
wasn't set in a job-related fashion. Could it be a problem legally?
Absolutely, if it has adverse impact and you can't provide the related
Staff Services Manager II
California Department of Justice
Division of Administrative Support
>>> "Shekerjian, Rene" <Rene.Shekerjian at cs.state.ny.us> 7/28/2010 11:58
The Standards seem to say that pass points can be set for employment
tests based solely on the number of positions to be filled (4.19 and
4.20). Based on things I have heard and read, I have the feeling that
that is not how things have played out over the years.
Is it really that simple for employment tests? (Gets ready to duck.) Of
course, I mean in the absence of factors that would suggest
discriminatory effect or intent.
I would appreciate any thoughts you would care to share on this issue.
René Shekerjian | Testing Services Division | NYS Department of Civil
IPAC-List at ipacweb.org
CONFIDENTIALITY NOTICE: This communication with its contents may contain
confidential and/or legally privileged information. It is solely for the
use of the intended recipient(s). Unauthorized interception, review, use
or disclosure is prohibited and may violate applicable laws including
the Electronic Communications Privacy Act. If you are not the intended
recipient, please contact the sender and destroy all copies of the
IPAC-List at ipacweb.org
More information about the IPAC-List