[IPAC-List] Pass points for employment tests and the Standardsforeducati...

[Jim Kuthy]

By suggesting that the highest scoring persons on the list should be chosen first, you assume that you have sufficient validity and/or reliability to "rank order" your job candidates.

Some have suggested a reliability coefficient of 0.85 to 0.90 for using test results in strict rank order (e.g., Aiken, L. R. [1988]. Psychological Testing and Assessment (2nd ed.; p. 100)). If the test is not sufficiently reliable, you should not separate candidates based on very small score differences.

For content-related validity, Section 14C(9) of the Uniform Guidelines specifies, "If a user can show, by a job analysis or otherwise, that a higher score on a content valid selection procedure is likely to result in better job performance, the results may be used to rank persons who score above minimum levels. Where a selection procedure supported solely or primarily by content validity is used to rank job candidates, the selection procedure should measure those aspects of performance which differentiate among levels of job performance." So, there are standards for ranking job candidates for tests that are content valid. 

Section 15C(7) states, " If the selection procedure is used with a cutoff score, the user should describe the way in which normal expectations of proficiency within the work force were determined and the way in which the cutoff score was determined (essential). In addition, if the selection procedure is to be used for ranking, the user should specify the evidence showing that a higher score on the selection procedure is likely to result in better job performance."

For criterion-related validity, several court cases have indicated that validity coefficients of 0.30 or higher are required if the list you wish to rank from has adverse impact (e.g., Brunet v. City of Columbus, 1993; Boston Chapter, NAACP Inc. v. Beecher, 1974; Clady v. County of Los Angeles, 1985; and Zamlen v. City of Cleveland, 1988).

Also, Section 15B(10) of the Uniform Guidelines states, "If the selection procedure is used with a cutoff score, the user should describe the way in which normal expectations of proficiency within the work force were determined and the way in which the cutoff score was determined (essential)." So, there are standards for setting cutoffs when using criterion-related validity also.

Jim Kuthy, Ph.D.
Biddle Consulting Group, Inc.
800-999-0438 x 239
jkuthy at biddle.com

-----Original Message-----
From: RPClare at aol.com [mailto:RPClare at aol.com] 
Sent: Wednesday, July 28, 2010 2:01 PM
To: Bryan.Baldwin at doj.ca.gov; ipac-list at ipacweb.org; Rene.Shekerjian at cs.state.ny.us
Subject: Re: [IPAC-List] Pass points for employment tests and the Standardsforeducati...

This has always been a thorny issue. In the absence of any  union issues or 
other binding "list use" rules, there is no reason to set a pass  point. If 
1000 folks take a "test" and there are 14 vacancies the last 986 folks  
don't have a chance of getting a job. Other factors such as "rule of...", "life of the list" and unavailability of one or more of the "reachable" folks   shape what that final "reachable" number is. The issue of a 
non-discriminatory list (test result) rings hollow if the protected groups are in the bottom  half  of the list and have no realistic chance of getting a job. Regardless of what we call it, the group who have a reasonable chance of getting a position is the "true" list no matter how it may be identified. There is no more "job related" factor than the number of people having a reasonable chance of appointment.
 
 
In a message dated 7/28/2010 3:41:49 P.M. Eastern Daylight Time,  
Bryan.Baldwin at doj.ca.gov writes:

The key  to your question (or is it the answer) lies with "in the absence
of factors  that would suggest discriminatory effect or intent."  To some
extent  then this is the age-old "if a tree falls in the forest..."
question, but  looking at cases that have been brought against employers
(e.g., SEPTA,  City of Chicago), courts have not looked particularly
fondly upon employers  that have set cut-scores based purely on
administrative  convenience.

Can you set pass points based on number of positions to  fill?  Sure. 
You could also set it based on the fact that 85 is your  favorite number.
Is this a problem?  Probably, from purely a utility  perspective if it
wasn't set in a job-related fashion.  Could it be a  problem legally? 
Absolutely, if it has adverse impact and you can't  provide the related
evidence.


Bryan Baldwin
Staff Services  Manager II
California Department of Justice
Division of Administrative  Support
Personnel Programs
(916) 322-5446


>>>  "Shekerjian, Rene" <Rene.Shekerjian at cs.state.ny.us> 7/28/2010  11:58

AM >>>
The Standards seem to say that pass points can be  set for employment
tests based solely on the number of positions to be  filled (4.19 and
4.20). Based on things I have heard and read, I have the  feeling that
that is not how things have played out over the years.  

Is it really that simple for employment tests? (Gets ready to duck.)  Of
course, I mean in the absence of factors that would  suggest
discriminatory effect or intent.

I would appreciate any  thoughts you would care to share on this  issue.

Thanks,

René

René Shekerjian | Testing Services  Division | NYS Department of Civil
Service  
==============================================================

_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org  
http://www.ipacweb.org/mailman/listinfo/ipac-list


CONFIDENTIALITY  NOTICE: This communication with its contents may contain
confidential  and/or legally privileged information. It is solely for the
use of the  intended recipient(s). Unauthorized interception, review, use
or disclosure  is prohibited and may violate applicable laws including
the Electronic  Communications Privacy Act. If you are not the intended
recipient, please  contact the sender and destroy all copies of  the
communication.
_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
http://www.ipacweb.org/mailman/listinfo/ipac-list