[IPAC-List] Reliability for T&Es

[RPClare at aol.com]

I can't help offering a rambling contribution to this  discussion. We 
discuss T&E as if it were a single entity when is truly is a  widely divergent 
practice applied by practitioners with vastly different  competencies to 
highly diverse jobs. From the unions perspective, the tighter  the rule of list 
(or seniority) the less management has the opportunity to "play  favorites" 
with its members and the fewer challenges they get ("they appointed  the top 
scorer as required by rule/contract"). Whether our selection tool ranks  or 
not, the appointing authority "ranks", often on criteria unrelated to job  
performance.
If the "job" is clearly defined (often it is not), the  (job-related) 
criteria clearly defined in writing, the raters well trained,  random samples 
cross rated to check consistency, and the entire process  communicated 
effectively. We are likely to get relatively high reliability. From  a PR stand 
point, lack of reliability/consistency will destroy the process  internally.
Many public sector jobs are in flux as we "downsize", broaden  functions, 
eliminate levels and outsource. Our job descriptions usually don't  reflect 
that because they are static documents in a rapidly changing  environment. If 
the jobs don't have reliability, the instrument can't have it. 
 
In a message dated 6/22/2010 11:31:13 A.M. Eastern Daylight Time,  
Bryan.Baldwin at doj.ca.gov writes:

I would  actually be perfectly okay with setting these exams up at 
pass/fail rather  than get into banding at all.  Unfortunately civil service rules 
don't  always allow us to do what we want!  Additionally, when we've tried to 
 establish these as pass/fail the unions have argued this degrades merit.   
Kinda funny/sad that it's the change to T&Es that degrades merit, not what  
you do with the scores.


>>> "Winfred Arthur, Jr."  <w-arthur at neo.tamu.edu> 6/21/2010 8:36 PM >>>

interestingly,  Dennis' position is consonant with the argument to band 
on the (predicted)  _*criterion*_ score instead of the predictor score.  
ah banding   :)

- winfred

On 6/21/2010 10:27 PM, Doverspike,Dennis  wrote:

> This suggests I believe the logical problem with banding,  especially 

based on reliability. Beyond the question of the psychometric  basis. If you 
are really serious about it, why band based upon reliability,  why not band 
based upon validity? Then, if we know the validity of T&E to  be extremely 
low, the band range would be huge. Probably as wide as the score  range. As a 
purely hypothetical question, if a test had perfect reliability  but zero 
relationship with merit, what should the band be?

>  ________________________________________

> From:  ipac-list-bounces at ipacweb.org [ipac-list-bounces at ipacweb.org] On 

Behalf Of  Joel Wiesen [jwiesen at appliedpersonnelresearch.com] 

> Sent: Monday, June  21, 2010 7:24 PM

> To: Bryan Baldwin

> Cc: ipac-list at ipacweb.org  

> Subject: Re: [IPAC-List] Reliability for T&Es

>

> How  about test-retest reliability?

>

>

>  Joel

>

>

> - -

> Joel P. Wiesen, Ph.D.,  Director

> Applied Personnel Research

> 62 Candlewood Road

>  Scarsdale, NY 10583-6040

> (617) 244-8859

>  http://appliedpersonnelresearch.com  

>

>

>

>

>

> Bryan Baldwin  wrote:

>    

>> Can anyone point me to a source for  how to compute reliability for T&Es 

(assuming it can be done)?   Ultimate goal is to use this to create bands.

>>

>>  Thanks-

>>

>> Bryan Baldwin

>> Staff Services  Manager II

>> California Department of Justice

>> Division  of Administrative Support

>> Personnel Programs

>> (916)  322-5446

>>

>>

>> CONFIDENTIALITY NOTICE: This  communication with its contents may 

contain confidential and/or legally  privileged information. It is solely for the 
use of the intended recipient(s).  Unauthorized interception, review, use or 
disclosure is prohibited and may  violate applicable laws including the 
Electronic Communications Privacy Act.  If you are not the intended recipient, 
please contact the sender and destroy  all copies of the communication.

>>

>>       

> _______________________________________________________

>  IPAC-List

> IPAC-List at ipacweb.org 

>  http://www.ipacweb.org/mailman/listinfo/ipac-list 

>  _______________________________________________________

>  IPAC-List

> IPAC-List at ipacweb.org 

>  http://www.ipacweb.org/mailman/listinfo/ipac-list 

>     

_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org  
http://www.ipacweb.org/mailman/listinfo/ipac-list


CONFIDENTIALITY  NOTICE: This communication with its contents may contain 
confidential and/or  legally privileged information. It is solely for the use 
of the intended  recipient(s). Unauthorized interception, review, use or 
disclosure is  prohibited and may violate applicable laws including the 
Electronic  Communications Privacy Act. If you are not the intended recipient, 
please  contact the sender and destroy all copies of the  communication.
_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
http://www.ipacweb.org/mailman/listinfo/ipac-list