[IPAC-List] Measurement education

[Van Atta, Nancy]

Encouraging to read an example of fact trumping (mistaken) belief,
especially when the belief emanates from personal relationship. Thanks
for your post.

Nancy L. Van Atta
Employment Analyst, City of Dayton, Ohio
nancy.vanatta at daytonohio.gov
937-333-2070
937-333-2125 (fax)
 
David Friedland wrote, "After [current employees] were [tested and]
replaced by individuals who placed higher on the list, their supervisor
was struck by the fact that their replacements were much more
knowledgeable and better workers than the ones who were replaced."

-----Original Message-----
From: ipac-list-bounces at ipacweb.org
[mailto:ipac-list-bounces at ipacweb.org] On Behalf Of David Friedland
Sent: Thursday, October 25, 2012 3:05 PM
To: IPAC-List at ipacweb.org
Subject: Re: [IPAC-List] Measurement education

There have been many excellent ideas presented in this thread. I would
like to present my two cents as well. I believe that the problem as
presented by Steven is a very common one, in which the people in the
department involved have very definite opinions about the candidates
competing in the examination. The issues involved are complex but very
familiar to us, as indicated by the interest expressed in this
discussion. 

1. Civil service examinations are not designed or intended to be
individual assessments. They are intended to be valid in the sense that
they are intended to improve our true positives and minimize false
positives. There is always a degree of error to be expected. 

2. Impressions of supervisors and managers of who is the best prepared
for promotion or who is the best performer are individual assessments
made without the benefit of formal assessment. These impressions are
often not correct, and may be influenced by subjective perceptions based
on factors that may have nothing to do with performance, such as
likability. 

3. When examinations are prepared by HR without the involvement of
people in the appointing authority there is a built-in resistance to
accepting outcomes that disagree with their expectations of who should
come out on top. We can't assume that the examination is flawed if it
doesn't produce results that match the pre-conceived expectations of the
appointing authority. 

4. Perceptions of supervisors and managers are not always accurate.
Examples from my own experience include:

     a. As a result of the outcome of a legal case, more than 30
employees were moved from civil service exempt jobs to civil service
jobs. They were then required to compete in an examination to retain
their jobs. They were
unable to    score high enough on the valid examination to be reachable
on
the list and retain their jobs. Their supervisor testified in trial that
they were highly qualified for their jobs and were excellent workers.
After they were replaced by individuals who placed higher on the list,
their supervisor was struck by the fact that their replacements were
much more knowledgeable and better workers than the ones who were
replaced. 

     b. In a manufacturing company a sample of 150 quality control
employees in a glass manufacturing company participated in a controlled
study employing an exact replication of their production line inspection
job using a known sample of glassware with known flaws mixed in randomly
with glassware with no flaws. The employees who supervisors believed
were the most accurate inspectors were not the best performers in the
simulation. 

A good approach, though there may be no ultimate solution, is to involve
individuals from the department involved in designing the examination
process. Joel Wiesen alluded to this in his post. This approach may
create security challenges, but may have some success in increasing
acceptance. 

At the root of this problem is a difference in approach. Supervisors and
managers are making subjective individual assessments concerning who
they believe is best qualified, while the civil service examination
process is a more objective group assessment that may not result in the
same ranking as the informal individual assessment of the supervisors
and managers. We may be able to optimize our assessment processes, but
we may always have to live with some tension between these two
approaches. 

David Friedland


David Friedland, Ph.D., SPHR
Friedland & Associates, Inc.
Ph.	(310) 204-0045
Cell	(310) 699-5881
David at FriedlandHR.com




_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
http://www.ipacweb.org/mailman/listinfo/ipac-list