[IPAC-List] Testing Education

[Dennis Doverspike]

Educated - by whom? One of my great fears regarding testing, is that the
problem is getting worse rather than better. Due to standardized testing
pressures in the schools, from 1st grade on, or even before, school children
are now being taught by their teachers that tests are bad, unfair, do not
assess knowledge, and are otherwise full of testing tricks. The situation
does not get better when students get to college, in Introduction to
Psychology classes, students are taught about the evils of testing including
the inherent biases against most cultures. The government can point to SIOP
members and argue that there is no consensus that ability tests are really
valid and free of bias, even if one could argue there is as much consensus
as there is in the medical community regarding drugs or surgeries.

My esteemed colleague Winfred Arthur often asks the question as to whether
if a test could be 100% accurate, would people want to use it or see the
result? I fear we know how most people would answer that question. 

We have met the enemy - and it is testing.

Of course, this problem is not limited to psychological tests for employment
purposes. People often cheat or fake on clinical psychology tests, usually
to fake bad rather than good (and see for example the huge controversy
recently over the release of the Rorschach ok Wikipedia). People cheat on
drug tests. People cheat on medical exams. 

And believe it or not, I am an optimist regarding human nature.

Dennis Doverspike, Ph.D., ABPP
Professor of Psychology
Director, Center for Organizational Research
Senior Fellow of the Institute for Life-Span Development and Gerontology
Psychology Department
University of Akron
Akron, Ohio  44325-4301
330-972-8372 (Office)
330-972-5174 (Office Fax)
ddoverspike at uakron.edu

The information is intended only for the person or entity to which it is
addressed and may contain confidential, privileged and/or a work product for
the sole use of the intended recipient.  No confidentiality or privilege is
waived or lost by any errant transmission. If you receive this message in
error, please destroy all copies of it and notify the sender. If the reader
of this message is not the intended recipient, you are hereby notified that
any dissemination, distribution or copying of this communication is strictly
prohibited. In the case of E-mail or electronic transmission, immediately
delete it and all copies of it from your system and notify the sender.
E-mail and fax transmission cannot be guaranteed to be secure or error-free
as information could be intercepted, corrupted, lost, destroyed, arrive late
or incomplete, or contain viruses. 

-----Original Message-----
From: ipac-list-bounces at ipacweb.org [mailto:ipac-list-bounces at ipacweb.org]
On Behalf Of Mark Hammer
Sent: Sunday, November 22, 2009 9:18 AM
To: IPAC-List at ipacweb.org
Subject: Re: [IPAC-List] A question of personality

When one uses any tool that looks at the social properties of the 
individual, some of what you'll get is "managed" information from 
the respondent, and some is the unguarded spontaneous information.  
As Harry's informants suggested, it is not really possible to completely
escape the managed information component, even if one were simply 
observing the person in a closed windowless room via videocam.  So 
the question becomes one of whether there is *enough* non-managed 
spontaneous information to make a reasoned judgment about the social 
properties of that person.  In some respects, high scores on faking and 
social desirability on a personality test are equivalent to a person leaving

much of an exam unanswered - no basis for evaluation - although in 
other respects they ARE information of a sort, aren't they?  Here is where 
RPClare's advice to declare and define such responding AS information 
up front is well-taken.  Far too many may interpret a personality inventory 
as intended to show "them" that you can be who you think they want you 
to be, and not necessarily intended to show who they are when no one is 
looking.  That it is a promotional exam merely shifts the balance more in 
the direction of the first interpretation than the second ("See?  I can be 
that kind of lieutenant too.")

When you say you have been asked to recommend substitute procedures, 
do you mean for the two candidates in question?  for the candidates overall?

or simply for any future competitions?  By the way, is there anything
relevant 
from the original writing of the test that could be salvaged as
interpretable, or
does the response distortion information negate everything?

Finally, one of the messages that doesn't get out to candidates often enough
is the manner in which many of the things they perceive as obstacles to
their
own success in competitions are intended to provide a level playing field
FOR
THEIR BENEFIT.  In other words, should a candidate bare their soul and self-
report in a throughly honest manner, they should not be penalized for it
because everyone else is faking, either the first time or in any subsequent
competition.  It's the same reason we don't provide too much feedback about
some kinds of assessments; open the gates to feedback too much and one 
risks giving some candidates an unfair advantage the next time by providing 
more usable feedback for some than for others, or by simply giving the 
advantage to re-takers over first-timers.  The candidate, meanwhile, thinks 
you're just holding out on them, rather than holding out on everyone else
who 
might beat them unfairly.  They need to be educated to see things in the big

picture.  We need to work harder at that.

My sympathies, Harry.

Mark Hammer
Ottawa


>>> <RPClare at aol.com> 2009/11/21 8:34 am >>>

I agree with Lance that a retake creates the impression of a second chance  
that is not provided to others who failed. I also would have concerns 
regarding  your "overly generous" interpretation. It suggests that they were
not 
only given  a second chance but the requirements were also eased. 
I do believe this type of test can have a place in Civil service. We  
routinely use "non-scored" hurdles that do not impact the rankings (e.g.  
degrees, trainings, licenses, physical exams).
The magic is to clearly communicate what the rules are in advance. For my  
own benefit, I would have a clearly written "pass point" for the test. If 
that  can't be done because there is a "clinical" judgment involved, I would

make sure  a credentialed clinician make the call (even if it's only on the 
close  ones)
 
 
In a message dated 11/20/2009 5:51:21 P.M. Eastern Standard Time,  
Harry.Brull at pdininthhouse.com writes:

Folks,

I'm interested in any advice in a sticky  situation:

Recently I conducted a police lieutenant promotional process  for a 
department of approximately 100 sworn personnel. One of the components  was
an 
omnibus personality inventory which I "scored". (It happened to be a  PDI 
proprietary instrument, but the situation would have been no different
with, for 
example, the California Personality Inventory (CPI) or  equivalent.)

Two of the candidates produced invalid profiles (as  evidenced by a good 
impression and response distortion scale). I asked them to  re-take the 
instrument and was able to interpret the results (although one  resulting
profile 
was marginally valid and I probably was overly generous in  interpreting the

results)

Afterwards, even though his total results  placed him well out of the 
running for promotion, one of the candidates  complained that he was
unfairly 
treated. Others wondered publically why I  didn't just "fail" him for this 
portion of the process.

Two Civil  Service hearings later, I am feeling somewhat frustrated trying 
to explain  personality testing, faking, etc. to people who either don't 
understand or  don't want to understand.

So here's my questions:
*    Do  self-report measures such as personality profiles have a place in 
Civil  service procedures which must produce a rank-ordered list of  scores?
*    Are there instruments which overcome the hurdle of  faking (my 
internal experts tell me "no")
*    What should one do  with a candidate profile that is uninterpretable?

I've been asked to  recommend substitute procedures. So far I've suggested 
oral interview,  role-plays, and the Promotability index (a procedure where 
multiple raters  place candidates of numerical "rungs" of a ladder - 
candidate score is the  arithmetic average of assigned ratings).

Any ideas out  there?

Thanks,
Harry Brull
Harry Brull |  Senior  Vice-President 
PDI Ninth House
Global Leadership  Solutions

1.612.337.8233  office
1.612.414.8998   mobile
1.612.337.3695  fax
Harry.Brull at pdininthhouse.com  <mailto:First.Last at pdininthhouse.com> 

33 South Sixth  Street
Suite 4900
Minneapolis, MN 55402

www.pdininthhouse.com  


Confidentiality Notice: All information in this communication,  including 
any files or attachments, is intended for the sole use of the  individual or

entity to which it is addressed and may contain information that  is 
confidential, proprietary and/or trade secret information entitled to
protection 
and/or exempt from disclosure under applicable law. If you are not  the 
intended recipient, you are hereby notified that any dissemination,  
distribution, or copying of this communication is strictly prohibited. If
you  have 
received this communication in error, please notify the sender by return
email 
and delete this communication from your system. Thank you for your  
cooperation.
_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org 
http://www.ipacweb.org/mailman/listinfo/ipac-list 


_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org 
http://www.ipacweb.org/mailman/listinfo/ipac-list
_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
http://www.ipacweb.org/mailman/listinfo/ipac-list