[IPAC-List] selection validation question

[Dennis Doverspike]

Megan

1. I am responding based on the general information you have given - sort of
as a hypothetical.

2. OK, given that, I wonder if the consultant really means "validity
generalization" when they say "validity generalization." A lot of
consultants, when they say "validity generalization" really mean what we
would call "transfer validity." The uniform guidelines were written before
the validity generalization articles were published, more or less, so of
course the Uniform Guidelines do not mention validity generalization as we
mean it. They could not have unless they were somehow able to see the
future. But the Uniform Guidelines do mention a type of validity generally
referred to as "transfer" that a lot of consulting firms do use with their
published tests. So I am wondering whether the consultant really wants to
use or conduct some type of transfer study. 

3. The section you cite about content validity from the Uniform Guidelines
has probably been superseded by court cases since the Uniform Guidelines
were written (not a lawyer). However, in the situation you are describing,
it would be very difficult to support the test based only on a content
strategy. 

4. The bigger issue I guess no one yet has addressed - is it the
consultant's test or intellectual property? If it is, and they have certain
rules for the use of their test, I would think I would be reluctant to use a
test, or a drug, or any instrument, where I do not follow the procedures the
professional offering the service wants me to follow. If they say I have to
do a VG study before I use their test, isn't that their right to do so? Of
course, then it is my right to go to someone else who does not require me to
do a VG study and buy a test from a different consultant. I would doubt this
is the only consultant selling a test of this type. I might give an analogy.
I might go to Doctor A, who says, before I operate on you, you have to lose
30 pounds. Then I might go to Doctor B who says, you don't have to lose
weight before I operate on you, there are studies that suggest it has no
effect. Now, if I want the surgery, I would not go back to Doctor A and say
- operate on me right now, I don't need to lose 30 pounds. Because I imagine
Doctor A would say - no way, I am not taking the chance. If I don't want to
lose the weight, I would have Doctor B operate.  


Dennis Doverspike, Ph.D., ABPP
Professor of Psychology
Director, Center for Organizational Research
Senior Fellow of the Institute for Life-Span Development and Gerontology
Psychology Department
University of Akron
Akron, Ohio  44325-4301
330-972-8372 (Office)
330-972-5174 (Office Fax)
ddoverspike at uakron.edu

The information is intended only for the person or entity to which it is
addressed and may contain confidential, privileged and/or a work product for
the sole use of the intended recipient.  No confidentiality or privilege is
waived or lost by any errant transmission. If you receive this message in
error, please destroy all copies of it and notify the sender. If the reader
of this message is not the intended recipient, you are hereby notified that
any dissemination, distribution or copying of this communication is strictly
prohibited. In the case of E-mail or electronic transmission, immediately
delete it and all copies of it from your system and notify the sender.
E-mail and fax transmission cannot be guaranteed to be secure or error-free
as information could be intercepted, corrupted, lost, destroyed, arrive late
or incomplete, or contain viruses. 

-----Original Message-----
From: ipac-list-bounces at ipacweb.org [mailto:ipac-list-bounces at ipacweb.org]
On Behalf Of Megan Paul
Sent: Thursday, May 28, 2009 5:49 PM
To: IPAC-List at ipacweb.org
Subject: [IPAC-List] selection validation question

A few points of clarification...

I have every reason to believe that this measure will *not* result in 
adverse impact, but I support following the Uniform Guidelines as a 
matter of good practice. I'm just not sure I understand / agree with the 
interpretation I'm being given.

I definitely understand the importance of an updated job analysis, and 
it is likely that we will update the previous one ourselves (the biggest 
issue is convincing the agency to give us SME time). What I mostly 
question is the insistence on all the validity generalization work as a 
precursor to the criterion-related validation, and I definitely don't 
see where it's "required" by the Guidelines.

The following paragraph from the UG applies to this particular test, so 
content validity isn't going to be sufficient:
"A selection procedure based upon inferences about mental processes 
cannot be supported solely or primarily on the basis of content 
validity. Thus, a content strategy is not appropriate for demonstrating 
the validity of selection procedures which purport to measure traits or 
constructs, such as intelligence, aptitude, personality, commonsense, 
judgment, leadership, and spatial ability."

The test would be used in conjunction with several other selection tools 
that measure many other KSAs, so it does not need to "do it all."

-- 
Megan E. Paul, Ph.D.
Research Assistant Professor
University of Nebraska-Lincoln
Center on Children, Families, and the Law
206 S. 13th Street, Suite 1000
Lincoln, NE 68588-0227

(402) 472-9812 Office
(402) 472-8412 Fax


_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
http://www.ipacweb.org/mailman/listinfo/ipac-list