[IPAC-List] How to build a cognitive ability test with lower mean group differences

[Joel Wiesen]

Hi Mike,

Here are a few, quick push-back comments on your PPTs.

Topics not addressed may leave the audience with a wrong impression that 
tests of g are the only reasonable way to select employees and that high 
levels of adverse impact are unavoidable.

Yes, g has a certain level of importance for job performance for many 
jobs,  but there are important indications of limitations of such tests 
and of our lack of understanding of job performance and/or the 
determinants of job performance.

Your presentation implicitly assumes that g is valid, important, and 
fair, and that there are no predictors of import other than g.  Your 
presentation seems to ignore many research-supported findings (some 
based on your own published research), such as:

1.  Validity of g decreases with time on the job.
2.  Conscientiousness is generally valid and uncorrelated with g.
3.  The validity of published studies of validity is greater than that 
of unpublished studies.
4.  Validity is higher for training than for job performance.
5.  Validity is higher for job knowledge than ability.
6.  Thorough training mitigates differences in ability.
7.  Strong indications of criterion bias call test fairness into question.
8.  Putting organizational resources into recruiting can have bigger 
payoff (higher utility) than putting money into selection refinements.
9.  Critical, creative thinking and problem solving requires more than g.
10. Thinking biases are generally independent of cognitive ability.
11. Some adverse impact may be due to test instructions (e.g., SJT).
12. Differential validity.
13. Relatively higher false negative rates for lower scoring groups.
14. Larger d for test performance than job performance.
15. Larger within person variance in job performance than between person.
16. Non g predictors can be more valid than tests of g.

Some of the above findings suggest that tests of g are less valid than 
previously thought (e.g., #1, 3, 4, 5, 6, 10, 11, 15, 16), that tests of 
g are sometimes (or often) unfair (e.g., #7, 12), and that there are 
substitutes for tests of g that can be expected to result in good job 
performance (e.g., #2, 5, 6, 8, 9, 16).

Yes, your presentation reports on your research, which did not have as 
broad a scope as some of my comments here imply.  Yet, I think the 
topics above provide a much needed perspective on your research.

Thank you again for posting your PPTs and talking notes and for asking 
for comment.

Joel


- -
Joel P. Wiesen, Ph.D., Director
Applied Personnel Research
62 Candlewood Road
Scarsdale, NY 10583-6040
http://www.linkedin.com/in/joelwiesen
(617) 244-8859
http://appliedpersonnelresearch.com




Note: This e-mail and any attachments may contain confidential and/or 
legally privileged information. Please do not forward any contents 
without permission. If you have received this message in error please 
destroy all copies, completely remove it from your computer, and notify 
the sender. Thank you.

On 2/15/18 5:06 PM, Michael McDaniel (WSF) wrote:
 > I am scheduled to give a presentation at PTC/MW next week on how to 
build a cognitive ability test with lower mean group differences. For 
those outside of the Washington, DC area, I attach my slides with note 
pages.
 >
 > Best wishes,
 >
 > Mike
 >
 > --
 > Michael A. McDaniel, Ph.D.
 > Work Skills First, Inc.
 > Voice: 804-277-9730
 > E-Mail: McDaniel at WorkSkillsFirst.com
 >
 >
 >
 >
 > _______________________________________________________
 > IPAC-List
 > IPAC-List at ipacweb.org
 > https://pairlist9.pair.net/mailman/listinfo/ipac-list