[IPAC-List] Value of cognitive ability testing for Journey-Level skilled trade jobs

[Reindl, Kevin]

Hi again everyone,

I just wanted to thank everyone for their replies, advice, and opinions...this truly affirms the reason I subscribe to this discussion list - to gain access to the collective thinking of an amazingly smart, insightful and diverse set of assessment and selection professionals. The collective experience and knowledge you all bring truly helps to advance the assessment science and practice.

Just a few follow-ups and responses to some of you:


*         Harry B. - thanks for the offer to share your performance measure. Please do send it to me if you can. I'd love to suggest a quick study to collect the data we need to confirm/disconfirm local criterion validity for some of the tests we have in place for this job.

*         Ken D. - yes, your opinion regarding internal empirical validation is spot on - I am planning on going this route as part of a larger effort later this year

*         Don W. - thanks for the simple thought on just eliminating the cognitive test when advanced knowledge is critical and assessed...

*         Liz R. - you ask a lot of great questions that challenge the VG evidence of this test for the job...I agree that sometimes VG evidence gets pretty far removed from local use of the assessment; re: weighting - unfortunately, this may not be an option due to the way we use these tests across so many occupations. We tend to treat all of our assessments as hurdles with minimum cuts.

*         Joel W. - thanks for getting me thinking in the direction of crystallized (vs. fluid) intellect...it's true that for apprentice levels, we tend to want candidates with more fluid intelligence, but as experience is gained, perhaps journey level employees rely more on that which is crystallized. Also I agree with your thinking about potential disgruntled/outraged candidates...this could be an added risk factor to consider...thanks.

*         Rob M. - thanks for your input and for sharing the excerpt from your book chapter. Also thanks for the follow up call to discuss utility industry and EEI stance.

*         Ann N. - thanks for the idea about using different cut scores for different levels - we have talked about this as a potential consideration

*         Mark H. - thanks for your suggestion about using the test experimentally, but unfortunately this test is used widely across many jobs in the company, and the way we recruit is ongoing, so there is not an "eligibility" list that we work off.

*         Lance S. - spot on with your thinking about pass rates and utility of the tests. The relatively low pass rate on the cognitive test may indicate it's doing a good job screening out low-ability/low-motivated candidates, or it may indicate the test is not the right test for this job and this set of experienced candidates...

*         Keith P. - you are in the same thinking as us on this...speed in hiring (or simply removing some of the unneeded hurdles) is sometimes key  to hiring for "hard to fill" jobs. Other companies are competing for the same talent, and the more arduous we make our process, the more difficult it is to fill jobs. At the same time, we don't want to throw out the baby with the bathwater...

I'll try to post to this list again to let you all know what happens...

Again, thanks to all of you for the discussion!


Kevin Reindl
Human Resources
Assessments and Organizational Insights
Office: 415-973-7013
Mobile: 619-322-3368

From: Reindl, Kevin
Sent: Tuesday, March 15, 2016 2:10 PM
To: 'IPAC-List' (ipac-list at ipacweb.org<mailto:ipac-list at ipacweb.org>)
Subject: Value of cognitive ability testing for Journey-Level skilled trade jobs

Hi all,

I wanted to get others' opinions/experiences on a question that our company is grappling with (if you can point to specific research that helps shed light on this, even better):

Here's the situation: We typically hire employees into entry-level training or apprentice-level jobs for our skilled trades (electrical, mechanical maintenance, etc.). When we hire entry level employees, we use an array of basic aptitude, cognitive and personality-based assessments in the selection process (e.g., mechanical aptitude, numerical reasoning, spatial skills, conscientiousness, etc.).

However, because of recent gaps in our journey-level workforce (e.g., due to retirements, etc.), we have been recruiting and hiring experienced journey-level employees...i.e., those who have been performing the same work at other companies or have already completed apprenticeships elsewhere. For these jobs we have been using the same set of aptitude/cognitive and personality assessments, PLUS a test of technical knowledge required in the trade. There is also a behavior-based interview and technical interview to round out the selection process.

There is debate in our company regarding whether there is value in continuing the use of broad-based aptitude/cognitive tests for these journey level hires.

1.       On one side of the argument, the cognitive tests are well-validated assessments that have shown to be predictive of success in a wide range of similar jobs, and since we "require" candidates to take them for the entry level jobs, why wouldn't we require them for higher level jobs?

2.       On the other side of the argument, why would we need to assess general cognitive ability for experienced journey-level candidates, since they have presumably been performing similar work at other companies. Also, since we also assess their technical knowledge in other tests, require X years of experience, and in many cases require them to have completed a certified apprenticeship...then what added value is measuring their basic aptitude to do this kind of work?

I know that internal local research/a validity study might provide some answers, but before we initiate that level of research, I'd be interested in thoughts from the IPAC group...

*         Which side do you lean toward #1 or #2 above, and why?

*         What do you do for similar situations in your organization?

*         Do you know of any research (single study or meta-analysis) that may shed some light on the matter?

As always, I look forward to your thoughts!

Kevin Reindl
Pacific Gas & Electric
Human Resources
Assessments and Organizational Insights
245 Market Street, N2J
San Francisco, CA 94105
Office: 415-973-7013
Mobile: 619-322-3368
k1rq at pge.com<mailto:k1rq at pge.com> or kevin.reindl at pge.com<mailto:kevin.reindl at pge.com>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist9.pair.net/pipermail/ipac-list/attachments/20160318/d35ba62d/attachment-0001.html>