[IPAC-List] Value of cognitive ability testing for Journey-Level skilled trade jobs

Tue Mar 15 19:27:59 EDT 2016

Thanks Lance for the insights. There are a fair number of experienced folks who fail the cognitive test (spatial/numerical), and interestingly, this set of candidates actually fail at a higher rate than our general applicant population for these jobs. The reality is that our hiring managers and even our HR team is starting to question the value of a test when experienced, presumably successful performers cannot pass.  There are several theories about why these folks are failing including:

*         Test is not valid (although we have evidence saying otherwise)

*         Because many of these candidates are heavily recruited, they may not be taking the test seriously (e.g., "if you really want me to work for you, why are you giving me this test to pass?")

*         Many of these candidates are older and may not be used to taking tests (we have evidence saying otherwise)

*         Since these candidates are experienced, and may not have been screened on these skills previously, perhaps they have learned to "compensate" for lack of these skills on the job and still perform successfully
My hunch is that it may be a combination of different factors...but we may never know without doing some controlled research.

There is minimal cost, and little time savings by eliminating, so it is not an efficiency/cost issue. The issues are around 1) the test screening out candidates a large number of candidates who we've sourced/recruited because of their experience, 2) the job is difficult to fill because of the low supply/high demand, and 3) the test being viewed as a "basic skills test" being used for highly skilled/technical/experienced candidates.

Kevin Reindl
Human Resources
Assessments and Organizational Insights
Office: 415-973-7013
Mobile: 619-322-3368

From: Lance Seberhagen [mailto:sebe at erols.com]
Sent: Tuesday, March 15, 2016 4:06 PM
To: Reindl, Kevin; 'IPAC-List' (ipac-list at ipacweb.org)
Subject: Re: [IPAC-List] Value of cognitive ability testing for Journey-Level skilled trade jobs

Alert: This message originated outside of PG&E. Use caution when opening attachments, clicking links or responding to requests for information.
*************************************
Hi Kevin:

I have faced this question with some of my clients.  If PG&E has been giving the full test battery to experienced journey-level candidates for some time now, what percentage of these candidates pass the basic aptitude and cognitive tests?  If all candidates pass with flying colors, perhaps those tests aren't really needed.  On the other hand, if a significant number of candidates fail, perhaps those tests are needed, if PG&E wants to maintain the same quality of employees that were hired in the past.

PG&E's personality tests probably measure things like safety, teamwork, and work ethic.  These worker characteristics still have a high degree of variability among experienced candidates, so PG&E should probably continue administering those tests to experienced journey-level candidates.

So how much savings are we really talking about?  How much does it cost to administer the basic aptitude and cognitive tests?  How much extra time is required to administer these tests?  Would the benefit from dropping these tests be enough to out-weigh the risk of hiring skilled trades people who lack basic reading and math ability?  Is that risk greater for PG&E than for other employers?  What could possibly go wrong?

Lance Seberhagen, Ph.D.
Seberhagen & Associates
9021 Trailridge Court
Vienna, VA 22182
703-790-0796
www.seberhagen.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.seberhagen.com&d=BQMD-g&c=hLS_V_MyRCwXDjNCFvC1XhVzdhW2dOtrP9xQj43rEYI&r=Vs3zcJCUXAHYb8RXqbaBaw&m=eH7ALW1fr2LCRFkonB0M50NqMgrlbQhTf-hkDyHts6s&s=_gV38Bjd81MisMl57kd1G7rZT5kanCz2feDjeyjofsA&e=>

On 3/15/2016 5:10 PM, Reindl, Kevin wrote:
Hi all,

I wanted to get others' opinions/experiences on a question that our company is grappling with (if you can point to specific research that helps shed light on this, even better):

Here's the situation: We typically hire employees into entry-level training or apprentice-level jobs for our skilled trades (electrical, mechanical maintenance, etc.). When we hire entry level employees, we use an array of basic aptitude, cognitive and personality-based assessments in the selection process (e.g., mechanical aptitude, numerical reasoning, spatial skills, conscientiousness, etc.).

However, because of recent gaps in our journey-level workforce (e.g., due to retirements, etc.), we have been recruiting and hiring experienced journey-level employees...i.e., those who have been performing the same work at other companies or have already completed apprenticeships elsewhere. For these jobs we have been using the same set of aptitude/cognitive and personality assessments, PLUS a test of technical knowledge required in the trade. There is also a behavior-based interview and technical interview to round out the selection process.

There is debate in our company regarding whether there is value in continuing the use of broad-based aptitude/cognitive tests for these journey level hires.

1.       On one side of the argument, the cognitive tests are well-validated assessments that have shown to be predictive of success in a wide range of similar jobs, and since we "require" candidates to take them for the entry level jobs, why wouldn't we require them for higher level jobs?

2.       On the other side of the argument, why would we need to assess general cognitive ability for experienced journey-level candidates, since they have presumably been performing similar work at other companies. Also, since we also assess their technical knowledge in other tests, require X years of experience, and in many cases require them to have completed a certified apprenticeship...then what added value is measuring their basic aptitude to do this kind of work?

I know that internal local research/a validity study might provide some answers, but before we initiate that level of research, I'd be interested in thoughts from the IPAC group...

*         Which side do you lean toward #1 or #2 above, and why?

*         What do you do for similar situations in your organization?

*         Do you know of any research (single study or meta-analysis) that may shed some light on the matter?

As always, I look forward to your thoughts!

Kevin Reindl
Pacific Gas & Electric
Human Resources
Assessments and Organizational Insights
245 Market Street, N2J
San Francisco, CA 94105
Office: 415-973-7013
Mobile: 619-322-3368
k1rq at pge.com<mailto:k1rq at pge.com> or kevin.reindl at pge.com<mailto:kevin.reindl at pge.com>

_______________________________________________________

IPAC-List

IPAC-List at ipacweb.org<mailto:IPAC-List at ipacweb.org>

https://pairlist9.pair.net/mailman/listinfo/ipac-list<https://urldefense.proofpoint.com/v2/url?u=https-3A__pairlist9.pair.net_mailman_listinfo_ipac-2Dlist&d=BQMD-g&c=hLS_V_MyRCwXDjNCFvC1XhVzdhW2dOtrP9xQj43rEYI&r=Vs3zcJCUXAHYb8RXqbaBaw&m=eH7ALW1fr2LCRFkonB0M50NqMgrlbQhTf-hkDyHts6s&s=9wKBR3hkbrH6rG_Gn3JmCDD97WMEYXeCU1Z-zLxTNK0&e=>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist9.pair.net/pipermail/ipac-list/attachments/20160315/62098297/attachment-0001.html>