[IPAC-List] How significant is significant enough?

[Lance Seberhagen]

What are you trying to predict -- training performance or work 
performance?  The basic math test from the Apprenticeship entrance exam 
correlates .36 with the Apprenticeship final exam, which has a strong 
math component.  In other words, one math test correlates .36 with 
another math test.  Thus, the basic math test predicts passing the final 
exam, but you don't really know if math skills are actually needed for 
performance as an Electrical Lineman (or Line Worker).  A content 
validation study would be helpful in answering this question.

What is the date of the current job description for Electrical 
Lineman?   Is there a job analysis report to go with it?   Does the job 
analysis comply with the Uniform Guidelines?  You probably need to 
conduct a new job analysis (including the apprenticeship program) to get 
a clear understanding of what math skills, as well as other KSAs, are 
needed for the job of Electrical Lineman.  If the job analysis shows 
that math skills are needed, the next step would be to have job experts 
evaluate the relevance of math test items on the entrance exam and final 
exam, as well as to obtain suggestions for improving the relevance of 
the math questions on both exams.  Does the entrance exam consist only 
of a basic math test?  If so, you might want to consider adding other 
tests to the entrance exam (e.g., reading, teamwork, safety), based on 
the results of the job analysis.

Lance Seberhagen, Ph.D.
Seberhagen & Associates
9021 Trailridge Ct
Vienna, VA 22182
Tel 703-790-0796
www.seberhagen.com


Jason Bowling wrote:


>Hi all,

>

> 

>

>I recently joined this list-serv and greatly enjoyed the conversations

>thus far. 

>

> 

>

>So, I have a question and would appreciate any thoughts.

>

> 

>

>I am considering administering a test of basic math skills to candidates

>for apprentice electrical lineman. Actually, prior to my starting with

>the company, a math test was SOP for candidates to these jobs, but no

>one ever asked if the test was a valid predictor. Furthermore, the

>weight placed on test results was always ambiguous. We seemed to be

>taking a pragmatic approach to our employment processes... that is, it

>always seemed to work in the past.

>

> 

>

>Our apprenticeship does include testing components, and math does play a

>large role in those tests. Furthermore, the job description does

>describe math skills to be an essential competency, and we have had to

>let apprentices go for failing these apprenticeship tests. 

>

> 

>

>So, I gave the test to our current apprentices and recent graduates

>(N=30), and compared their test results to their average apprenticeship

>test results (r=.36, and p=.02). The correlation seems weak to me, but I

>have no basis of comparison. 

>

> 

>

>I could probably get more subjects to bolster the significance, but

>again, I'm not sure at what point I should feel satisfied.

>

> 

>

>Any help?

>

> 

>

>Thanking you I am,

>

> 

>

> 

>

>Jason Bowling, PHR

>

>Human Resources Manager

>

>S.S.V.E.C

>

>Ofc: 520.515.3480

>

>Cel: 520.220.6563

>  

>