[IPAC-List] How significant is significant enough?

[Winfred Arthur, Jr.]

forgot to also add that whereas the statistical significance is 
obviously of some value, you shld focus more or just as much on the 
magnitude of the relationship.  thus, a confidence interval around it 
would be particularly informative.

- winfred

Winfred Arthur, Jr. wrote:

> . . . assuming that i am understanding you correctly, a couple of points:

>

> 1.  if the .36 represents a criterion-related validity estimate, that 

> it is really not small in the context of what one generally sees in 

> the literature and practice.  of course it is based on a relatively 

> small sample size and so it may be susceptible to some sampling error 

> biases.

>

> 2.  based on your description of the sample, it would seem you may 

> have range restriction issues as well and so the .36 is probably an 

> underestimate.

>

> 3.  merely increasing the sample size is likely to increase, well, 

> really decrease the p-value but is unlikely to systematically increase 

> the point estimate (i.e., the correlation) if the additional cases are 

> drawn from the same "pool" as the present sample.

>

> hope this is of some help.

>

> - winfred

>

> Jason Bowling wrote:

>> Hi all,

>>

>>  

>>

>> I recently joined this list-serv and greatly enjoyed the conversations

>> thus far.

>>  

>>

>> So, I have a question and would appreciate any thoughts.

>>

>>  

>>

>> I am considering administering a test of basic math skills to candidates

>> for apprentice electrical lineman. Actually, prior to my starting with

>> the company, a math test was SOP for candidates to these jobs, but no

>> one ever asked if the test was a valid predictor. Furthermore, the

>> weight placed on test results was always ambiguous. We seemed to be

>> taking a pragmatic approach to our employment processes... that is, it

>> always seemed to work in the past.

>>

>>  

>>

>> Our apprenticeship does include testing components, and math does play a

>> large role in those tests. Furthermore, the job description does

>> describe math skills to be an essential competency, and we have had to

>> let apprentices go for failing these apprenticeship tests.

>>  

>>

>> So, I gave the test to our current apprentices and recent graduates

>> (N=30), and compared their test results to their average apprenticeship

>> test results (r=.36, and p=.02). The correlation seems weak to me, but I

>> have no basis of comparison.

>>  

>>

>> I could probably get more subjects to bolster the significance, but

>> again, I'm not sure at what point I should feel satisfied.

>>

>>  

>>

>> Any help?

>>

>>  

>>

>> Thanking you I am,

>>

>>  

>>

>>  

>>

>> Jason Bowling, PHR

>>

>> Human Resources Manager

>>

>> S.S.V.E.C

>>

>> Ofc: 520.515.3480

>>

>> Cel: 520.220.6563

>>

>>  

>>

>> _______________________________________________________

>> IPAC-List

>> IPAC-List at ipacweb.org

>> http://www.ipacweb.org/mailman/listinfo/ipac-list

>>   

> _______________________________________________________

> IPAC-List

> IPAC-List at ipacweb.org

> http://www.ipacweb.org/mailman/listinfo/ipac-list