[IPAC-List] "False Positive Psychology"

[Dennis Doverspike]

Reid,

The article makes a great point. Although I would argue that the term
"authors" or "researchers" should not be take too narrowly. The problem is
perhaps even more acute among consultants under pressure to report
statistically significant results from small, operational sample sizes. Of
course, it reaches its pinnacle in our search for adverse impact, where
regulatory agencies may require conducting significance tests on large
number of job groups and then narrow attention to the one job group
demonstrating statistically significant differences.

Thanks to bringing this article to our attention.

Dennis

On Tue, Nov 29, 2011 at 11:59 AM, Reid Klion <RKlion at panpowered.com> wrote:


> There is an interesting article in Psychological Science (free

> download!: http://pss.sagepub.com/content/22/11/1359.full.pdf) on the

> perils of what the authors call unchecked "researcher degrees of

> freedom" that can readily lead to false positive findings in the

> never-ending search for "statistically significant" publishable results.

> They point out how when researchers don't disclose things like all the

> variables collected in the study, failed manipulations, and the specific

> rationale for the statistical analysis that was ultimately used, the

> probability of obtaining false positive findings increases (which they

> demonstrate by "showing" how subjects listening to the Beatles song "

> When I'm Sixty-Four" makes them older vs. subjects who listened to other

> music.) They go on to provide useful suggestions for both authors and

> reviewers to help mitigate these risks.

>

>

>

> Reid

>

>

>

>

>

>

>

> Reid E. Klion, Ph.D.

>

> Chief Science Officer

>

> pan - A TALX Company

>

> 11590 North Meridian St.

>

> Suite 200

>

> Carmel, IN 46032 USA

>

> 317.814.8808  Office

>

> 317.908.4312  Mobile

>

> 317.814.8888  Fax

>

> rklion at panpowered.com <mailto:rklion at panpowered.com>

>

> www.panpowered.com <http://www.panpowered.com/>

>

>

>

> _______________________________________________________

> IPAC-List

> IPAC-List at ipacweb.org

> http://www.ipacweb.org/mailman/listinfo/ipac-list

>




-- 
Dennis Doverspike, PhD., ABPP
Licensed Psychologist, #3539 (OHIO)
Independent Consultant
Professor of Psychology, University of Akron
dennisdoverspike at gmail.com

The information is intended only for the person or entity to which it is
addressed and may contain confidential, privileged and/or a work product
for the sole use of the intended recipient. No confidentiality or privilege
is waived or lost by any errant transmission. If you receive this message
in error, please destroy all copies of it and notify the sender. If the
reader of this message is not the intended recipient, you are hereby
notified that any dissemination, distribution or copying of this
communication is strictly prohibited. In the case of E-mail or electronic
transmission, immediately delete it and all copies of it from your system
and notify the sender. E-mail and fax transmission cannot be guaranteed to
be secure or error-free as information could be intercepted, corrupted,
lost, destroyed, arrive late or incomplete, or contain viruses.