[IPAC-List] KSA Validity Studies?

[Michael McDaniel (WSF)]

John,

If KSA narratives are stories related to accomplishments in a KSA domain
(e.g., project management. leadership), they have previously been called
"behavioral consistency method T&E" and "accomplishment records."

See this paper for validity data:
http://www.people.vcu.edu/~mamcdani/Publications/McDaniel,%20Schmidt,%20Hunter%20%281988%29%29%20T&E.pdf

The data are described as Behavioral Consistency. The data are listed in
Appendix A. The uncorrected mean validity is listed as .248.  I suspect
that overestimates the validity of the method. I say this because most of
the studies were conducted by a consulting firm known for high quality
work.  They likely had ample funding for instrument development, rater
training, and checks on reliability of ratings.  I doubt that most
applications of the method are as well funded and well conducted. I have
used this approach a few times and often had difficulty getting rater
agreement on some of the essay topics.

That study was from 1998.  If you find more data, I can update the analysis.

Best wishes,

Mike






On Tue, Jun 18, 2013 at 12:38 PM, John Ford <johnf at us.net> wrote:


> Hi Folks!****

>

> All of us are familiar with KSA narratives, even though they are less

> commonly the first hurdle in U.S. Federal hiring.  They are still with us.

> ****

>

> Can anybody point me to a validity study of KSA narratives?  Or anything

> close to it?****

>

> Help in this area would be much appreciated!****

>

> John****

>

> --****

>

> John M. Ford, Ph.D.

> Senior Research Psychologist

> Policy and Evaluation

> U.S. Merit Systems Protection Board

> (202) 254-4499****

>

> _______________________________________________________

> IPAC-List

> IPAC-List at ipacweb.org

> http://nine.pairlist.net/mailman/listinfo/ipac-list

>

>



-- 
*Ten Free Employment Tests Per Month*
www.workskillsfirst.com

Michael A. McDaniel, Ph.D.
President
Work Skills First, Inc.
Voice: 804-277-9730
E-Mail: McDaniel at WorkSkillsFirst.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://nine.pairlist.net/pipermail/ipac-list/attachments/20130618/746a23d2/attachment.html