[IPAC-List] KSA Validity Studies?

Michael McDaniel (WSF) McDaniel at WorkSkillsFirst.com
Tue Jun 18 18:31:58 EDT 2013


If KSA narratives are stories related to accomplishments in a KSA domain
(e.g., project management. leadership), they have previously been called
"behavioral consistency method T&E" and "accomplishment records."

See this paper for validity data:

The data are described as Behavioral Consistency. The data are listed in
Appendix A. The uncorrected mean validity is listed as .248. I suspect
that overestimates the validity of the method. I say this because most of
the studies were conducted by a consulting firm known for high quality
work. They likely had ample funding for instrument development, rater
training, and checks on reliability of ratings. I doubt that most
applications of the method are as well funded and well conducted. I have
used this approach a few times and often had difficulty getting rater
agreement on some of the essay topics.

That study was from 1998. If you find more data, I can update the analysis.

Best wishes,


On Tue, Jun 18, 2013 at 12:38 PM, John Ford <johnf at us.net> wrote:

> Hi Folks!****


> All of us are familiar with KSA narratives, even though they are less

> commonly the first hurdle in U.S. Federal hiring. They are still with us.

> ****


> Can anybody point me to a validity study of KSA narratives? Or anything

> close to it?****


> Help in this area would be much appreciated!****


> John****


> --****


> John M. Ford, Ph.D.

> Senior Research Psychologist

> Policy and Evaluation

> U.S. Merit Systems Protection Board

> (202) 254-4499****


> _______________________________________________________

> IPAC-List

> IPAC-List at ipacweb.org

> http://nine.pairlist.net/mailman/listinfo/ipac-list



*Ten Free Employment Tests Per Month*

Michael A. McDaniel, Ph.D.
Work Skills First, Inc.
Voice: 804-277-9730
E-Mail: McDaniel at WorkSkillsFirst.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://nine.pairlist.net/pipermail/ipac-list/attachments/20130618/746a23d2/attachment.html

More information about the IPAC-List mailing list