[IPAC-List] Angoff method of setting pass points - howmanyminimally competent candidates pass at a given pass point?

[Dr. Dennis Doverspike]

Mark

Sure we can determine the reliability of a test (or perhaps more exactly its
standard error of measurement) at any point using either classical test
theory or even better IRT theory.

An advantage of IRT theory is that it specifically gives us the amount of
information offered by a test at any specific score or latent trait value.

However, to answer your question, we do not always use a test only to
discriminate at the cutoff score. For example, in police testing, I may have
a cutoff score on a reading test but then will also combine it with other
tests to create an overall score for rank ordering. So in that case I would
not want to maximize reliability simply at the cutoff. 

In a case where I would know I only want to discriminate at a specific
cutoff, it would be best, if possible, to develop the test for maximum
reliability at that point.

Dennis Doverspike, Ph.D., ABPP

Professor of Psychology
Director, Center for Organizational Research Senior Fellow of the Institute
for Life-Span Development and Gerontology Licensed Psychologist, #3539
(OHIO) Psychology Department University of Akron Akron, Ohio  44325-4301
330-972-8372 (Office)
330-972-5174 (Office Fax)

ddoverspike at uakron.edu

-----Original Message-----
From: ipac-list-bounces at ipacweb.org [mailto:ipac-list-bounces at ipacweb.org]
On Behalf Of Mark Hammer
Sent: Monday, July 20, 2009 4:51 PM
To: ipac-list at ipacweb.org
Subject: Re: [IPAC-List] Angoff method of setting pass points -
howmanyminimally competent candidates pass at a given pass point?

Question:  Is it reasonable to suggest that scores on any test should be
maximally reliable at the pass-mark?  And if so, how would you demonstrate
that?

E.G., on a tool where the scores could range between 1 and 100, if the pass
mark was 60, do I care about the reliability of scores below, say 30, the
way I care about the reliability of scores between 57 and 63?

This would be most particularly the case where the tool in question is used
as an initial screen before applying one or more other (more labour
intensive and costly) tools.  One would think that someone who misses by a
point, or squeaks through by a point, does so for all the right reasons.

Mark Hammer
Ottawa

_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
http://www.ipacweb.org/mailman/listinfo/ipac-list