[IPAC-List] Divergent views of validation

[Jeff Feuquay]

If Borsboom asserts validity is what a test "should" measure, then it seems
he is discussing a concept quite different from what most assessment folk
think of as validity, i.e., whether a test measures what it purports to
measure. That idiosyncratic definition would quite reasonably be expected to
lead him down the rabbit hole to a very different world.

Some of the discussion seems to imply we are again caught in the trap of
hoping/believing that correlation reflects causation. Ah, were it only so.
Then, maybe we could say that whatever our tests are measuring (not
predicting . . . actually measuring) cause the future behavior sought. But,
we continue to measure the employee version of phlogiston. We've got some
understanding of which characteristics are likely to result in improved
performance, but it remains rudimentary. I mean, as Borsboom knows, it would
be a whole lot simpler if we were measuring temperature, something we now
undersatnd way better than we understand ourselves.

As to the level of measurment: Unless one has gone through a thorough
scaling process, certainly variations in item difficulty make us fall short
of interval mesurement. But on a practical level, how short of interval is
too short? When some jurisdictions continue to round T&E scores to
thousandths of a point and then take the top 3, is ordinal versus interval
or even ratio a big issue? Yeah, it is. Scaling is an issue, significant
digits is an issue, but most scales and most of the stats we commonly use
are extremely robust. We may bend the prediction but we're unlikely to break
it.

Given another hundred years to work on this, measurement of employee
phlogistan could conceivably evolve into something more direct and more
definitive . . . I mean, everybody knows that variations in the long arm of
chromsome 6 of the human genome account for 4% of the variance in
intelligence. See, we're on the way.
Jeff
-----------------------------------------
Dr. Jeffrey P Feuquay, I/O Psychologist & Attorney
Managing Consultant, Psychology-Law Center, LLC
108 W. Walnut, Nevada, Mo 64772
ofc: 417-667-5076   cell: 417-549-0997


On Wed, Sep 1, 2010 at 11:37 AM, Mark Hammer <Mark.Hammer at psc-cfp.gc.ca>wrote:


> Would it be wrong to suggest that this is one of those perpetual

> discrepancies between theoreticians and clinicians?  In this case, the

> discrepancy is between test/measurement specialists (the theoreticians),

> and test-users (the clinicians).

>

> Mark Hammer

> Ottawa

>

> >

> This e-mail message is intended for the named recipient(s) and may

> contain information that is privileged, confidential and/or exempt from

> disclosure under applicable law. Unauthorized disclosure, copying or

> re-transmission is prohibited. If you are not a named recipient or not

> authorized by the named recipient(s), or if you have received this

> e-mail in error, then please notify the sender immediately and delete

> the message and any copies.

>  >

> Ce courriel est destiné exclusivement au destinataire mentionné en titre

> et peut contenir de l'information privilégiée, confidentielle ou

> soustraite à la communication aux termes des lois applicables. Toute

> divulgation non autorisée, toute reproduction ou réacheminement est

> interdit. Si vous n'êtes pas le destinataire de ce courriel, ou n'êtes

> pas autorisé par le destinataire visé, ou encore, si vous l'avez reçu

> par erreur, veuillez le mentionner immédiatement à l'expéditeur et

> supprimer le courriel et les copies.

>

> _______________________________________________________

> IPAC-List

> IPAC-List at ipacweb.org

> http://www.ipacweb.org/mailman/listinfo/ipac-list

>