[IPAC-List] Sensitivity Reviews

[Mark Hammer]

I think I had to read some of those back in the day.  The take home message is very often that one should not assume that sensitivity reviews substitute particularly well for adverse-impact assessments.....in both directions.  Items/tests perceived as problematic by judges/committees can have little or no demonstrable adverse impact, and tests that pass muster by judges and review committees can have adverse impact up the wazoo.

Make no mistake, sensitivity to cultural/gender/racial/age/orientation/religious/linguistic/other issues is VERY important. No one should walk out of a test feeling like their dignity has been reduced by the nature of the content, or feeling marginalized in any way.  But it's not the same thing as whether items, categories of items, or tests demonstrate systematic bias in detecting what identifiable groups bring to the job, or how much of it.  My sense is that such committees need to be well-prepped on the difference between the two to be able to provide effective and helpful sensitivity review.

The same would be true of running a pilot instrument by a norming sample and asking them for an opinion on the bias or fairness of the test they just took.  In the absence of any experience with the correspondence between test content and outcomes of specific groups, their judgments are not well-calibrated and they both miss stuff and see stuff that isn't there.

Mark Hammer
Ottawa