[IPAC-List] Algorithms vs clinical judgment

[Mark Hammer]

I am currently wending my way, at an unjustifiably slow rate, through
Daniel Kahneman's marvelous, well-written book "Thinking, fast and
Slow".  For the unfamiliar, Kahneman shared the Nobel in Economics with
is long-time research partner Amos Tversky, and is one of the godfathers
of behavioural economics.  Fundamentally, he is a cognitive scientist,
and folks in social cognition find reasons to cite his and Tversky's
work at least several times a year.  their contribution to economics lay
in demonstrating the many ways in which human reasoning and
decision-making is far from the rational thing we too often presume it
to be.
 
One of the central themes and questions running through the book, that
Kahneman asks, is "Why don't people think like statisticians?".  By
"statisticians", he means rigorous methodical
rule-and-probability-guided use of empirical information.  Not quite the
same as the classic contrast between heuristic and algorithm, but
definitely in that ballpark.  He uses this jumping-off point to link a
diverse array of research, to address the manner in which humans often,
and sometimes to their own detriment, reason through decisions using
unsystematic, superficial, "System 1" (his categorization of rapid,
near-automatic, judgment) information and processes.
 
Some of the book addresses in detail the seeming inability of people to
think in terms of, or even accept, regression to the mean, and the
predictive power of established regressions and demonstrable statistical
relationships.  It occurred to me, as I read it, that the struggle
assessment professionals often have to address is identical to those
laid out in the book:  managers would often rather rely on their
"clinical judgment" than on algorithmic prediction of performance via
well-developed tests, even where it can be demonstrated that the tests
have greater predictive power.  I don't expect hiring managers to BE
Jack Hunter, but it is their resistance to admitting that maybe he's
right that sits at the root of the challenge to assessment
professionals.  Gut feel lies at the basis of much human reasoning and
decision-making, probably more than we care to admit, and this includes
hiring/promotion decisions.
 
Kahneman explains the "rules" for how humans are persuaded by
superficial associations that can convince them of their validity.  You
may recall from your undergrad Psych 100 days the poster child for this
of the "availability heuristic"; the human tendency to infer that if
detailed examples of something come to mind easily, then that outcome
must happen with great frequency, since actual frequent occurrences tend
to prompt easily mentally available instances.
 
There is much in the book to prompt deeper, and useful, thought about
how "assessment" all-too-often happens in the real world.  A worthy
read, and a worthy career, documented in delightful prose.  As a
psychologist, we sort of stand outside the world of Nobel Prizes,
scratching out heads at the reasoning that goes into the nomination
process for chemistry, physics, medicine, etc.  Reading this gives me
confidence that those folks know what they're doing.
 
Mark Hammer
Ottawa


>

This e-mail message is intended for the named recipient(s) and may
contain information that is privileged, confidential and/or exempt from
disclosure under applicable law. Unauthorized disclosure, copying or
re-transmission is prohibited. If you are not a named recipient or not
authorized by the named recipient(s), or if you have received this
e-mail in error, then please notify the sender immediately and delete
the message and any copies.
 >
Ce courriel est destiné exclusivement au destinataire mentionné en titre
et peut contenir de l'information privilégiée, confidentielle ou
soustraite à la communication aux termes des lois applicables. Toute
divulgation non autorisée, toute reproduction ou réacheminement est
interdit. Si vous n'êtes pas le destinataire de ce courriel, ou n'êtes
pas autorisé par le destinataire visé, ou encore, si vous l'avez reçu
par erreur, veuillez le mentionner immédiatement à l'expéditeur et
supprimer le courriel et les copies.      

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://nine.pairlist.net/pipermail/ipac-list/attachments/20130822/7a0c32fa/attachment.htm