[IPAC-List] Algorithms vs clinical judgment

[Winfred Arthur, Jr]

here is the abstract of Scott's paper:

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
*Abstract*
The focus of this article is on implicit beliefs that inhibit adoption 
of selection decision aids (e.g., paper-and-pencil tests, structured 
interviews, mechanical combination of predictors). Understanding these 
beliefs is just as important as understanding organizational constraints 
to the adoption of selection technologies and may be more useful for 
informing the design of successful interventions. One of these is the 
implicit belief that it is theoretically possible to achieve 
near-perfect precision in predicting performance on the job. That is, 
people have an inherent resistance to analytical approaches to selection 
because they fail to view selection as probabilistic and subject to 
error. Another is the implicit belief that prediction of human behavior 
is improved through experience. This myth of expertise results in an 
overreliance on intuition and a reluctance to undermine one’s own 
credibility by using a selection decision aid.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

On 8/22/2013 11:12 AM, Mark Hammer wrote:

> In fact, one of the listserv members just sent me an off-line note 

> connecting that very paper.

> Mark

>

> >>> "Winfred Arthur, Jr." <w-arthur at tamu.edu> 2013/08/22 12:09 PM >>>

> don't know if you have seen it or not but Scott's (Highhouse) 2008 

> piece ("stubborn reliance on intuition and subjectivity in employee 

> selection") in IOP is also an interesting and informative read on this 

> issue.

>

> - winfred

>

>

> On 8/22/2013 10:22 AM, Mark Hammer wrote:

>> I am currently wending my way, at an unjustifiably slow rate, through 

>> Daniel Kahneman's marvelous, well-written book "Thinking, fast and 

>> Slow".  For the unfamiliar, Kahneman shared the Nobel in Economics 

>> with is long-time research partner Amos Tversky, and is one of the 

>> godfathers of behavioural economics.  Fundamentally, he is a 

>> cognitive scientist, and folks in social cognition find reasons to 

>> cite his and Tversky's work at least several times a year.  their 

>> contribution to economics lay in demonstrating the many ways in which 

>> human reasoning and decision-making is far from the rational thing we 

>> too often presume it to be.

>> One of the central themes and questions running through the book, 

>> that Kahneman asks, is "Why don't people think like statisticians?".  

>> By "statisticians", he means rigorous methodical 

>> rule-and-probability-guided use of empirical information.  Not quite 

>> the same as the classic contrast between heuristic and algorithm, but 

>> definitely in that ballpark.  He uses this jumping-off point to link 

>> a diverse array of research, to address the manner in which humans 

>> often, and sometimes to their own detriment, reason through decisions 

>> using unsystematic, superficial, "System 1" (his categorization of 

>> rapid, near-automatic, judgment) information and processes.

>> Some of the book addresses in detail the seeming inability of people 

>> to think in terms of, or even accept, regression to the mean, and the 

>> predictive power of established regressions and demonstrable 

>> statistical relationships.  It occurred to me, as I read it, that the 

>> struggle assessment professionals often have to address is identical 

>> to those laid out in the book: /managers would often rather rely on 

>> their "clinical judgment" than on algorithmic prediction of 

>> performance via well-developed tests, even where it can be 

>> demonstrated that the tests have greater predictive power. /I don't 

>> expect hiring managers to BE Jack Hunter, but it is their resistance 

>> to admitting that maybe he's right that sits at the root of the 

>> challenge to assessment professionals.  Gut feel lies at the basis of 

>> much human reasoning and decision-making, probably more than we care 

>> to admit, and this includes hiring/promotion decisions.

>> Kahneman explains the "rules" for how humans are persuaded by 

>> superficial associations that can convince them of their validity.  

>> You may recall from your undergrad Psych 100 days the poster child 

>> for this of the "availability heuristic"; the human tendency to infer 

>> that if detailed examples of something come to mind easily, then that 

>> outcome must happen with great frequency, since actual frequent 

>> occurrences tend to prompt easily mentally available instances.

>> There is much in the book to prompt deeper, and useful, thought about 

>> how "assessment" all-too-often happens in the real world.  A worthy 

>> read, and a worthy career, documented in delightful prose.  As a 

>> psychologist, we sort of stand outside the world of Nobel Prizes, 

>> scratching out heads at the reasoning that goes into the nomination 

>> process for chemistry, physics, medicine, etc.  Reading this gives me 

>> confidence that those folks know what they're doing.

>> Mark Hammer

>> Ottawa

>>

>> >

>> This e-mail message is intended for the named recipient(s) and may 

>> contain information that is privileged, confidential and/or exempt 

>> from disclosure under applicable law. Unauthorized disclosure, 

>> copying or re-transmission is prohibited. If you are not a named 

>> recipient or not authorized by the named recipient(s), or if you have 

>> received this e-mail in error, then please notify the sender 

>> immediately and delete the message and any copies.

>> >

>> Ce courriel est destiné exclusivement au destinataire mentionné en 

>> titre et peut contenir de l'information privilégiée, confidentielle 

>> ou soustraite à la communication aux termes des lois applicables. 

>> Toute divulgation non autorisée, toute reproduction ou réacheminement 

>> est interdit. Si vous n'êtes pas le destinataire de ce courriel, ou 

>> n'êtes pas autorisé par le destinataire visé, ou encore, si vous 

>> l'avez reçu par erreur, veuillez le mentionner immédiatement à 

>> l'expéditeur et supprimer le courriel et les copies.

>>

>>

>> _______________________________________________________

>> IPAC-List

>> IPAC-List at ipacweb.org

>> http://nine.pairlist.net/mailman/listinfo/ipac-list

>

>

> >

> This e-mail message is intended for the named recipient(s) and may 

> contain information that is privileged, confidential and/or exempt 

> from disclosure under applicable law. Unauthorized disclosure, copying 

> or re-transmission is prohibited. If you are not a named recipient or 

> not authorized by the named recipient(s), or if you have received this 

> e-mail in error, then please notify the sender immediately and delete 

> the message and any copies.

> >

> Ce courriel est destiné exclusivement au destinataire mentionné en 

> titre et peut contenir de l'information privilégiée, confidentielle ou 

> soustraite à la communication aux termes des lois applicables. Toute 

> divulgation non autorisée, toute reproduction ou réacheminement est 

> interdit. Si vous n'êtes pas le destinataire de ce courriel, ou n'êtes 

> pas autorisé par le destinataire visé, ou encore, si vous l'avez reçu 

> par erreur, veuillez le mentionner immédiatement à l'expéditeur et 

> supprimer le courriel et les copies.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://nine.pairlist.net/pipermail/ipac-list/attachments/20130822/c044b8b4/attachment.html