[IPAC-List] Divergent views of validation

[Mark Hammer]

This thread is drifting, albeit in interesting ways.

I was not trained by Donald Hebb directly, though he did show up to one
of our undergrad classes and give a wonderful historical talk about his
career. Nevertheless, our department (McGill) was pervaded by his
influence everywhere. And one of his influences was that of prudent
judgment when it came to "neurologizing"; searching for, or attributing,
neural loci for constructs identified or elaborated via other means. As
the father of "cell assembly" theory, he certainly wasn't against it
(and castigated those, like Watson who refused any connection between
biology and the study of psychology), but neither was he inclined to
want to instantly pair up anything and everything that interested
psychology with a set of neural mechanisms.

"Personality" is first and foremost a social construct: the perception
of seeming consistencies in the behaviour of individuals across
contexts. While early writers like William McDougall may have posited
several thousand personality characteristics, based on an examination of
the lexicon for personality (i.e., if there is another word for it then
it must be a different trait, right?), later personality theorists, like
Hans Eysenck and Paul Costa pushed towards boiling all of such human
individual differences into as few macro-traits as possible, chiefly via
factor analysis, but also via useful approaches like longitudinal study.

At a certain point, however, the search for profound deep consistencies
that link broad spectra of human behaviour together in immutable ways
inevitably becomes biological, and the study and theorizing of
"personality" really starts to turn into the examination of temperament
and neurological predispositions that are nonspecific. The observation
of associations between brain morphology or response patterns (P300,
etc.), or even chemistry, and what gets measured, or rather clustered
together, as "conscientiousness" tells us nothing about HOW
conscientiousness might be expressed in the context of work, and
especially in the context of the job we are trying to fill. It is a sort
of "validation" of the Big 5, but completely divorced from the role we
want the Big 5 to have for us.

Some years back, I had the pleasure of attending a week-long series of
discussions with the late Nobel winner Roger Sperry. Sperry's chief
contribution was his work on split-brain patients -(an idea originally
suggested by Hebb, who seems to have influenced anybody of consequence
in neuropsychology) - and the focus of the discussions was mind-body
dualism. I shall never forget what Sperry responded when asked about his
comfort level with dualism. He said he was entirely comfortable dwelling
in both the realms of biology and the conceptual language of mind, going
back and forth as appropriate. He noted that, of course, everything is
ultimately a product of its molecular structure, but "You don't explain
the behaviour of a rubber ball by its molecules; you explain it in terms
of the properties of balls". And for Sperry that meant it was perfectly
reasonable to ignore the biology and talk simply in terms of mental
mechanisms if that was he more productive level of discourse.

Observation of neurological correlates of the Big 5 is a bit like the
relationship between subatomic particles and our rubber ball. Yes, it's
interesting, and we're pretty dang sure there's subatomic particles in
there somewhere, but it still doesn't put us any farther ahead in
understanding why, or predicting how, rubber balls bounce. 

And maybe that's the schism in these divergent views of what constitutes
validation. When we folk mean "validation", we want it to be related to
outcomes in a practical way. Again, back to the theoretical/clinical
split I noted earlier. For us "clinicians" we want validation to say
whether the patient is going to make it or is going to die. Validation
of what the drug does to ion channels is not really of any immediate
use.

If any of this makes no sense, let me know. I'm recovering from a long
drive in exceedingly hot weather in the total absence of air
conditioning or even a breeze. There are no assurances that I am
coherent.

In the meantime, carry on, and have a pleasant and safe Labour Day
weekend, and a sweet New Year to those of you observing it next week..

Mark Hammer
Orillia


>>> "Winfred Arthur, Jr." 09/03/10 11:41 AM >>> 

i will send you a copy of the paper offline. if anyone else is 
interested in a copy as well, just let me know. 

- winfred 


On 9/3/2010 7:24 AM, Shekerjian, Rene wrote: 

> I'd be interested in knowing the extent to which the measures of

volume for the various brain regions covaried with the measures of the
personality traits. My internet access to journals does not allow access
to the most recent year for Psychological Science. 

> 

> Has anyone had a chance to see what it says? 

> 

> René 

> 

> René Shekerjian | Testing Services Division | NYS Department of Civil

Service | 518-474-3778 

>

==========================================================================


> 

> -----Original Message----- 

> From: Reid Klion [mailto:RKlion at panpowered.com] 

> Sent: Thursday, September 02, 2010 5:00 PM 

> To: ipac-list at ipacweb.org 

> Subject: Re: [IPAC-List] Divergent views of validation 

> 

> Good discussion! ...there is also a literature that delves into some

of the challenges that have been found when attempting to replicate fMRI
research. Here is a quick overview from Science News:
http://tinyurl.com/ycveaac. Enjoy- 

> 

> Reid 

> 

> 

> Reid E. Klion, Ph.D. 

> Chief Science Officer 

> pan - A TALX Company 

> 11590 North Meridian St. 

> Suite 200 

> Carmel, IN 46032 USA 

> 317.814.8808 Office 

> 317.908.4312 Mobile 

> 317.814.8888 Fax 

> rklion at panpowered.com 

> www.panpowered.com 

> 

> 

> -----Original Message----- 

> From: ipac-list-bounces at ipacweb.org

[mailto:ipac-list-bounces at ipacweb.org] On Behalf Of Bryan Baldwin 

> Sent: Thursday, September 02, 2010 9:25 AM 

> To: ipac-list at ipacweb.org; w-arthur at neo.tamu.edu 

> Subject: Re: [IPAC-List] Divergent views of validation 

> 

> ..and to piggyback on Winfred's post, there are a couple articles in

the September Psych Bulletin dealing in some sense with biological bases
of personality (Kotov et al. and Dietrich et al.): 

> 

> http://psycnet.apa.org/journals/bul/136/5/ 

> 

> Bryan 

> 

> Bryan Baldwin 

> Staff Services Manager II 

> California Department of Justice 

> Division of Administrative Support 

> Personnel Programs 

> (916) 322-5446 

>>>> "Winfred Arthur, Jr." 09/01/10 6:03 PM>>> 

> as a follow up to Jeff's last paragraph, DeYoung et al.'s (2010) 

> mri/FFM study might be of interest to some of you. the full reference

is: 

> 

> DeYoung et al. (2010), "Testing predictions from personality 

> neuroscience: Brain structure and the big five", /Psychological

Science, 

> 21,/ 820-828. 

> 

> and the abstract reads as follows: 

> 

> "We used a new theory of the biological basis of the Big Five 

> personality traits to generate hypotheses about the association of

each 

> trait with the volume of different brain regions. Controlling for age,



> sex, and whole-brain volume, results from structural magnetic

resonance 

> imaging of 116 healthy adults supported our hypotheses for four of the



> five traits: Extraversion, Neuroticism, Agreeableness, and 

> Conscientiousness. Extraversion covaried with volume of medial 

> orbitofrontal cortex, a brain region involved in processing reward 

> information. Neuroticism covaried with volume of brain regions 

> associated with threat, punishment, and negative affect. Agreeableness



> covaried with volume in regions that process information about the 

> intentions and mental states of other individuals. Conscientiousness 

> covaried with volume in lateral prefrontal cortex, a region involved

in 

> planning and the voluntary control of behavior. These findings support



> our biologically based, explanatory model of the Big Five and 

> demonstrate the potential of personality neuroscience (i.e., the 

> systematic study of individual differences in personality using 

> neuroscience methods) as a discipline." 

> 

> 

> - winfred 

> 

> 

> On 9/1/2010 3:21 PM, Jeff Feuquay wrote: 

>> If Borsboom asserts validity is what a test "should" measure, then it

seems 

>> he is discussing a concept quite different from what most assessment

folk 

>> think of as validity, i.e., whether a test measures what it purports

to 

>> measure. That idiosyncratic definition would quite reasonably be

expected to 

>> lead him down the rabbit hole to a very different world. 

>> 

>> Some of the discussion seems to imply we are again caught in the trap

of 

>> hoping/believing that correlation reflects causation. Ah, were it

only so. 

>> Then, maybe we could say that whatever our tests are measuring (not 

>> predicting . . . actually measuring) cause the future behavior

sought. But, 

>> we continue to measure the employee version of phlogiston. We've got

some 

>> understanding of which characteristics are likely to result in

improved 

>> performance, but it remains rudimentary. I mean, as Borsboom knows,

it would 

>> be a whole lot simpler if we were measuring temperature, something we

now 

>> undersatnd way better than we understand ourselves. 

>> 

>> As to the level of measurment: Unless one has gone through a thorough



>> scaling process, certainly variations in item difficulty make us fall

short 

>> of interval mesurement. But on a practical level, how short of

interval is 

>> too short? When some jurisdictions continue to round T&E scores to 

>> thousandths of a point and then take the top 3, is ordinal versus

interval 

>> or even ratio a big issue? Yeah, it is. Scaling is an issue,

significant 

>> digits is an issue, but most scales and most of the stats we commonly

use 

>> are extremely robust. We may bend the prediction but we're unlikely

to break 

>> it. 

>> 

>> Given another hundred years to work on this, measurement of employee 

>> phlogistan could conceivably evolve into something more direct and

more 

>> definitive . . . I mean, everybody knows that variations in the long

arm of 

>> chromsome 6 of the human genome account for 4% of the variance in 

>> intelligence. See, we're on the way. 

>> Jeff 

>> ----------------------------------------- 

>> Dr. Jeffrey P Feuquay, I/O Psychologist& Attorney 

>> Managing Consultant, Psychology-Law Center, LLC 

>> 108 W. Walnut, Nevada, Mo 64772 

>> ofc: 417-667-5076 cell: 417-549-0997 

>> 

>> 

>> On Wed, Sep 1, 2010 at 11:37 AM, Mark Hammerwrote: 

>> 

>>> Would it be wrong to suggest that this is one of those perpetual 

>>> discrepancies between theoreticians and clinicians? In this case,

the 

>>> discrepancy is between test/measurement specialists (the

theoreticians), 

>>> and test-users (the clinicians). 

>>> 

>>> Mark Hammer 

>>> Ottawa 

>

This e-mail message is intended for the named recipient(s) and may
contain information that is privileged, confidential and/or exempt from
disclosure under applicable law. Unauthorized disclosure, copying or
re-transmission is prohibited. If you are not a named recipient or not
authorized by the named recipient(s), or if you have received this
e-mail in error, then please notify the sender immediately and delete
the message and any copies.
 >
Ce courriel est destiné exclusivement au destinataire mentionné en titre
et peut contenir de l'information privilégiée, confidentielle ou
soustraite à la communication aux termes des lois applicables. Toute
divulgation non autorisée, toute reproduction ou réacheminement est
interdit. Si vous n'êtes pas le destinataire de ce courriel, ou n'êtes
pas autorisé par le destinataire visé, ou encore, si vous l'avez reçu
par erreur, veuillez le mentionner immédiatement à l'expéditeur et
supprimer le courriel et les copies.