[IPAC-List] Michael McDaniel's Reference to the so-called Validity-Diversity Dilemma

Sat Jun 4 20:08:24 EDT 2016

Hi Lance,

That was a long time ago, Lance.  The PRDC group was terrific.  I got to attend periodic briefings by people like Bill Gorham and our hero in PRDC, Frank Schmidt, as well as others who were terrifically skilled and knowledgeable.  They were all great.

Regarding the Uniform Guidelines, most of what I remember is the feeling that we were under seige, and that merit hiring was being put to the test.  I remember a lot of debates internally about banding or other approaches that would expand the number of people certified, such as the rule of reliability (Michigan used this, as I recall), and while I believe we were open to more latitude than a rule of 3, our main impetus was to keep a "reasonable" certification rule (e.g., 3, 5, 7).  OPM as an agency was totally opposed to a rule of the list (for ranking devices), and totally opposed to selective certification based on race.  Beyond this, I don't have any specific recollection of a position taken on banding in writing or otherwise, but based on my understanding of our overall stance, I'm pretty sure PRDC was against banding.  The idea that tests weren't perfectly reliable or valid didn't come as news to any of our psychologists, thus the idea that tests weren't perfect didn't convince anyone that I knew that we should therefore greatly expand certification rules.  Basically, we were all committed to the concepts of individual merit and test utility.  You might say we were pretty effectively indoctrinated, or you might say we were trained very well.   Take your pick.

Regarding job analysis provisions in the Uniform Guidelines, I know that Ernie Primoff and other adherents of our job element job analysis method weren't happy because identifying job tasks was not necessary in the job element system.  Our X-118C staffing guide listed the exam plan (the job elements) for every blue-collar occupation in the federal system, and everyone thought that the system used to arrive at those elements was very sound -- yet if you bought into the Uniform Guidelines, you had to conclude that our job element job analysis system was flawed.  I think our PRDC group thought the job element method was fine as practiced, and I agree with that.  However...we didn't hold firm on this.

As I recall, our people were resistant to a lot of the early pressure by EEOC for organizations to have to conduct differential validity studies in order to meet federal standards.  I think that was one of their initial negotiating positions.  It was my understanding that our OPM team fought to have three co-equal validation methods, and they prevailed over the EEOC on this.  Their main argument, I was told at the time, was that most government agencies could not conduct criterion-related validity studies, much less differential validity studies, owing to sample size limitations and they wouldn't budge on the need to let government agencies use content validity in order to be in compliance.  So, while holding firm on this, I believe we did give in on things like the need to include tasks in any job analysis and link them to KSA's (thus undermining our own job element job analysis system).

Overall, my understanding was that a series of compromises were made to reach agreement and finally issue the Uniform Guidelines.  Frankly, the process of developing the guidelines went on so long that everyone was sick of it.  So, we got some things, but agreed to other things that we didn't think were truly appropriate.

This was a very exciting era.  PRDC was a terrific organization.  PRDC produced many outstanding publications on a variety of testing topics -- truly a great outfit under Gorham's leadership.  I just want to add that I got my initial training in assessment centers from the PRDC assessment team headed up by Dale Baker.  What a great resource they were for those of us out in the boondocks!

  ----- Original Message ----- 
  From: Lance Seberhagen 
  To: Richard Joines ; IPAC List 
  Sent: Saturday, June 04, 2016 12:55 PM
  Subject: Re: [IPAC-List] Michael McDaniel's Reference to the so-called Validity-Diversity Dilemma

  Hi Richard:

  PRDC had a great staff, and the 1970s were important in I/O history.  As a former OPM Regional Psychologist during that time, can you share any other battles that PRDC had with EEOC over the development of the Uniform Guidelines and EEO compliance?  For example, weren't Bill Gorham and PRDC opposed to category ranking (aka "banding") as a way to reduce unnecessary adverse impact?

  Lance S.

  On 6/4/2016 2:48 PM, Richard Joines wrote:

    Hi Lance:

    You're absolutely correct, and as an OPM Regional Psychologist located in S.F. during that era, we got weekly reports on the battles and skirmishes taking place with EEOC.  Our PRDC group did the best they could to block some of the more extreme measures that the EEOC wanted...one sore point with PRDC was the battle over whether or not tasks had to be identified with linkages to KSA's.  We didn't do that in our job element method for blue-collar jobs, and our method was very successful, but finally...well, you know that some compromises had to be made, and they weren't all based on the science that existed at that point in time.  People entering our field today must believe that there were some really great scientific reasons for all of the decisions reflected in the Uniform Guidelines, else why would those decisions be promulgated???

    And as I'm sure you know, there is nothing in the Uniform Guidelines about a validity-diversity dilemma, thankfully.

      ----- Original Message ----- 
      From: Lance Seberhagen 
      To: Richard Joines ; IPAC List 
      Sent: Saturday, June 04, 2016 11:06 AM
      Subject: Re: [IPAC-List] Michael McDaniel's Reference to the so-called Validity-Diversity Dilemma

      Lest we forget, didn't Bill Gorham and PRDC help to write the...

      UNIFORM GUIDELINES ON EMPLOYEE SELECTION PROCEDURES

      § 1607.1 Statement of purpose.

      A. Need for uniformity—Issuing agencies. The Federal government's need for a uniform set of principles on the question of the use of tests and other selection procedures has long been recognized. The Equal Employment Opportunity Commission, the Civil Service Commission, the Department of Labor, and the Department of Justice jointly have adopted these uniform guidelines to meet that need, and to apply the same principles to the Federal Government as are applied to other employers.

      Lance S.

      On 6/4/2016 1:50 PM, Richard Joines wrote:

        Hi Mark, 

          From the Taylor-Russell Tables: 

        If you increase the test time allowed and this reduces validity from .5 to .4 where you have a selection ratio of .20 and current percent successful on the job at 50%, you're talking approximately a 10% reduction in the overall success rate that the test produces (78% down to 73%, where 5% reduction/50% base rate=10% reduction in success rate).  Why not accept a 25% or 50 reduction? 

        You indicate it wouldn't be ok to drop validity from .6 to .2, which would basically result in dropping the success rate from 84% to 61% of those hired (46% reduction in success rate), but maybe someone else would say that's fine.  Why isn't this ok with you?  Seriously, why not? 

        You have to forgive me for how I think about these issues...I was successfully indoctrinated by OPM during the 1970's where our 80+ psychologists (in OPM's Personnel Research and Development Center), led by Bill Gorham, and greatly assisted by Frank Schmidt and others, debunked most of what was coming out of the EEOC.  Well, temporarily debunked it...looks like we've got a sequel, The EEOC Strikes Back. 

        ----- Original Message ----- From: <mhammer at 295.ca> 
        To: <ipac-list at ipacweb.org> 
        Sent: Friday, June 03, 2016 3:19 PM 
        Subject: Re: [IPAC-List] Michael McDaniel's Reference to the so-called Validity-Diversity Dilemma 

          I liked Dan's post very much.  The magnitude of the forfeiture is 
          important in considering the balance.  If bending timing drops predictive 
          validity from .5 down to .42, but improves representativeness, I'm okay 
          with that.  If we're talking a drop from .6 to .2, that's a whole other 
          story. 

          _______________________________________________________ 
          IPAC-List 
          IPAC-List at ipacweb.org 
          https://pairlist9.pair.net/mailman/listinfo/ipac-list 

        _______________________________________________________ 
        IPAC-List 
        IPAC-List at ipacweb.org 
        https://pairlist9.pair.net/mailman/listinfo/ipac-list 

           Virus-free. www.avast.com  

_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
https://pairlist9.pair.net/mailman/listinfo/ipac-list

       Virus-free. www.avast.com  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist9.pair.net/pipermail/ipac-list/attachments/20160604/2df0dbf2/attachment-0001.html>