[IPAC-List] Speed vs Power Tests

Michael McDaniel (WSF) McDaniel at workskillsfirst.com
Thu Jun 2 17:22:20 EDT 2016

Most tests have time limits even if they are not intended to be speed test
(in assembled testing, the test administrator needs to go home

If the test is presented such that the items are arranged by difficulty,
the items the applicant never gets to answer due to the time limit have a
lower probability of being answered correctly by the applicant than the
questions that the applicant completed. This potentially minimizes any
detrimental effects of speed. Example of such tests are the cognitive tests
on the GATB (one form of which is now called The Ability Profiler [
https://www.onetcenter.org/AP.html] and is freely available). Another
example is the Wonderlic cognitive ability tests.

Speed may also be job-related. If a job requires reading for some or many
tasks and the applicant reads slower than most, the applicant, if hired,
will likely complete their job assignments more slowly and thus their job
performance will suffer, on average. In such a scenario, speed is not an
undesirable characteristic of the test if one is interested in hiring the
applicants with the highest probability of being a well-performing

On the other hand, if job-related reading speed has undesirable
consequences such as group differences, one may wish to sacrifice merit
hiring for diversity hiring and increase the time limit of the exam.  This
will improve the diversity of hires, on average, and will cause an increase
in group job performance differences on average.  As long as there are
group differences in job-related abilities, we will face this
validity-diversity dilemma.

I infer that EEOC sees no upside to acknowledging the validity-diversity
dilemma. Parties interested in learning more about the validity-diversity
dilemma may wish to access these papers:

McDaniel, M.A. & Kepes, S. (2014). An evaluation of Spearman’s Hypothesis
by manipulating *g* saturation. *International Journal of Selection and
Assessment, 22, *333-342*.*

McDaniel, M.A., Kepes, S., Banks, G. C. (2011). The *Uniform Guidelines*
are a detriment to the field of personnel selection. *Industrial and
Organizational Psychology: Perspectives on Science and Practice, 4, *419-514


McDaniel, M.A., Kepes, S., Banks, G. C. (2011). Encouraging debate on the
Uniform Guidelines and the disparate impact theory of discrimination.
and Organizational Psychology: Perspectives on Science and Practice, 4, *

The second paper in the list resulted in several commentaries that may be
of interest as well (you would need access to that journal). Those
commentaries are commented upon in the third paper.

I acknowledge that this e-mail is, in part, a self-promotion of my own

Best wishes,


On Thu, Jun 2, 2016 at 12:13 PM, Joel Wiesen <
jwiesen at appliedpersonnelresearch.com> wrote:

> Mike,
> If speed is relevant for some jobs, as Winfred points out, and if speed
> shows no mean difference by ethnic/racial group, we are missing an
> opportunity to reduce adverse impact.
> Do you have a reference for the lack of difference in speed or was that
> part of the IOOB unpublished presentation?
> Thx.
> Joel
> P.S.  Rich, your restatement of my question correctly captured my
> motivation in asking the question.
> - -
> Joel P. Wiesen, Ph.D., Director
> Applied Personnel Research
> 62 Candlewood Road
> Scarsdale, NY 10583-6040
> http://www.linkedin.com/in/joelwiesen
> (617) 244-8859
> http://appliedpersonnelresearch.com
> Note: This e-mail and any attachments may contain confidential and/or
> legally privileged information. Please do not forward any contents without
> permission. If you have received this message in error please destroy all
> copies, completely remove it from your computer, and notify the sender.
> Thank you.
> On 6/1/16 8:18 AM, Aamodt, Mike wrote:
>> The discussion on speed versus power tests reminded me of some research
>> that four of my graduate students (Dave Sharrer, Marcelle Clavette,
>> Jeanne Donaghy, & Rob Lytle) did back in 2010 (they presented their
>> research at IOOB but did not write a formal paper).  They were
>> investigating a construct called “personal tempo” which relates to the
>> speed with which we complete daily tasks such as talking, walking,
>> brushing teeth, and taking tests.  The idea here is that some people do
>> things quickly and others take more time.
>> Given that the research on test-taking times shows a negligible
>> relationship between time to complete an exam and performance on the
>> exam, the students wondered if test-taking speed was a function of
>> personal tempo rather than cognitive ability or knowledge. For those who
>> have taught, you have probably noticed that the same students finish all
>> of their exams way ahead of the class and there are always two or three,
>> bless their hearts, who stay to the bitter end of a three-hour class.
>> Some interesting findings:
>> -          - A meta-analysis of 34 studies yielded a mean correlation of
>> .00 between order of finish and test scores
>>  -
>> -           - Test completion times were similar for whites and
>> minorities and men and women
>> -
>> -          -  Test completion times across four exams were highly
>> correlated (this is consistent with meta-analytic results)
>> -          - The concept of personal tempo was not unidimensional: It
>> had six factors (personal hygiene, classroom behavior, communication,
>> motor skills, ordering food, and decision making)
>> -          Only the classroom behavior factor was correlated with
>> test-taking times
>> Not sure if this info is helpful but it was a nice trip down memory lane
>> for me J
>> _________________________________________
>> Michael G. Aamodt, Ph.D. (Mike)
>> Professor Emeritus
>> Department of Psychology
>> Radford University
>> Radford, VA  24142-6946
>> (202) 280-2172
>> maamodt at radford.edu <mailto:maamodt at radford.edu>
>> ------------------------------------------------------------------------
>> *From:* IPAC-List [ipac-list-bounces at ipacweb.org] on behalf of Winfred
>> Arthur, Jr. [w-arthur at tamu.edu]
>> *Sent:* Tuesday, May 31, 2016 9:29 PM
>> *To:* ipac-list at ipacweb.org; RICHARD.TONOWSKI at EEOC.GOV
>> *Subject:* Re: [IPAC-List] Speed vs Power Tests
>> Richard, i think the speed/power continuum is just another test
>> characteristic along which (the administration of) tests vary.  however,
>> it is important that where a test falls on this continuum be consonant
>> w/ the intended construct and the inferences one wants to draw from the
>> test scores.  hence, one would typically expect a typing test (to extent
>> that we still have those) or a data entry test to be more on the speed
>> end of things b/c there is typically an interest in making inferences
>> about how quickly (and accurately) a person can type or enter data.
>> likewise, 1st responders, such as EMT or paramedics, typically have to
>> access walking knowledge and make decisions based thereon fairly
>> rapidly.  consequently, most job analyses would support having tests
>> that assess this activity fall more on the speed end of the continuum.
>> in shape contrast, our comprehensive exams for our phd candidates (which
>> are not atypical) are multi-day exams, somewhat reflecting the extreme
>> power end of the continuum, b/c we as a program have decided that this
>> more accurately reflects how the students will typically be originating
>> solutions to the problems and queries they encounter as scientists and
>> scholars, and yes, as practitioners as well.
>> of course, the speed/power continuum, which has to do specifically w/
>> the amt of time provided to the test taker to complete an assessment, is
>> quite different from constructs such as "processing speed", "perceptual
>> speed", etc., which are specific abilities under the general ability
>> (i.e., g) construct.
>> thus, in answer to your question Richard, whether "the speed component
>> is construct contamination or a bonus for g-loaded constructs that are
>> supposedly being measured" again depends on the intended construct.  if
>> i have a specific ability measure of processing speed or perceptual
>> speed, or even my EMT/paramedic example, then speeding the test will be
>> aligned w/ the intended construct and there not a contaminate.  however,
>> in my comprehensive exam example, if we gave students an hr to dsgn a
>> study and write w/ write up the associated rsch proposal as an NSF grant
>> application --- yes, i know i am being extreme  :)  ---  then the speed
>> component would definitely be construct-irrelevant and therefore a
>> contaminant.
>> is this helpful?
>> - winfred
>> On 5/31/2016 7:42 PM, RICHARD TONOWSKI wrote:
>>> I have a vague recollection that Jack Hunter did a factor analysis of
>>> some test (GATB?  ASVAB?) where, to his surprise, he found a speed
>>> factor independent of /g/.  It was in a research report; I haven't
>>> seen this mentioned in a journal article.
>>> While trying to find this, I noted that John Carrol's three-stratum
>>> model (with its fluid and crystallized components) had "broad
>>> cognitive speed" and "processing speed" as constructs.  Speed may
>>> involve different constructs to be measured.  I'm not familiar with
>>> the model, but it seems that everything interrelates, with /g/ as the
>>> superordinate stratum.
>>> There's more I stumbled across regarding what is meant by speed or
>>> speededness.  This from W.J. van der Linden (2011).  Test design and
>>> speededness. /Journal of Educational Measurement/, /48/, 44-60:
>>> "The notion of speededness in testing refers to an interaction between
>>> three important factors: the cognitive speed at which the test taker
>>> works during the test, the amount of labor required by the items, and
>>> the time limit on the test. A test is more speeded when a test taker
>>> has to work faster, answering the items requires more labor,and/or the
>>> time limit is tightened. As the speed of the test taker is one of
>>> these factors, it actually is incorrect to refer to the speededness of
>>> a test."
>>> The issue here seems to be that introduction of speededness may
>>> introduce a construct other than what was intended.  He continues with
>>> an example of someone with a disability not related to the intended
>>> construct, but who takes the test slower than others because of the
>>> disability.  The speed measured in the test situation may not be the
>>> speed in using the intended construct (assuming that speed is intended
>>> to be included in the measurement).
>>> Taking the liberty to rephrase Joel's questions, I'm wondering if the
>>> speed component is construct contamination or a bonus for /g/-loaded
>>> constructs that are supposedly being measured.
>>> Somebody please provide answers!
>>> Rich
>>> .
>>> >>> <mhammer at 295.ca> 5/31/2016 6:53 PM >>>
>>> And in keeping with Winfred's comments, the format of each can provide
>>> one
>>> with different sorts of measurement of the construct in question; speed
>>> tests being prone to multiple-choice and power more amenable to
>>> open-ended.
>>> Insomuch as speed tests are often a product of convenience contingencies
>>> (e.g., automated scoring), I would think the real question is whether
>>> power tests increment to validity over speed tests.  After all, if there
>>> were a reasonably large number of testees to process, one would opt for a
>>> speed test first since it is less labor-intensive as far as scoring. And
>>> once you have those scores, does an additional power test of some kind
>>> tell you more?
>>> Mark Hammer
>>> Ottawa
>>> > Joel, i think it would have to be a function of the construct assessed,
>>> > right?  so, if by "To what extent do speed and power tests measure the
>>> > same constructs?" you mean the _*same test*_ administered under speeded
>>> > and power conditions, then the answer would have to be "no" b/c
>>> > additional variance, maybe construct-relevant or construct-irrelevant
>>> > has been introduced into the test scores.  consequently, one would
>>> > expect the convergence b/n the two sets of scores to be low
>>> >
>>> > and "Do speed tests add validity over power tests?" => again, it will
>>> > seem the answer would depend on the constructs assessed by the two
>>> > tests.  thus, for example, tests of processing speed and reaction time,
>>> > are by definition speeded tests; whereas for example tests of fluid
>>> > intelligence (e.g., the Ravens) and declarative knowledge are typically
>>> > better conceptualized as being closer to the power end of the
>>> continuum.
>>> >
>>> > hope this helpful?
>>> >
>>> > - winfred
>>> >
>>> > On 5/31/2016 12:34 PM, Joel Wiesen wrote:
>>> >> What is the current thinking on these questions?
>>> >>
>>> >> To what extent do speed and power tests measure the same constructs?
>>> >>
>>> >> Do speed tests add validity over power tests?
>>> >>
>>> >> Thx
>>> >>
>>> >> Joel
>>> >>
>>> >>
>>> >
>>> > _______________________________________________________
>>> > IPAC-List
>>> > IPAC-List at ipacweb.org
>>> > https://pairlist9.pair.net/mailman/listinfo/ipac-list
>>> >
>>> _______________________________________________________
>>> IPAC-List
>>> IPAC-List at ipacweb.org
>>> https://pairlist9.pair.net/mailman/listinfo/ipac-list
>>> _______________________________________________________
>>> IPAC-List
>>> IPAC-List at ipacweb.org
>>> https://pairlist9.pair.net/mailman/listinfo/ipac-list
>> _______________________________________________________
>> IPAC-List
>> IPAC-List at ipacweb.org
>> https://pairlist9.pair.net/mailman/listinfo/ipac-list
>> _______________________________________________________
> IPAC-List
> IPAC-List at ipacweb.org
> https://pairlist9.pair.net/mailman/listinfo/ipac-list

Michael A. McDaniel, Ph.D.
Work Skills First, Inc.
Voice: 804-277-9730
E-Mail: McDaniel at WorkSkillsFirst.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist9.pair.net/pipermail/ipac-list/attachments/20160602/d745546f/attachment-0001.html>

More information about the IPAC-List mailing list