[IPAC-List] Speed vs Power Tests

Thu Jun 2 12:13:52 EDT 2016

Mike,

If speed is relevant for some jobs, as Winfred points out, and if speed 
shows no mean difference by ethnic/racial group, we are missing an 
opportunity to reduce adverse impact.

Do you have a reference for the lack of difference in speed or was that 
part of the IOOB unpublished presentation?

Thx.

Joel

P.S.  Rich, your restatement of my question correctly captured my 
motivation in asking the question.

- -
Joel P. Wiesen, Ph.D., Director
Applied Personnel Research
62 Candlewood Road
Scarsdale, NY 10583-6040
http://www.linkedin.com/in/joelwiesen
(617) 244-8859
http://appliedpersonnelresearch.com

Note: This e-mail and any attachments may contain confidential and/or 
legally privileged information. Please do not forward any contents 
without permission. If you have received this message in error please 
destroy all copies, completely remove it from your computer, and notify 
the sender. Thank you.

On 6/1/16 8:18 AM, Aamodt, Mike wrote:
>
> The discussion on speed versus power tests reminded me of some research
> that four of my graduate students (Dave Sharrer, Marcelle Clavette,
> Jeanne Donaghy, & Rob Lytle) did back in 2010 (they presented their
> research at IOOB but did not write a formal paper).  They were
> investigating a construct called “personal tempo” which relates to the
> speed with which we complete daily tasks such as talking, walking,
> brushing teeth, and taking tests.  The idea here is that some people do
> things quickly and others take more time.
>
>
> Given that the research on test-taking times shows a negligible
> relationship between time to complete an exam and performance on the
> exam, the students wondered if test-taking speed was a function of
> personal tempo rather than cognitive ability or knowledge. For those who
> have taught, you have probably noticed that the same students finish all
> of their exams way ahead of the class and there are always two or three,
> bless their hearts, who stay to the bitter end of a three-hour class.
>
>
> Some interesting findings:
>
> -          - A meta-analysis of 34 studies yielded a mean correlation of
> .00 between order of finish and test scores
>
>  -
>
> -           - Test completion times were similar for whites and
> minorities and men and women
>
> -
>
> -          -  Test completion times across four exams were highly
> correlated (this is consistent with meta-analytic results)
>
>
> -          - The concept of personal tempo was not unidimensional: It
> had six factors (personal hygiene, classroom behavior, communication,
> motor skills, ordering food, and decision making)
>
> -          Only the classroom behavior factor was correlated with
> test-taking times
>
>
> Not sure if this info is helpful but it was a nice trip down memory lane
> for me J
>
> _________________________________________
>
> Michael G. Aamodt, Ph.D. (Mike)
> Professor Emeritus
> Department of Psychology
> Radford University
> Radford, VA  24142-6946
> (202) 280-2172
> maamodt at radford.edu <mailto:maamodt at radford.edu>
>
>
> ------------------------------------------------------------------------
> *From:* IPAC-List [ipac-list-bounces at ipacweb.org] on behalf of Winfred
> Arthur, Jr. [w-arthur at tamu.edu]
> *Sent:* Tuesday, May 31, 2016 9:29 PM
> *To:* ipac-list at ipacweb.org; RICHARD.TONOWSKI at EEOC.GOV
> *Subject:* Re: [IPAC-List] Speed vs Power Tests
>
> Richard, i think the speed/power continuum is just another test
> characteristic along which (the administration of) tests vary.  however,
> it is important that where a test falls on this continuum be consonant
> w/ the intended construct and the inferences one wants to draw from the
> test scores.  hence, one would typically expect a typing test (to extent
> that we still have those) or a data entry test to be more on the speed
> end of things b/c there is typically an interest in making inferences
> about how quickly (and accurately) a person can type or enter data.
> likewise, 1st responders, such as EMT or paramedics, typically have to
> access walking knowledge and make decisions based thereon fairly
> rapidly.  consequently, most job analyses would support having tests
> that assess this activity fall more on the speed end of the continuum.
>
> in shape contrast, our comprehensive exams for our phd candidates (which
> are not atypical) are multi-day exams, somewhat reflecting the extreme
> power end of the continuum, b/c we as a program have decided that this
> more accurately reflects how the students will typically be originating
> solutions to the problems and queries they encounter as scientists and
> scholars, and yes, as practitioners as well.
>
> of course, the speed/power continuum, which has to do specifically w/
> the amt of time provided to the test taker to complete an assessment, is
> quite different from constructs such as "processing speed", "perceptual
> speed", etc., which are specific abilities under the general ability
> (i.e., g) construct.
>
> thus, in answer to your question Richard, whether "the speed component
> is construct contamination or a bonus for g-loaded constructs that are
> supposedly being measured" again depends on the intended construct.  if
> i have a specific ability measure of processing speed or perceptual
> speed, or even my EMT/paramedic example, then speeding the test will be
> aligned w/ the intended construct and there not a contaminate.  however,
> in my comprehensive exam example, if we gave students an hr to dsgn a
> study and write w/ write up the associated rsch proposal as an NSF grant
> application --- yes, i know i am being extreme  :)  ---  then the speed
> component would definitely be construct-irrelevant and therefore a
> contaminant.
>
> is this helpful?
>
> - winfred
>
> On 5/31/2016 7:42 PM, RICHARD TONOWSKI wrote:
>> I have a vague recollection that Jack Hunter did a factor analysis of
>> some test (GATB?  ASVAB?) where, to his surprise, he found a speed
>> factor independent of /g/.  It was in a research report; I haven't
>> seen this mentioned in a journal article.
>>
>> While trying to find this, I noted that John Carrol's three-stratum
>> model (with its fluid and crystallized components) had "broad
>> cognitive speed" and "processing speed" as constructs.  Speed may
>> involve different constructs to be measured.  I'm not familiar with
>> the model, but it seems that everything interrelates, with /g/ as the
>> superordinate stratum.
>>
>> There's more I stumbled across regarding what is meant by speed or
>> speededness.  This from W.J. van der Linden (2011).  Test design and
>> speededness. /Journal of Educational Measurement/, /48/, 44-60:
>>
>> "The notion of speededness in testing refers to an interaction between
>> three important factors: the cognitive speed at which the test taker
>> works during the test, the amount of labor required by the items, and
>> the time limit on the test. A test is more speeded when a test taker
>> has to work faster, answering the items requires more labor,and/or the
>> time limit is tightened. As the speed of the test taker is one of
>> these factors, it actually is incorrect to refer to the speededness of
>> a test."
>>
>> The issue here seems to be that introduction of speededness may
>> introduce a construct other than what was intended.  He continues with
>> an example of someone with a disability not related to the intended
>> construct, but who takes the test slower than others because of the
>> disability.  The speed measured in the test situation may not be the
>> speed in using the intended construct (assuming that speed is intended
>> to be included in the measurement).
>>
>> Taking the liberty to rephrase Joel's questions, I'm wondering if the
>> speed component is construct contamination or a bonus for /g/-loaded
>> constructs that are supposedly being measured.
>>
>> Somebody please provide answers!
>>
>> Rich
>>
>>
>> .
>>
>> >>> <mhammer at 295.ca> 5/31/2016 6:53 PM >>>
>> And in keeping with Winfred's comments, the format of each can provide one
>> with different sorts of measurement of the construct in question; speed
>> tests being prone to multiple-choice and power more amenable to
>> open-ended.
>>
>> Insomuch as speed tests are often a product of convenience contingencies
>> (e.g., automated scoring), I would think the real question is whether
>> power tests increment to validity over speed tests.  After all, if there
>> were a reasonably large number of testees to process, one would opt for a
>> speed test first since it is less labor-intensive as far as scoring. And
>> once you have those scores, does an additional power test of some kind
>> tell you more?
>>
>> Mark Hammer
>> Ottawa
>>
>> > Joel, i think it would have to be a function of the construct assessed,
>> > right?  so, if by "To what extent do speed and power tests measure the
>> > same constructs?" you mean the _*same test*_ administered under speeded
>> > and power conditions, then the answer would have to be "no" b/c
>> > additional variance, maybe construct-relevant or construct-irrelevant
>> > has been introduced into the test scores.  consequently, one would
>> > expect the convergence b/n the two sets of scores to be low
>> >
>> > and "Do speed tests add validity over power tests?" => again, it will
>> > seem the answer would depend on the constructs assessed by the two
>> > tests.  thus, for example, tests of processing speed and reaction time,
>> > are by definition speeded tests; whereas for example tests of fluid
>> > intelligence (e.g., the Ravens) and declarative knowledge are typically
>> > better conceptualized as being closer to the power end of the continuum.
>> >
>> > hope this helpful?
>> >
>> > - winfred
>> >
>> > On 5/31/2016 12:34 PM, Joel Wiesen wrote:
>> >> What is the current thinking on these questions?
>> >>
>> >> To what extent do speed and power tests measure the same constructs?
>> >>
>> >> Do speed tests add validity over power tests?
>> >>
>> >> Thx
>> >>
>> >> Joel
>> >>
>> >>
>> >
>> > _______________________________________________________
>> > IPAC-List
>> > IPAC-List at ipacweb.org
>> > https://pairlist9.pair.net/mailman/listinfo/ipac-list
>> >
>>
>>
>> _______________________________________________________
>> IPAC-List
>> IPAC-List at ipacweb.org
>> https://pairlist9.pair.net/mailman/listinfo/ipac-list
>>
>>
>> _______________________________________________________
>> IPAC-List
>> IPAC-List at ipacweb.org
>> https://pairlist9.pair.net/mailman/listinfo/ipac-list
>
>
>
> _______________________________________________________
> IPAC-List
> IPAC-List at ipacweb.org
> https://pairlist9.pair.net/mailman/listinfo/ipac-list
>