[IPAC-List] Speed vs Power Tests

Tue May 31 21:29:35 EDT 2016

Richard, i think the speed/power continuum is just another test 
characteristic along which (the administration of) tests vary. however, 
it is important that where a test falls on this continuum be consonant 
w/ the intended construct and the inferences one wants to draw from the 
test scores.  hence, one would typically expect a typing test (to extent 
that we still have those) or a data entry test to be more on the speed 
end of things b/c there is typically an interest in making inferences 
about how quickly (and accurately) a person can type or enter data.  
likewise, 1st responders, such as EMT or paramedics, typically have to 
access walking knowledge and make decisions based thereon fairly 
rapidly.  consequently, most job analyses would support having tests 
that assess this activity fall more on the speed end of the continuum.

in shape contrast, our comprehensive exams for our phd candidates (which 
are not atypical) are multi-day exams, somewhat reflecting the extreme 
power end of the continuum, b/c we as a program have decided that this 
more accurately reflects how the students will typically be originating 
solutions to the problems and queries they encounter as scientists and 
scholars, and yes, as practitioners as well.

of course, the speed/power continuum, which has to do specifically w/ 
the amt of time provided to the test taker to complete an assessment, is 
quite different from constructs such as "processing speed", "perceptual 
speed", etc., which are specific abilities under the general ability 
(i.e., g) construct.

thus, in answer to your question Richard, whether "the speed component 
is construct contamination or a bonus for g-loaded constructs that are 
supposedly being measured" again depends on the intended construct.  if 
i have a specific ability measure of processing speed or perceptual 
speed, or even my EMT/paramedic example, then speeding the test will be 
aligned w/ the intended construct and there not a contaminate.  however, 
in my comprehensive exam example, if we gave students an hr to dsgn a 
study and write w/ write up the associated rsch proposal as an NSF grant 
application --- yes, i know i am being extreme  :)  ---  then the speed 
component would definitely be construct-irrelevant and therefore a 
contaminant.

is this helpful?

- winfred

On 5/31/2016 7:42 PM, RICHARD TONOWSKI wrote:
> I have a vague recollection that Jack Hunter did a factor analysis of 
> some test (GATB?  ASVAB?) where, to his surprise, he found a speed 
> factor independent of /g/.  It was in a research report; I haven't 
> seen this mentioned in a journal article.
> While trying to find this, I noted that John Carrol's three-stratum 
> model (with its fluid and crystallized components) had "broad 
> cognitive speed" and "processing speed" as constructs.  Speed may 
> involve different constructs to be measured.  I'm not familiar with 
> the model, but it seems that everything interrelates, with /g/ as the 
> superordinate stratum.
> There's more I stumbled across regarding what is meant by speed or 
> speededness.  This from W.J. van der Linden (2011). Test design and 
> speededness. /Journal of Educational Measurement/, /48/, 44-60:
>
> "The notion of speededness in testing refers to an interaction between 
> three important factors: the cognitive speed at which the test taker 
> works during the test, the amount of labor required by the items, and 
> the time limit on the test. A test is more speeded when a test taker 
> has to work faster, answering the items requires more labor,and/or the 
> time limit is tightened. As the speed of the test taker is one of 
> these factors, it actually is incorrect to refer to the speededness of 
> a test."
> The issue here seems to be that introduction of speededness may 
> introduce a construct other than what was intended.  He continues with 
> an example of someone with a disability not related to the intended 
> construct, but who takes the test slower than others because of the 
> disability.  The speed measured in the test situation may not be the 
> speed in using the intended construct (assuming that speed is intended 
> to be included in the measurement).
> Taking the liberty to rephrase Joel's questions, I'm wondering if the 
> speed component is construct contamination or a bonus for /g/-loaded 
> constructs that are supposedly being measured.
> Somebody please provide answers!
> Rich
> .
>
> >>> <mhammer at 295.ca> 5/31/2016 6:53 PM >>>
> And in keeping with Winfred's comments, the format of each can provide one
> with different sorts of measurement of the construct in question; speed
> tests being prone to multiple-choice and power more amenable to
> open-ended.
>
> Insomuch as speed tests are often a product of convenience contingencies
> (e.g., automated scoring), I would think the real question is whether
> power tests increment to validity over speed tests.  After all, if there
> were a reasonably large number of testees to process, one would opt for a
> speed test first since it is less labor-intensive as far as scoring. And
> once you have those scores, does an additional power test of some kind
> tell you more?
>
> Mark Hammer
> Ottawa
>
> > Joel, i think it would have to be a function of the construct assessed,
> > right?  so, if by "To what extent do speed and power tests measure the
> > same constructs?" you mean the _*same test*_ administered under speeded
> > and power conditions, then the answer would have to be "no" b/c
> > additional variance, maybe construct-relevant or construct-irrelevant
> > has been introduced into the test scores.  consequently, one would
> > expect the convergence b/n the two sets of scores to be low
> >
> > and "Do speed tests add validity over power tests?" => again, it will
> > seem the answer would depend on the constructs assessed by the two
> > tests.  thus, for example, tests of processing speed and reaction time,
> > are by definition speeded tests; whereas for example tests of fluid
> > intelligence (e.g., the Ravens) and declarative knowledge are typically
> > better conceptualized as being closer to the power end of the continuum.
> >
> > hope this helpful?
> >
> > - winfred
> >
> > On 5/31/2016 12:34 PM, Joel Wiesen wrote:
> >> What is the current thinking on these questions?
> >>
> >> To what extent do speed and power tests measure the same constructs?
> >>
> >> Do speed tests add validity over power tests?
> >>
> >> Thx
> >>
> >> Joel
> >>
> >>
> >
> > _______________________________________________________
> > IPAC-List
> > IPAC-List at ipacweb.org
> > https://pairlist9.pair.net/mailman/listinfo/ipac-list
> >
>
>
> _______________________________________________________
> IPAC-List
> IPAC-List at ipacweb.org
> https://pairlist9.pair.net/mailman/listinfo/ipac-list
>
>
> _______________________________________________________
> IPAC-List
> IPAC-List at ipacweb.org
> https://pairlist9.pair.net/mailman/listinfo/ipac-list

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist9.pair.net/pipermail/ipac-list/attachments/20160531/a3e2d639/attachment-0001.html>