[IPAC-List] promotional exam strategy

[Mark Hammer]

I used to use something like that for teaching as well.  I'd list all
the concepts I wanted students to know, and terms I wanted them to be
familiar with, and inform them that this was the master list, from which
I would draw questions, hopefully packing as much of that into a
predefined number of questions as I could.  (In some instances, the
distractors also require you to understand the concept, just so you can
classify it as a distractor and invalid response, such that one question
can actually address much more than just the material in the correct
response.) 
 
The advantage it provides to testees is that it defines what they
*don't* have to worry about when preparing, and they appreciate that. 
Assuming the range of information covered off by the list of stems is of
an appropriate size compared to the final list of questions (which in
this case it seems to be, 5:1), it will be perceived as fair by testees,
and also gives you some confidence that they've spent some time learning
about things that maybe you'd like them to know, but don't have
time/space to test them on.
 
The caveat is that you'd want some assurances that any given subset of
100 was roughly equal in both difficulty and predictive value.  Y
certainly don't want to find yourself with mean scores that fluctuate
from session/class to session/class.  And that entails a bit of work on
your part.
 
Mark Hammer
Ottawa
 

>>> <Harry.Brull at pdininthhouse.com> 2011/06/28 2:44 PM >>>

This would seem to me as a variant on providing a reading list as a
study guide. In this case, the jurisdiction is directing the
candidates
to the issues (items) that are considered important.
I have used a comparable technique in my collegiate teaching.
As for what to call it - beyond an inspired idea - I'd suggest
"item-based exam preparation/study guide".

As for the psychometric properties, I see no reason to expect anything
different than ant content-based multiple choice test - high
coefficient
alpha.
As for what you are really measuring, some combination of job
knowledge
and a healthy does of "G".

Harry Brull
H.A.R. 
PDI Ninth House 


------------------------------------------------
Harry Brull |  Senior Vice-President 
PDI Ninth House
Global Leadership Solutions

1.612.337.8233  office
1.612.414.8998  mobile
1.612.337.3695  fax
Harry.Brull at PDINinthHouse.com

33 South Sixth Street
Suite 4900
Minneapolis, MN 55402

www.pdininthhouse.com 

-----Original Message-----
From: ipac-list-bounces at ipacweb.org
[mailto:ipac-list-bounces at ipacweb.org] On Behalf Of Morris, Ramona
(JUS)
Sent: Tuesday, June 28, 2011 1:32 PM
To: ipac-list at ipacweb.org
Subject: [IPAC-List] promotional exam strategy

Situation: 

A police service is using a test on their local policy and procedures
as
part of the promotional process.  Candidates are provided with the
pool
of items (say 500)...just the stems without the distractors. The test
consists of a subset of multiple choice items(say 100). 

The argument is that they want candidates to know all of the p and p,
and it is up to the candidate to prepare themselves. And, exam
security
becomes less of a headache.

Questions: 

1.    What is this strategy called?

2.    Do you have any advice about its use?

3.    What do we know about the strategy (e.g. validity, reliability
efficiency etc.)



Ramona Morris

Ontario Police College

Aylmer, Ontario


_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
http://www.ipacweb.org/mailman/listinfo/ipac-list

Confidentiality Notice: All information in this communication,
including any files or attachments, is intended for the sole use of the
individual or entity to which it is addressed and may contain
information that is confidential, proprietary and/or trade secret
information entitled to protection and/or exempt from disclosure under
applicable law. If you are not the intended recipient, you are hereby
notified that any dissemination, distribution, or copying of this
communication is strictly prohibited. If you have received this
communication in error, please notify the sender by return email and
delete this communication from your system. Thank you for your
cooperation.
_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
http://www.ipacweb.org/mailman/listinfo/ipac-list


>

This e-mail message is intended for the named recipient(s) and may
contain information that is privileged, confidential and/or exempt from
disclosure under applicable law. Unauthorized disclosure, copying or
re-transmission is prohibited. If you are not a named recipient or not
authorized by the named recipient(s), or if you have received this
e-mail in error, then please notify the sender immediately and delete
the message and any copies.
 >
Ce courriel est destiné exclusivement au destinataire mentionné en titre
et peut contenir de l'information privilégiée, confidentielle ou
soustraite à la communication aux termes des lois applicables. Toute
divulgation non autorisée, toute reproduction ou réacheminement est
interdit. Si vous n'êtes pas le destinataire de ce courriel, ou n'êtes
pas autorisé par le destinataire visé, ou encore, si vous l'avez reçu
par erreur, veuillez le mentionner immédiatement à l'expéditeur et
supprimer le courriel et les copies.