[IPAC-List] Does the day you take an oral exam matter?

[Richard R. Carter]

Liz,

My two cents:

In New Orleans we just completed an assessment for Police Lieutenant and
had a rater from the Columbus Police Department who was very helpful
(Commander Bash).  I discussed this very issue with him.  

Our research on the subject mirrors that which Chris discussed in his
reply.  That is, we saw no statistical evidence that testing order made
a measurable difference.  

That said, in 1996 we moved away from giving the same test over a period
of days.  We now test administer the same exercise to all candidates on
one day, while sequestering those who had been exposed to this test so
that they could not contact people yet to be tested.  [This forum has
had nice discussions on the importance of controlling cell phone usage
that I won't repeat]

We made the move to one-day testing, despite the administrative
headaches, after discussions with the Police Superintendent.  Frankly,
the rank and file did not believe our data and assumed that those taking
a test later in the process would have access to information from those
taking the test on earlier days.  Thus, they assumed that those taking
the test later have an unfair advantage on the test.  Because this
perception was so strong, we felt the need to do what we could to
administer each exercise on a separate day and not to use the same
exercise over multiple days.  

Another solution that I've heard is to administer parallel forms of an
exam over a number of days.  Thus, responding to a group of protesters
at an abortion clinic on day one - becomes an exercise in which the
candidate faces a group opposed to animal testing on day two.  Assuring
that these exercises are truly parallel would be my concern with this
approach.  But, in speaking with another rater we had last week, there
doesn't seem to be too much concern about getting the "easy question"
vs. the "hard question".  

There are positives and negatives with any choice.  Factors to consider
include the number of applicants to be tested and your staff size.  

Hope this helps.

Rick Carter

-----Original Message-----
From: ipac-list-bounces at ipacweb.org
[mailto:ipac-list-bounces at ipacweb.org] On Behalf Of Reed, Elizabeth
Sent: Wednesday, April 01, 2009 10:47 AM
To: 'ipac-list at ipacweb.org'
Subject: [IPAC-List] Does the day you take an oral exam matter?

IPAC members and friends:

The City of Columbus utilizes oral exams in all of our promotional
testing for both police and fire.  These oral exams usually consist of
either two role-play exercises or one role-play exercise and one
structured interview.  Depending on the number of candidates, these
exams can take up to six full days to administer.  Over the years,
candidates have complained about being assigned on either the first day
of testing or the last day of testing.  The candidates' perception is
that the day of the exam makes a difference in the score that they
receive.  Questions for you:

Is this a common perception?  Is anyone aware of studies conducted to
analyze this?  What do you believe would be the best approach to study
this situation?

Thank you,
Liz Reed

Elizabeth A. Reed
Police and Fire Assessment Supervisor
Columbus Civil Service Commission
750 Piedmont Rd.
Columbus, Ohio 43224
Office: (614) 645-6032 Fax: (614) 645-0866
Email: Ereed1 at columbus.gov


_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
http://www.ipacweb.org/mailman/listinfo/ipac-list