[IPAC-List] Does the day you take an oral exam matter?

[Dennis Doverspike]

Liz 

I looked at Don and Mark's responses although I did not get your original
email yet.

Obviously this is a problem everyone faces. To me, the perception of the
candidates is correct and they have a legitimate concern.

I guess the questions are - do use the same role play every day and do you
use the same examiners everyday?

Do you use the same role play every day? If so, you could see where people
would think it was an advantage to go the last day. 

Do you use different examiners on different days? Again, you can see where
people might get upset.

If you give the same exam with the same examiners every day, then you can
run a simple statistical test to see if the results are equal. Of course,
this assumes candidates are assigned at random, which is unlikely.

The problem - there is no perfect solution. One solution is to standardize
by day or by examination team. The fact a lot of consultant do standardize
suggests they are worried about differences across days or examination
teams.

I would also recommend taking a look at the recent case in Akron - since it
involves similar issues and occurs in the State of Ohio.


Dennis Doverspike, Ph.D., ABPP
Professor of Psychology
Director, Center for Organizational Research
Senior Fellow of the Institute for Life-Span Development and Gerontology
Psychology Department
University of Akron
Akron, Ohio  44325-4301
330-972-8372 (Office)
330-972-5174 (Office Fax)
ddoverspike at uakron.edu

The information is intended only for the person or entity to which it is
addressed and may contain confidential, privileged and/or a work product for
the sole use of the intended recipient.  No confidentiality or privilege is
waived or lost by any errant transmission. If you receive this message in
error, please destroy all copies of it and notify the sender. If the reader
of this message is not the intended recipient, you are hereby notified that
any dissemination, distribution or copying of this communication is strictly
prohibited. In the case of E-mail or electronic transmission, immediately
delete it and all copies of it from your system and notify the sender.
E-mail and fax transmission cannot be guaranteed to be secure or error-free
as information could be intercepted, corrupted, lost, destroyed, arrive late
or incomplete, or contain viruses. 

-----Original Message-----
From: ipac-list-bounces at ipacweb.org [mailto:ipac-list-bounces at ipacweb.org]
On Behalf Of Don Nott
Sent: Wednesday, April 01, 2009 11:56 AM
To: Reed, Elizabeth; ipac-list at ipacweb.org
Subject: Re: [IPAC-List] Does the day you take an oral exam matter?

Hi Elizabeth,

Our company specializes in the development of public safety (police &
fire) testing & assessment; so, we deal with your candidate culture all
the time.

Simple answer: the more the candidate's understand the process, the less
likely they'll complain; assuming the process is objective and fair.

We provide a candidate orientation to candidate's so that they A)
understand how the exercises were developed, and B) understand that
assessors are using objective, job-related, behavioral criteria when
scoring; therefore, it doesn't matter the time/date that they are
tested.

My quick answer is that you need to start with a good process and ensure
that you educate candidates.  This reduces candidate skepticism and
increases their understanding that, hey, the process/exercises are fair
and objective.


:Don


-----------------------------------------------------
Donald M. Nott
I/O Associate
  
Fields Consulting Group
7926 Jones Branch Drive
McLean, VA 22102
(703) 506-9400 

-----Original Message-----
From: ipac-list-bounces at ipacweb.org
[mailto:ipac-list-bounces at ipacweb.org] On Behalf Of Reed, Elizabeth
Sent: Wednesday, April 01, 2009 11:47 AM
To: 'ipac-list at ipacweb.org'
Subject: [IPAC-List] Does the day you take an oral exam matter?

IPAC members and friends:

The City of Columbus utilizes oral exams in all of our promotional
testing for both police and fire.  These oral exams usually consist of
either two role-play exercises or one role-play exercise and one
structured interview.  Depending on the number of candidates, these
exams can take up to six full days to administer.  Over the years,
candidates have complained about being assigned on either the first day
of testing or the last day of testing.  The candidates' perception is
that the day of the exam makes a difference in the score that they
receive.  Questions for you:

Is this a common perception?  Is anyone aware of studies conducted to
analyze this?  What do you believe would be the best approach to study
this situation?

Thank you,
Liz Reed

Elizabeth A. Reed
Police and Fire Assessment Supervisor
Columbus Civil Service Commission
750 Piedmont Rd.
Columbus, Ohio 43224
Office: (614) 645-6032 Fax: (614) 645-0866
Email: Ereed1 at columbus.gov


_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
http://www.ipacweb.org/mailman/listinfo/ipac-list
_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
http://www.ipacweb.org/mailman/listinfo/ipac-list