[IPAC-List] Essay computer based testing: Seeking thoughts on dealing with computer glitches

[Walker, Kathleen A.]

Yes, I can certainly attest to equipment malfunctions and disappearing documents.  The more people and equipment involved, the higher the probability for a problem.  If we are testing for practical skills (i.e. keyboarding, word, excel), and an investigation determines that there is a possible computer glitch, we let them re-test.  The reasoning is that a candidate who does not possess that particular skill set will not be able to improve their skill set before a re-test.  However, this scenario is only for civilian positions, which are typically non-confrontational/low-stakes positions.

With our police/fire promotional testing, it becomes quite the legal issue.  If there is an equipment mal-function with a primary and back-up cameras (and yes, both have gone down), then we have to re-administer an entire NEW exercise again.  The problem is the perception, because conspiracy theories abound.  We are very transparent and explain exactly what happened, but it does not solve the trust issue.  Since we do not use live raters, these recordings are essential.  We have also had a candidate pull a flash drive out pre-maturely after completing a document, and yes, all data was gone.

Basically, I have come to the conclusion you have two options: you can keep the same process and do not change (because you have figured out what can go wrong) or you can be innovative and open to change, but there will be issues and malfunctions. We have even run our stats and we have a record of malfunctions of a .114 % for entry and promotional, and a .6% for promotional exams.  This data was run for a two year, almost 900 recordings, period.

The bottom line, is when it works, life is great :)
Any other insights?

Kathleen Walker
Employment Services Manager
City of Little Rock




From: ipac-list-bounces at ipacweb.org [mailto:ipac-list-bounces at ipacweb.org] On Behalf Of Dennis Doverspike
Sent: Tuesday, April 08, 2014 1:49 PM
To: Joel Wiesen
Cc: IPAC-List
Subject: Re: [IPAC-List] Essay computer based testing: Seeking thoughts on dealing with computer glitches

Yes to the disappearing. Over the years, various organizations I have been associated with have put into place different protections against that happening, such as having all data go to a server, or be downloaded to a server right after completion, or copying to a flash or other drive, or other redundancies. That requires taking a server and setting it up at remote testing sites, and even then you can run into unique technical problems.
I have an individual assessment story I tell (which is less of a concern than the typical high stakes testing situation). At the time I was using tests from a national vendor and if at the end of the test you hit one particular button on the keyboard, then your whole test would disappear - not sure why the vendor had that protection, I believe to keep candidates from accessing their own score reports or messing with their own files - but be that as it may.  Anyway, I would warn the candidates repeatedly, no matter what you do at the end of the test, do not hit the XX button, whatever the button was. Of course, one curious candidate wanted to see if his file would really get deleted if he hit the XX button. Of course, he hit it and his test got deleted. So he had to retake the whole hour long test.


Never had a candidate try to con with such a claim.

Dennis

On Tue, Apr 8, 2014 at 11:41 AM, Joel Wiesen <jwiesen at appliedpersonnelresearch.com<mailto:jwiesen at appliedpersonnelresearch.com>> wrote:
Have you ever had a candidate in a room full of test takers tell you that his or her computer file with all the exam answers just disappeared?

What do you do to forestall such an occurance?

Have any candidates ever tried to "con" you with such a claim?

Thanks.

Joel



--
Joel P. Wiesen, Ph.D., Director
Applied Personnel Research
62 Candlewood Road
Scarsdale, NY 10583-6040
http://www.linkedin.com/in/joelwiesen
(617) 244-8859<tel:%28617%29%20244-8859>
http://appliedpersonnelresearch.com




Note: This e-mail and any attachments may contain confidential and/or legally privileged information. Please do not forward any contents without permission. If you have received this message in error please destroy all copies, completely remove it from your computer, and notify the sender. Thank you.

_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org<mailto:IPAC-List at ipacweb.org>
http://nine.pairlist.net/mailman/listinfo/ipac-list



--
Dennis Doverspike, PhD., ABPP
Licensed Psychologist, #3539 (OHIO)
Independent Consultant
Professor of Psychology, University of Akron
dennisdoverspike at gmail.com<mailto:dennisdoverspike at gmail.com>

The information is intended only for the person or entity to which it is addressed and may contain confidential, privileged and/or a work product for the sole use of the intended recipient. No confidentiality or privilege is waived or lost by any errant transmission. If you receive this message in error, please destroy all copies of it and notify the sender. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. In the case of E-mail or electronic transmission, immediately delete it and all copies of it from your system and notify the sender. E-mail and fax transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://nine.pairlist.net/pipermail/ipac-list/attachments/20140408/c941eb1d/attachment-0001.html