[IPAC-List] Essay computer based testing: Seeking thoughts on dealing with computer glitches

[Richard Joines]

Hi Joel,

I assume you're talking about online testing.  If online, does the test 
appear in one window and the response form in another?  Do you have an 
auto-save feature that operates periodically?  Can your proctor log the 
candidate back in and get the test back -- at least as of the last saving 
action?

In our online system, the test appears as a pdf file embedded in a web 
browser and comes in flush left.  The response form appears flush right in a 
different window.  Every so often, a candidate claims that the response form 
window just disappeared.  It's never the test window.

In one case, the candidate's response window disappeared repeatedly and I 
asked the test proctor to observe the candidate for a few minutes.  This 
candidate was maximizing the response form window, then when she needed to 
refer back to the test information in the other window, she would click the 
X in the upper right hand corner of her browser window.  The computer didn't 
give the standard warning asking are you sure...etc., it just closed the 
window.  When she wanted the window back, she was confused and claimed it 
had magically disappeared again.  Based on this experience, we started 
advising candidates to get their two windows properly organized side-by-side 
so they can read the test (our text automatically gets bigger if window is 
made wider), then just click on any piece of the response form window when 
they want to continue typing.  Usually there is no overlap of the two 
windows these days -- what with higher screen resolutions and larger flat 
panel monitors.

One time a candidate took our essay test but what we saw was nothing 
written.  The test was blank.  However, the test proctor reported seeing the 
candidate type a lot during the two hour test.  My client thought that there 
must be a glitch in the computer system.  I asked the client to have their 
tech support staff search the computer, and low and behold, the candidate 
typed their essay into a Word document and saved it on the computer!  Go 
figure.  And naturally, our instructions clearly state that no other 
program, including Word, can be fired up during the online test.  I assume 
the candidate thought that clicking our Submit Exam button, then confirming 
this in our pop-up window, then clicking our red Exit button would 
mysteriously and efficiently forward his Word document to us.

Anyway, having done online testing for about 10 years now, I don't believe 
windows just magically disappear, and I don't think that completed tests 
turn up blank.  However, try to convince a candidate of that.  My hit on 
this is to design the system to auto-save on a frequent basis and make sure 
you can log the candidate back in for a period of time.  By this, I mean 
that if the test is two hours long, you add code that lets you log back in 
within about 2 1/4 to 2 1/2 hours of the original login and get the 
candidate's test back (provided all the login info matches).  So, if you're 
frequently saving and you can get the test back, you limit how much work can 
be lost, then let the test proctor add some additional time if warranted. 
We add the down time to get the test back, then add up to an additional 
five minutes if it looks like 5 minutes or more work was lost (based on 
questioning the candidate).  I think this is reasonable if you have an 
auto-save of every 8 minutes or so because it takes less time to re-type 
something after it has been thought through and written the first time. 
This doesn't happen often but nobody has ever complained.

Rich Joines
Mgt & Personnel Systems, Inc.
www.mps-corp.com
925-330-0650


----- Original Message ----- 
From: "Joel Wiesen" <jwiesen at appliedpersonnelresearch.com>
To: "IPAC-List" <IPAC-List at ipacweb.org>
Sent: Tuesday, April 08, 2014 8:41 AM
Subject: [IPAC-List] Essay computer based testing: Seeking thoughts on 
dealing with computer glitches



> Have you ever had a candidate in a room full of test takers tell you that 

> his or her computer file with all the exam answers just disappeared?

>

> What do you do to forestall such an occurance?

>

> Have any candidates ever tried to "con" you with such a claim?

>

> Thanks.

>

> Joel

>

>

>

> -- 

> Joel P. Wiesen, Ph.D., Director

> Applied Personnel Research

> 62 Candlewood Road

> Scarsdale, NY 10583-6040

> http://www.linkedin.com/in/joelwiesen

> (617) 244-8859

> http://appliedpersonnelresearch.com

>

>

>

>

> Note: This e-mail and any attachments may contain confidential and/or 

> legally privileged information. Please do not forward any contents without 

> permission. If you have received this message in error please destroy all 

> copies, completely remove it from your computer, and notify the sender. 

> Thank you.

>

> _______________________________________________________

> IPAC-List

> IPAC-List at ipacweb.org

> http://nine.pairlist.net/mailman/listinfo/ipac-list