[IPAC-List] collusion detection software

[Morris, Ramona (JUS)]

While on the subject of "cheating" Do any of you have recommendations
from experience with collusion detection software for multiple choice
exams. 
We have test driven several software packages: 
a) Caveon's test detective
b) Castlerock's integrity
c) Professor Wesolowsky's Scheck (he provides Harpp-Hogan results as
well) 
e) University of Western Ontario's SCANEX

As each uses different algorithms it can be difficult to explain why
each comes up with different pairs of "suspicious" cases. 
I would be most grateful for advice. 

Ramona Morris 
Research and Evaluation Unit 
Ontario Police College 
10716 Hacienda Rd. 
Aylmer, Ontario N5H 2T2 
ph.  519 773 4417 
fax: 519 773 8225 
ramona.morris at ontario.ca 
 
-----Original Message-----
From: ipac-list-bounces at ipacweb.org
[mailto:ipac-list-bounces at ipacweb.org] On Behalf Of Mark Hammer
Sent: Friday, May 15, 2009 5:02 PM
To: IPAC-List at ipacweb.org
Subject: Re: [IPAC-List] Test preparation/orientation websites

Just when you thought it was safe.  Our organization just released a
report concerning the use of "practice" tests by one of the many
language training schools in the capital.  In the Canadian system, a
great many upper level positions require dual-language proficiency
(English and French).  So, it should not surprise anyone that the
capital, where many such positions are located, has an abundance of
language training schools.  When tested, candidates can be assessed as
meeting one of 4 different proficiency levels for each of oral fluency,
writing and written comprehension, with different positions requiring
different profiles.  The highest level of proficiency is termed
"exempt", implying that the fluency is of such a degree that no further
testing would ever be required.  This is in contrast to other assessed
levels, whose score is valid for 5 years, after which time it has to be
re-assessed. 

Having been through training twice (the second time for a higher level),
we were given numerous practice tests to get us accustomed to the
self-pacing and strategic thinking needed to do well, in addition to
allaying our fears about how hard it might be.  Seems the "practice
tests" used by one such school were a little uncomfortably close to the
real test, such that their graduates were receiving something like a 90%
exemption rate, compared to the usual rate of 10% or so.  You can read
about it here:  http://www.psc-cfp.gc.ca/inv-enq/rprt/npu/index-eng.htm 

The good news, I suppose, is that this only pertains to written tests,
which most will report as being easier than the oral fluency test.  The
oral test is still a high wall to climb for many and difficult to get
over by means as unethical as that included in the report.

Back to the drawing board.  Oy!!

Mark Hammer
Ottawa

_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
http://www.ipacweb.org/mailman/listinfo/ipac-list