[IPAC-List] Vendors for written exams - entry police and fire

[Partain, Steven C.]

Hi Kelley.  We have been utilizing the FireTEAM test from Ergometrics for about 10 years.  The test has four components: 1) human relations, 2) mechanical reasoning, 3) reading comprehension, and 4) math comprehension.  The first two components involve video-based situational judgement test items.  >From an assessment professional's perspective, it appears to perform well.  The psychometrics (reliability, validity, low adverse impact) are at least as good as any alternatives we've seen.  The content domain it targets is quite good.

>From a non-assessment professional's perspective, it is fatally flawed.  It cannot be "mastered" via hard work (effort put into studying) and there is little effective coaching that can be offered to test takers to give them an edge.   (Most assessment professionals would see this as a positive.)  I believe the fire service tends to attract individuals who believe everything can be mastered via hard work and training.  This test frustrates them as they try to support others trying to get into our organization.

Also, there are test takers who are deemed high-quality candidates by members of our workforce who don't score high enough on the test to be considered for hire, and people who do score high who are deemed to be of lesser quality by members of our workforce (false positives and false negatives, which I'm sure you'll appreciate are present in any test instrument).  For these reasons, a large portion of our workforce has rejected the test, and we have been charged with researching alternatives.  Despite our efforts, we have been unable to overcome the negative perception among laypersons in our workforce and communicate persuasively re: the science of assessment.

I share all of that out of recognition that in shopping for a test instrument (or package of instruments), workforce acceptance is at least as important as test instrument quality.  (That may be why you are focusing on written tests?)

Good luck!


____________________________________________________
Steven Partain | HR Manager
Tualatin Valley Fire & Rescue
Direct: 503-259-1292
www.tvfr.com<http://www.tvfr.com/>




From: IPAC-List [mailto:ipac-list-bounces at ipacweb.org] On Behalf Of Hughes, Kelley A.
Sent: Monday, January 04, 2016 11:59 AM
To: ipac-list at ipacweb.org
Subject: [IPAC-List] Vendors for written exams - entry police and fire

Hello all and Happy New Year,

We are currently in the process of evaluating vendors for both entry fire and police written exams and I would be interested in any feedback and/or recommendations the group might have on the subject.  Thank you in advance.

Kelley Hughes
Human Resources Analyst, Sr.
City of Little Rock Human Resources
500 W. Markham Suite W-130
Little Rock AR 72205
(501) 371-4564
mailto:kahughes at littlerock.org


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist9.pair.net/pipermail/ipac-list/attachments/20160104/bef83088/attachment-0001.html>