[IPAC-List] Entry-Level Firefighter Written Examination Providers

[Harry Brull]

Again, I’ve been asked by a medium/large Midwestern city to research written test offerings for entry-level firefighter applicants. These are not yet trained, so the focus is on both basic cognitive and personality attributes related to success in the position.
Following are vendors whom I’ve contacted in the past. I plan on re-contacting them and asking for updates.
CWH Research, Inc.
Cooperative testing service (CPS)
Ergometrics/Fire and Police Selection, Inc.
I/O Solutions
E.B. Jacobs
I’d appreciate knowing if anyone has had positive experience  with a vendor not already on my radar.

Thanks in advance.
Harry Brull




Harry Brull - Senior Partner
BCG consulting group
612.414.8998
harry at bcgconsultinggroup.com<mailto:harry at bcgconsultinggroup.com>

www.BCGconsultinggroup.com

From: IPAC-List [mailto:ipac-list-bounces at ipacweb.org] On Behalf Of Dennis Doverspike
Sent: Monday, October 17, 2016 11:38 AM
To: ipac-list at ipacweb.org
Subject: Re: [IPAC-List] Appealed Items

For those who have asked for the rationale for other than give credit to "B," I would give several:
1. From a psychological and PR perspective, we know gains are treated differently than losses. Better to let everyone gain, than most people lose.
2. The people who answered "A" thought they had the correct answer. So they have not had a chance to appeal and argue "A" was the best answer; should they now get a chance to appeal since the answer key has changed? And what about the people who answered "C" or "D" and thought that was a better answer than "B" but not a better answer than "A."
3. If there is some type of litigation or other action, you have a test writer who believes the best answer is "A."
4. It could be the item was just confusing, therefore the confusion over A and B, leading to the decision to simply give everyone credit for the item.
5. As for simply deleting an item, the rationale for keeping the item but giving everyone credit would be that 1) it does not change the content validity of the exam, since the item was not deleted and 2) if you have a 100 item exam with a cutoff at 70, you have not reduced the effective number of items to say 90, in a case where you would have a number of appeals.
Dennis


On Mon, Oct 17, 2016 at 12:43 PM, Dennis Doverspike <dennisdoverspike at gmail.com<mailto:dennisdoverspike at gmail.com>> wrote:
You are working for a city producing the final scoring key for a test. An item has been appealed where the original best answer was "A." The test review committee considers an appeal that "B" is the best answer, and agrees that "B" is the best answer. This was not simply a typo or miscoding, the original item writer believed that A was the best answer.
Do you:
1. Score A and B as correct; award one point for each.
2. Only score B as correct, counting A as being wrong.
3. Delete or throw out the item.
4. Give credit to all responses, A, B, C, and D.
5. Other
Does your jurisdiction have a stated or understood practice for handling such situations?
Do you have a rationale for your practice.
Please feel free to elaborate. Consultants, feel free to offer your usual practice.
Dennis

Dennis Doverspike, PhD., ABPP
Licensed Psychologist, #3539 (OHIO)
Independent Consultant
Professor of Psychology, University of Akron
dennisdoverspike at gmail.com<mailto:dennisdoverspike at gmail.com>

The information is intended only for the person or entity to which it is addressed and may contain confidential, privileged and/or a work product for the sole use of the intended recipient. No confidentiality or privilege is waived or lost by any errant transmission. If you receive this message in error, please destroy all copies of it and notify the sender. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. In the case of E-mail or electronic transmission, immediately delete it and all copies of it from your system and notify the sender. E-mail and fax transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses.



--
Dennis Doverspike, PhD., ABPP
Licensed Psychologist, #3539 (OHIO)
Independent Consultant
Professor of Psychology, University of Akron
dennisdoverspike at gmail.com<mailto:dennisdoverspike at gmail.com>

The information is intended only for the person or entity to which it is addressed and may contain confidential, privileged and/or a work product for the sole use of the intended recipient. No confidentiality or privilege is waived or lost by any errant transmission. If you receive this message in error, please destroy all copies of it and notify the sender. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. In the case of E-mail or electronic transmission, immediately delete it and all copies of it from your system and notify the sender. E-mail and fax transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist9.pair.net/pipermail/ipac-list/attachments/20161112/61ae3ec6/attachment-0001.html>