[IPAC-List] Appealed Items

[Reid Klion]

Whenever something like this happens, one of the first things I suggest doing is analyzing the impact of the error and coming to an understanding the real world impact. While there is certainly the potential for a mess, it’s also possible the actual impact will be minimal depending upon how the candidates responded to items and how the results are used for decision making. This certainly isn’t to say that we shouldn’t adjudicate these things fairly but understanding the impact is often helpful in managing the situation.


On Oct 17, 2016, at 3:25 PM, Shekerjian, Rene (CS) <Rene.Shekerjian at cs.ny.gov<mailto:Rene.Shekerjian at cs.ny.gov>> wrote:

Boy this shows me how much you can read into the scenario Dennis described. I took it to mean that one person, the original author, thought that A was correct and upon later review, a group of people determined that B was correct. That would lead me to believe that the original author missed something and that given the appeal, the item analysis, and the group’s evaluation of the facts, that B was indeed the correct answer.
Thanks for all the additional perspectives, except the one about the lawyers arguing…
René

René Shekerjian

Department of Civil Service
Empire State Plaza, Agency Building 1, Albany, NY 12239

(518) 402-2660 | Rene.Shekerjian at cs.ny.gov<mailto:Rene.Shekerjian at cs.ny.gov>

www.cs.ny.gov<http://www.cs.ny.gov/>




From: Patrick McCoy [mailto:Patrick.McCoy at cfp-psc.gc.ca]
Sent: Monday, October 17, 2016 2:27 PM
To: Dennis Doverspike; ipac-list at ipacweb.org<mailto:ipac-list at ipacweb.org>
Subject: Re: [IPAC-List] Appealed Items

Interesting scenario Dennis:

Seems to me that it might be safest to give credit for both A and B, assuming that groups of competent SMEs helped to establish both answers.

Calculating the item-total correlation might also help in determining whether to go with A & B or whether or not the item should be dropped.

Pat McCoy
Ottawa, Canada



>>> Dennis Doverspike <dennisdoverspike at gmail.com<mailto:dennisdoverspike at gmail.com>> 2016/10/17 12:43 PM >>>
You are working for a city producing the final scoring key for a test. An item has been appealed where the original best answer was "A." The test review committee considers an appeal that "B" is the best answer, and agrees that "B" is the best answer. This was not simply a typo or miscoding, the original item writer believed that A was the best answer.
Do you:
1. Score A and B as correct; award one point for each.
2. Only score B as correct, counting A as being wrong.
3. Delete or throw out the item.
4. Give credit to all responses, A, B, C, and D.
5. Other
Does your jurisdiction have a stated or understood practice for handling such situations?
Do you have a rationale for your practice.
Please feel free to elaborate. Consultants, feel free to offer your usual practice.
Dennis

Dennis Doverspike, PhD., ABPP
Licensed Psychologist, #3539 (OHIO)
Independent Consultant
Professor of Psychology, University of Akron
dennisdoverspike at gmail.com<mailto:dennisdoverspike at gmail.com>

The information is intended only for the person or entity to which it is addressed and may contain confidential, privileged and/or a work product for the sole use of the intended recipient. No confidentiality or privilege is waived or lost by any errant transmission. If you receive this message in error, please destroy all copies of it and notify the sender. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. In the case of E-mail or electronic transmission, immediately delete it and all copies of it from your system and notify the sender. E-mail and fax transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses.



Ce courriel est destiné exclusivement au destinataire mentionné en titre et peut contenir de l'information privilégiée, confidentielle ou soustraite à la communication aux termes des lois applicables. Toute divulgation non autorisée, toute reproduction ou réacheminement est interdit. Si vous n'êtes pas le destinataire de ce courriel, ou n'êtes pas autorisé par le destinataire visé, ou encore, si vous l'avez reçu par erreur, veuillez le mentionner immédiatement à l'expéditeur et supprimer le courriel et les copies.

This e-mail message is intended for the named recipient(s) and may contain information that is privileged, confidential and/or exempt from disclosure under applicable law. Unauthorized disclosure, copying or re-transmission is prohibited. If you are not a named recipient or not authorized by the named recipient(s), or if you have received this e-mail in error, then please notify the sender immediately and delete the message and any copies.
_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org<mailto:IPAC-List at ipacweb.org>
https://pairlist9.pair.net/mailman/listinfo/ipac-list


________________________________
Legal Notice:

Performance Assessment Network (PAN) and its subsidiaries are located in the U.S. and the U.K.

This electronic mail and its attachments are intended solely for the person(s) to whom they are addressed and contain information which is confidential or otherwise protected from disclosure except for the purpose for which they are intended, and may be legally privileged. Dissemination, distribution or reproduction by anyone other than the intended recipient(s) is prohibited and may be illegal. If you are not an intended recipient, please immediately inform the sender by email and delete and permanently and irretrievably destroy any copies and attachments which may be in your possession. PAN screens electronic mail for viruses but does not warrant that any electronic mail is free of any viruses, and therefore does not accept any liability for any damage caused by any virus transmitted by any electronic mail.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist9.pair.net/pipermail/ipac-list/attachments/20161017/3a908e58/attachment-0001.html>