[IPAC-List] Distance Learning/Facetiming Assessors

Fri Oct 31 16:38:09 EDT 2014

I presented at a PTC-Northern California meeting where one of the other
presenters did a major workshop session on the topic of videotaping for
later rating. As Liz Reed suggested, it is probably a great idea in theory,
but takes a lot of money and answering a great deal of technical challenges
to do in practice. I am not sure of the name of the presenter, but my
understanding was that the State of California had a major initiative going
where they were taping orals and interviews for technical jobs for later
rating. I do remember it was an excellent workshop and I was impressed by
the thought that had gone into their approach. Perhaps someone can supply
the name of a contact person or even the presenter for the State of
California.

On Fri, Oct 31, 2014 at 2:42 PM, Richard R. Carter <rrcarter at nola.gov>
wrote:

>  Thanks to all for all of the thoughtful insights.  This is the right
> forum for great ideas and resources.  We are still gathering information
> and, clearly, caution is warranted in such matters.  I'll report back if
> there are more developments down here.
>
>
>
> Rick Carter
>
> Personnel Administrator - Test Development & Validation
>
> New Orleans Civil Service
>
>
>
> *From:* IPAC-List [mailto:ipac-list-bounces at ipacweb.org] *On Behalf Of *Blair,
> Michael [HR]
> *Sent:* Friday, October 31, 2014 1:05 PM
> *To:* 'IPAC-List' (ipac-list at ipacweb.org)
>
> *Subject:* Re: [IPAC-List] Distance Learning/Facetiming Assessors
>
>
>
> Excellent dialogue.  Our listserv remains viable when most others have
> gradually dies because of discussions such as this.
>
>
>
> Rick - thanks for posing this question!  I suggest contacting Brian
> Bellenger or Jeff Crenshaw at the Personnel Board of Jefferson County,
> Alabama.  If memory serves me right, they both have experience
> administering remote assessor scoring using a video-tape model similar to
> that described by Liz.
>
>
>
> _*________________________________*
> *Michael Blair*
> Manager, Integrated Talent Management & Technology
>
> *Sprint*
>
> Office: 913-439-5222/ Wireless: 913-832-6130
>
> *michael.blair at sprint.com* <michael.blair at sprint.com>
>
> *www.linkedin.com/in/blairmichaeld*
> <http://www.linkedin.com/in/blairmichaeld>
>
>
>
> Looking for a career with Sprint: *Jobs at Sprint
> <http://www.sprint.jobs/>*
>
> Join our Talent Community to receive job alerts: *Sprint Talent Community
> <http://www.sprint.jobs/talentcommunity/subscribe/?email=>*
>
>
>
> *From:* IPAC-List [mailto:ipac-list-bounces at ipacweb.org
> <ipac-list-bounces at ipacweb.org>] *On Behalf Of *Reed, Elizabeth
> *Sent:* Friday, October 31, 2014 11:15 AM
> *To:* 'IPAC-List' (ipac-list at ipacweb.org)
> *Subject:* Re: [IPAC-List] Distance Learning/Facetiming Assessors
>
>
>
> Interesting topic. I've been involved with police and fire promotional
> processes for 20 years and we bring in assessors from all over the nation
> for these exams. While it is costly, the costs pale in comparison to the
> benefits of selecting the most competent future leaders in safety sensitive
> positions. I believe that we have great police and fire divisions because
> we are selecting the right leaders--outside assessors play an integral part
> in this selection process.
>
>
>
> It's technically feasible to accomplish virtual assessments. However, we
> can't underestimate the reality of not pulling raters together in person.
>
>
>
> Let's diagram what this might look like with a few basic assumptions:
> Assume you need 6 assessors for a week to grade video recorded oral
> responses. Assume the assessors have volunteered a week of their time,
> while still being paid by their own organization.  Assessor training needs
> to occur prior to the assessment. Here are the concerns that may not be
> easily remedied:
>
>
>
> ·        People are distracted during virtual training; it's difficult to
> keep them focused on the topic and involved in discussion.
>
> ·        Test security is at greater risk. Who else can access the test
> materials while in the hands of assessors in the various jurisdictions?
> Printed materials, people walking into the room during training, etc., and
> assuring materials are not copied or recorded are all risk factors.
>
> ·        During grading, will you have "boards" continue to discuss their
> ratings? If not, how will you ensure consistency and accuracy in the
> ratings? If yes, are you prepared to handle the additional logistical
> demands of monitoring of the process?
>
> ·        Assessor engagement during grading. The fact is they are being
> paid by their own jurisdiction, if something comes up and they are in the
> next room they are much more likely to be called away to work on the issue.
>
> ·        Will you continue to get the assessors you need? What's in it
> for them? When they come to the city they network, they ask other
> professionals how they handle situations they are currently struggling
> with. They have the opportunity to see a different city and how their
> operations work. There is a definite gain for agencies who send personnel
> to assess in other jurisdictions.
>
>
>
> As a previous responder eluded to, if you have assessors who are being
> paid by a testing agency to conduct and grade the assessment, then you can
> overcome most of the issues that I listed above. If that's the route you
> go, I'd recommend asking pointed questions regarding rater training,
> consensus building, how test security is handled, re-use of test exercises,
> are the exercises interactive, rotation of assessors, and costs.  About
> rotation of assessors, our unions prefer not using the same set of
> assessors each time we assess to avoid the perception of assessors
> identifying people they like a continuing to "promote" them through the
> ranks.
>
>
>
> Food for thought. Good luck!
>
>
>
> Liz
>
>
>
>
>
> Elizabeth A. Reed
>
> Public Safety Assessment Team Manager
>
> Columbus Civil Service Commission
>
> *______________________________*
>
>
>
> *Direct:* 614.645.6032
>
> http://www.columbus.gov/
>
>
>
> *From:* IPAC-List [mailto:ipac-list-bounces at ipacweb.org
> <ipac-list-bounces at ipacweb.org>] *On Behalf Of *Reindl, Kevin
> *Sent:* Thursday, October 30, 2014 7:14 PM
> *To:* Richard R. Carter; 'IPAC-List' (ipac-list at ipacweb.org)
> *Subject:* Re: [IPAC-List] Distance Learning/Facetiming Assessors
>
>
>
> Hi Rick!
>
>
>
> I like the use of technology in assessments...in fact, without technology,
> assessment would quickly become extinct in today's world. However, as one
> of your former coworkers (as well as one of your volunteer assessors later
> on), the value I gained (besides a chance to travel to New Orleans), was
> the opportunity to network with others in the public sector doing similar
> work in other jurisdictions. You should not underestimate the value that
> your assessors get out of making connections on a personal and professional
> level with one another. In all honesty, it may be very difficult to get
> that level of participation from other jurisdictions if you take away those
> less tangible benefits. I attend a couple of small consortium meetings in
> person every year, and from time to time, we talk about doing it virtually
> to minimize the travel costs. However, we continue to meet in person year
> after year for these same reasons.
>
>
>
> First thing I would do is send out a quick survey to your past
> participants to gauge their attitudes/opinions...i.e., ask them directly
> whether they would volunteer if it meant no travel and no in-person
> networking. If costs MUST be cut, you may need to make the change, but you
> may also end up with other costs down the road if you cannot get assessors
> (e.g., outsourcing to paid professional assessors)
>
>
>
> Good luck my friend!
>
>
>
> *Kevin Reindl*
>
> Pacific Gas & Electric
>
> Human Resources
>
> HR Strategy & Organizational Effectiveness
>
> 245 Market Street, N2J
>
> San Francisco, CA 94105
>
> Office: 415-973-7013
>
> Mobile: 619-322-3368
>
> k1rq at pge.com
>
>
>
>
>
>
>
> *From:* IPAC-List [mailto:ipac-list-bounces at ipacweb.org
> <ipac-list-bounces at ipacweb.org>] *On Behalf Of *Richard R. Carter
> *Sent:* Thursday, October 30, 2014 3:28 PM
> *To:* 'IPAC-List' (ipac-list at ipacweb.org)
> *Subject:* [IPAC-List] Distance Learning/Facetiming Assessors
>
>
>
> I have a few of questions for the collective wisdom of this group.
>
>
>
> We in New Orleans, like many jurisdictions, currently fly in volunteer
> police and fire professionals to score assessment center exercises.  This
> testing plays a large role in determining promotions in NOPD and NOFD.  We
> work folks hard for a week and pick up the tab for the hotel, airfare and
> incidentals.
>
>
>
> The Chief of our Fire Department would like for us to explore using
> technology to control costs.  Specifically, his suggestion to avoid the
> hotel and airfare expenses by having assessors interact only through
> technology (go to meeting, skype, facetime, something along those lines).
>
>
>
> First, is anyone else currently doing anything like this?  If so, what
> issues or concerns were faced? Even if you don't currently do this, feel
> free to opine with potential issues that those considering this choice may
> face.
>
>
>
> Also, what sort of cooperation should be expected from other jurisdictions
> should we go down this path?
>
> That is, *would* folks volunteer to stay where they are and work for us
> for a week?
>
>
>
> TIA
>
> Rick Carter
>
> Personnel Administrator - Test Development & Validation
>
> New Orleans Civil Service
>
>
>
>
>  ------------------------------
>
> PG&E is committed to protecting our customers' privacy.
> To learn more, please visit
> http://www.pge.com/about/company/privacy/customer/
>  ------------------------------
>
>
>  ------------------------------
>
>
> This e-mail may contain Sprint proprietary information intended for the
> sole use of the recipient(s). Any use by others is prohibited. If you are
> not the intended recipient, please contact the sender and delete all copies
> of the message.
>
> _______________________________________________________
> IPAC-List
> IPAC-List at ipacweb.org
> https://pairlist9.pair.net/mailman/listinfo/ipac-list
>
>

-- 
Dennis Doverspike, PhD., ABPP
Licensed Psychologist, #3539 (OHIO)
Independent Consultant
Professor of Psychology, University of Akron
dennisdoverspike at gmail.com

The information is intended only for the person or entity to which it is
addressed and may contain confidential, privileged and/or a work product
for the sole use of the intended recipient. No confidentiality or privilege
is waived or lost by any errant transmission. If you receive this message
in error, please destroy all copies of it and notify the sender. If the
reader of this message is not the intended recipient, you are hereby
notified that any dissemination, distribution or copying of this
communication is strictly prohibited. In the case of E-mail or electronic
transmission, immediately delete it and all copies of it from your system
and notify the sender. E-mail and fax transmission cannot be guaranteed to
be secure or error-free as information could be intercepted, corrupted,
lost, destroyed, arrive late or incomplete, or contain viruses.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist9.pair.net/pipermail/ipac-list/attachments/20141031/c1ba60b3/attachment-0001.html>