[IPAC-List] Distance Learning/Facetiming Assessors

Fri Oct 31 12:14:36 EDT 2014

Interesting topic. I've been involved with police and fire promotional processes for 20 years and we bring in assessors from all over the nation for these exams. While it is costly, the costs pale in comparison to the benefits of selecting the most competent future leaders in safety sensitive positions. I believe that we have great police and fire divisions because we are selecting the right leaders-outside assessors play an integral part in this selection process.

It's technically feasible to accomplish virtual assessments. However, we can't underestimate the reality of not pulling raters together in person.

Let's diagram what this might look like with a few basic assumptions: Assume you need 6 assessors for a week to grade video recorded oral responses. Assume the assessors have volunteered a week of their time, while still being paid by their own organization.  Assessor training needs to occur prior to the assessment. Here are the concerns that may not be easily remedied:

*         People are distracted during virtual training; it's difficult to keep them focused on the topic and involved in discussion.

*         Test security is at greater risk. Who else can access the test materials while in the hands of assessors in the various jurisdictions? Printed materials, people walking into the room during training, etc., and assuring materials are not copied or recorded are all risk factors.

*         During grading, will you have "boards" continue to discuss their ratings? If not, how will you ensure consistency and accuracy in the ratings? If yes, are you prepared to handle the additional logistical demands of monitoring of the process?

*         Assessor engagement during grading. The fact is they are being paid by their own jurisdiction, if something comes up and they are in the next room they are much more likely to be called away to work on the issue.

*         Will you continue to get the assessors you need? What's in it for them? When they come to the city they network, they ask other professionals how they handle situations they are currently struggling with. They have the opportunity to see a different city and how their operations work. There is a definite gain for agencies who send personnel to assess in other jurisdictions.

As a previous responder eluded to, if you have assessors who are being paid by a testing agency to conduct and grade the assessment, then you can overcome most of the issues that I listed above. If that's the route you go, I'd recommend asking pointed questions regarding rater training, consensus building, how test security is handled, re-use of test exercises, are the exercises interactive, rotation of assessors, and costs.  About rotation of assessors, our unions prefer not using the same set of assessors each time we assess to avoid the perception of assessors identifying people they like a continuing to "promote" them through the ranks.

Food for thought. Good luck!

Liz

Elizabeth A. Reed
Public Safety Assessment Team Manager
Columbus Civil Service Commission
______________________________

Direct: 614.645.6032
http://www.columbus.gov/

From: IPAC-List [mailto:ipac-list-bounces at ipacweb.org] On Behalf Of Reindl, Kevin
Sent: Thursday, October 30, 2014 7:14 PM
To: Richard R. Carter; 'IPAC-List' (ipac-list at ipacweb.org)
Subject: Re: [IPAC-List] Distance Learning/Facetiming Assessors

Hi Rick!

I like the use of technology in assessments...in fact, without technology, assessment would quickly become extinct in today's world. However, as one of your former coworkers (as well as one of your volunteer assessors later on), the value I gained (besides a chance to travel to New Orleans), was the opportunity to network with others in the public sector doing similar work in other jurisdictions. You should not underestimate the value that your assessors get out of making connections on a personal and professional level with one another. In all honesty, it may be very difficult to get that level of participation from other jurisdictions if you take away those less tangible benefits. I attend a couple of small consortium meetings in person every year, and from time to time, we talk about doing it virtually to minimize the travel costs. However, we continue to meet in person year after year for these same reasons.

First thing I would do is send out a quick survey to your past participants to gauge their attitudes/opinions...i.e., ask them directly whether they would volunteer if it meant no travel and no in-person networking. If costs MUST be cut, you may need to make the change, but you may also end up with other costs down the road if you cannot get assessors (e.g., outsourcing to paid professional assessors)

Good luck my friend!

Kevin Reindl
Pacific Gas & Electric
Human Resources
HR Strategy & Organizational Effectiveness
245 Market Street, N2J
San Francisco, CA 94105
Office: 415-973-7013
Mobile: 619-322-3368
k1rq at pge.com<mailto:k1rq at pge.com>

From: IPAC-List [mailto:ipac-list-bounces at ipacweb.org] On Behalf Of Richard R. Carter
Sent: Thursday, October 30, 2014 3:28 PM
To: 'IPAC-List' (ipac-list at ipacweb.org<mailto:ipac-list at ipacweb.org>)
Subject: [IPAC-List] Distance Learning/Facetiming Assessors

I have a few of questions for the collective wisdom of this group.

We in New Orleans, like many jurisdictions, currently fly in volunteer police and fire professionals to score assessment center exercises.  This testing plays a large role in determining promotions in NOPD and NOFD.  We work folks hard for a week and pick up the tab for the hotel, airfare and incidentals.

The Chief of our Fire Department would like for us to explore using technology to control costs.  Specifically, his suggestion to avoid the hotel and airfare expenses by having assessors interact only through technology (go to meeting, skype, facetime, something along those lines).

First, is anyone else currently doing anything like this?  If so, what issues or concerns were faced? Even if you don't currently do this, feel free to opine with potential issues that those considering this choice may face.

Also, what sort of cooperation should be expected from other jurisdictions should we go down this path?
That is, would folks volunteer to stay where they are and work for us for a week?

TIA
Rick Carter
Personnel Administrator - Test Development & Validation
New Orleans Civil Service

________________________________
PG&E is committed to protecting our customers' privacy.
To learn more, please visit http://www.pge.com/about/company/privacy/customer/
________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist9.pair.net/pipermail/ipac-list/attachments/20141031/0d288f64/attachment.html>