[IPAC-List] It's not what they say, but who is saying it

[Richard Arwood]

As always Mark; another great explanation of things, and a very interesting read!

…………..Richard

==============================================

Richard B. Arwood - Fire Chief (Retired-City of Memphis, TN)

  now residing in Collierville, TN

==============================================

 

From: ipac-list-bounces at ipacweb.org [mailto:ipac-list-bounces at ipacweb.org] On Behalf Of Mark Hammer
Sent: Tuesday, May 28, 2013 12:57 PM
To: ipac-list at ipacweb.org
Subject: [IPAC-List] It's not what they say, but who is saying it

 

My organization is tasked with oversight of merit, and assurance of integrity, in hiring in the Canadian federal context.  As one component of that mandate, we have an annual survey of about 95% of the public service, and ask people to tell us about one specific staffing activity they participated in, whether as hiring manager or somebody who tried to snag a job, or ended up getting one.  We ask a number of questions abut procedure, to get the hang of whether commendable practices are being observed.  We also ask a number of attitudinal and evaluative questions.

 

Among the questions are those concerning perceived fairness of how one was assessed and the fairness of the process overall, as well as whether candidates felted the assessment reflected the job.  And since we are interested in the efficiency of staffing, we also ask about how long things take.

 

One of the patterns observed over the last 3 years is that several indices and predictors of perceived fairness have gone downhill; enough that if you had no information other than the specific percent-positive/agree figures, you'd be worried about the health of the system.

 

Here is where it gets interesting, though.

 

Like governments nearly everywhere, ours has immersed itself into the austerity mindset, a little more each year, over the past few years.  One of the outcomes of that is that the composition of our survey respondents has been changing.  I should note that about 6 years ago, we embarked on a new survey methodology (for us, at least).  Understanding that the health of the merit system is not indexed solely by the opinions of the winners, we began including the losers.  So, the survey is a kind of "cold call", where we send the survey out to everyone, find out if people were involved in anything during the preceding year, ask them to tell us about the most recently completed staffing activity, and only after finding out about what they went through do we ask them if they were successful or not.

 

What you see is to be expected: winners have more positive views on the process than losers.  Sometimes there is a 20% spread between them.  Winners and losers all get combined and when we report to Parliament each year, one of the numbers we provide is what percentage of candidates said that the staffing process they were involved in was generally fair.

 

Although the aggregate number is falling by 4-6 percentage points per year (!!), the actual expressed opinions are no different.  Break out all those respondents into people who were successful in a competition, and those who applied and weren't, and you see no change whatsoever in how fair each group thought things were.  But what you DO see is a continuing shift in how much of the overall sample is made up of "winners" and "losers".  There are fewer jobs to give out, but you can't stop people from applying for them, even if their odds of snagging that promotion are slim under the present context.  And as the proportion of unsuccessful candidates in our sample increases, the aggregate attitudinal data descends.

 

But wait!  There's more!

 

The shifting ratio of successful to unsuccessful candidates has also altered estimates of time-to-staff.  We ask people how long things took to "conclude for you", and only use data from folks who give unambiguous signs they participated in the entire duration of the process and were not screened out near the start.  When we asked hiring managers to tell us how long processes were taking for them, it appears things are taking longer this year than in the recent past.  When we ask candidates how long things took for them, time-to-staff gone down.  What the....?  Well, consider that if you applied to a competition, and were unsuccessful, things end for you when you find that out.  You don't have to give anyone 2 weeks notice because you aren't going anywhere.  In our legal context, many jobs have a dual official-language requirement, so you don't need to be assessed for whether you meet the language fluency requirement, and of course, no one needs to do a security check on you to see if you meet the security requirements.  So successful candidates report processes taking about 3 more weeks than unsuccessful candidates report, for obvious reasons.  And, as you can imagine, with there being more unsuccessful candidates in our survey sample this year, magically staffing appeared to take about 2-3 weeks less than it did last year, from their perspective, even though successful and unsuccessful candidates are each reporting almost the identical times (<0.1month difference) they did last year.

 

The challenge here is reporting such results to the sort of folks who want a "dashboard" that presumes no underlying changes in the landscape against which staffing occurs.  Lest they think something is seriously wrong, we need to inform them about that landscape as simply as possible.

 

The moral of the story?  Pay attention to WHO is telling you things on organizational surveys.  It's an important consideration when surveying at one time only, but it becomes especially important when engaging in any sort of longitudinal comparison.

 

 

Mark Hammer

Ottawa



> 

This e-mail message is intended for the named recipient(s) and may contain information that is privileged, confidential and/or exempt from disclosure under applicable law. Unauthorized disclosure, copying or re-transmission is prohibited. If you are not a named recipient or not authorized by the named recipient(s), or if you have received this e-mail in error, then please notify the sender immediately and delete the message and any copies. 

> 

Ce courriel est destiné exclusivement au destinataire mentionné en titre et peut contenir de l'information privilégiée, confidentielle ou soustraite à la communication aux termes des lois applicables. Toute divulgation non autorisée, toute reproduction ou réacheminement est interdit. Si vous n'êtes pas le destinataire de ce courriel, ou n'êtes pas autorisé par le destinataire visé, ou encore, si vous l'avez reçu par erreur, veuillez le mentionner immédiatement à l'expéditeur et supprimer le courriel et les copies. 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://nine.pairlist.net/pipermail/ipac-list/attachments/20130528/bb346d96/attachment.html