[IPAC-List] Job Qualifications

[Mueller, Lorin]

Not to engage in endless rejoinders but I am stuck in an airport!

Anyway, I am sure I'll be called a heretic of sorts but I don't think a job analysis is the best way to gather this kind of information. Job analyses are designed to give us information about (a) what is done on the job and (b) what KSAOs are required to do those things. Most (inf not all) job analytic approaches don't do much to take a census of how someone might obtain those KSAOs.

Nor should it, IMO. You really have to have a special kind of SME who can think about all the different ways someone can gain insight into how people acquire KSAOs. They can confirm or deny, especially with decision supports like course syllabi or previous job descriptions, but they generally don't pull this stuff from thin air.

My proposed approach uses SMEs to review resumes for KSA-relevant information, which they can put into a matrix to determine whether a candidate has met the BQs. It's an iterative process that first attempts to build the matrix by initially classifying people as qualified or not qualified (and asks the reasons why), then re-classifies people using the matrix, then adjusts the matrix or classifications to promote consistency and validity in classifications. The matrix can be used and refined over multiple applicant pools. But it does need a competent professional to manage the process.

So the good news is we'll all still have jobs!

Lorni


________________________________
From: Bryan Baldwin [Bryan.Baldwin at doj.ca.gov]
Sent: Wednesday, May 12, 2010 4:07 PM
To: Mueller, Lorin; ipac-list at ipacweb.org; Mark.Hammer at psc-cfp.gc.ca
Subject: RE: [IPAC-List] Job Qualifications

Excellent points, well taken.  I suppose my main problem is instead of going back to basics (what does the job analysis say, what would be the best way to test given the required KSAs, what are the current best practices in selection), they seem to be running headlong backwards (technology-wise).  Would love to see more discussion of (proctored and unproctored) Internet-based tests.  Seems to me that's the direction cutting-edge employers are going and it's a way to maximize validity as well as speed (again, where it's appropriate).  Just don't want to have them throw the baby out with the bath water.

Part of my reaction might (95% probability) have to do with the fact that this is the type of movement I saw in the state of Washington and what's going on with the state of California.  Perhaps, as you point out, this is an opportunity for us all to speak up loud and clear about the right (and wrong) ways to modernize.

BB


>>> "Mueller, Lorin" <LMueller at air.org> 5/12/2010 12:58 PM >>>

Some thoughts from 9 or so years of peering over the shoulders of federal hiring managers.

1) Even is KSA statements are more valid than résumé -based decisions, it doesn't matter much if they discourage applicants or if applicants select-out before decisions can be made. The most common complaints among my federal clients is that they get a certificate of eligibles, start calling to arrange interviews, and everyone withdraws because they have already found other jobs. And these are folks who filled out the application in the first place.

2) I don't think it's a foregone conclusion that KSA statements are more valid than point-based T&Es. I think the problem with point-based T&Es is that we don't have a way to score them that is consistent with reality. These things aren’t additive, they are configural. Early in the federal hiring process, we just want to know whether someone has the basic qualifications to perform the work successfully. That typically requires one of many combinations of education and experience that we believe confer KSAs. In our SIOP workshop, Steve Ferrara and I presented (in brief) a potential method for matrix-based resume scoring that I think is more reliable, valid, and legally defensible than other approaches. It still may not be the equal to the validity of KSA statements in a lab setting, but it lowers the time investment for highly qualified applicants. In short, I think there are valid ways to score résumés that are valid for making basic qualifications decisions, but it’s not exactly a sexy research area (just try to do a lit review on the topic).

3) All of the above presumes that KSA statements are completed by the applicant. A common complaint among hiring managers is that they get applicants who were obviously coached on what to write in their KSA statements. Not surprisingly, this is a common complaint among applicants as well - that KSA statements are coached or the questions are slanted to a particular candidate.

I am not saying this is what I would have done (agencies who have flirted with résumé based hiring have had their won problems), but it does address some common complaints regarding the federal hiring process. I see our role as I/O psychologists and HR professionals is to propose and promote best practices to maximize the reliability, validity, fairness, and legal defensibility of the new processes.

Lorin Mueller, PhD, SPHR
Principal Research Scientist
American Institutes for Research


________________________________________
From: ipac-list-bounces at ipacweb.org [ipac-list-bounces at ipacweb.org] On Behalf Of Bryan Baldwin [Bryan.Baldwin at doj.ca.gov]
Sent: Wednesday, May 12, 2010 9:31 AM
To: ipac-list at ipacweb.org; Mark.Hammer at psc-cfp.gc.ca
Subject: Re: [IPAC-List] Job Qualifications

Not to sound overly cynical, but why does it seem like when organizations use a valid selection method (e.g, behavioral consistency T&Es) but implement it poorly (e.g., using said instrument for thousands of applicants), the solution is to replace it with something much less valid (e.g., resumes/applications, point-based T&Es)?  Or is this just one more example of the rush to make the selection process as fast as possible, regardless of the impact on utility (supported by their suggested, highly suspect, metric of time-to-fill)?

On a related point, how exactly do they intend to get managers and supervisors "more fully involved in the hiring process, including planning current and future workforce requirements, identifying the skills required for the job, and engaging actively in the recruitment and, when applicable, the interviewing process"?  I mean more power to them, but I gotta think this is something many of have been struggling with for years.

Kudos for trying to modernize the federal hiring system, but I wonder if there is an appreciation for the enormity of what is being suggested and the resources it will require.

End rant.

Bryan Baldwin
Staff Services Manager II
California Department of Justice
Division of Administrative Support
Personnel Programs
(916) 322-5446

>>> Mark Hammer <Mark.Hammer at psc-cfp.gc.ca> 05/12/10 6:16 AM >>>

Addendum

Here is the link to the White House memorandum - http://www.whitehouse.gov/the-press-office/presidential-memorandum-improving-federal-recruitment-and-hiring-process  Makes for interesting reading.

Mark

_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
http://www.ipacweb.org/mailman/listinfo/ipac-list


CONFIDENTIALITY NOTICE: This communication with its contents may contain confidential and/or legally privileged information. It is solely for the use of the intended recipient(s). Unauthorized interception, review, use or disclosure is prohibited and may violate applicable laws including the Electronic Communications Privacy Act. If you are not the intended recipient, please contact the sender and destroy all copies of the communication.
_______________________________________________________
IPAC-List
IPAC-List at ipacweb.org
http://www.ipacweb.org/mailman/listinfo/ipac-list


CONFIDENTIALITY NOTICE: This communication with its contents may contain confidential and/or legally privileged information. It is solely for the use of the intended recipient(s). Unauthorized interception, review, use or disclosure is prohibited and may violate applicable laws including the Electronic Communications Privacy Act. If you are not the intended recipient, please contact the sender and destroy all copies of the communication.