[IPAC-List] Computer or Online Administered Job Analysis Questionnaires

[Crenshaw, Jeffrey]

Fred,

I cannot speak specifically to Qualtrics, as I have never used it, but can speak to online JA questionnaire completion in general.  We use an in-house built online JA survey application.  It has made the survey preparation, administration, tracking, data collection, and reporting process much more efficient.  I cannot say, however, that the online administration makes the respondents' actual completion of the survey any more efficient.  Obviously, a paper and pencil questionnaire can be completed just about anywhere, but the online questionnaire makes the availability of a desktop or laptop a necessity for the respondent.  As Dennis mentioned, there are challenges with mobile device compatibility.  We are currently revamping our in-house application with an intent to make the survey as mobile-friendly as possible, but given the nature of the questionnaire it will likely be very difficult to make it truly friendly for all devices.

To address the issue of respondents who do not have ready access to a computer or lack the computer proficiency needed to complete the survey, we have computer labs that we can make available for respondents to use and, if needed, obtain assistance in the completion of the survey. Depending on the nature of the jobs being analyzed, I would think that having some provision for dealing with such issues would be a necessity.  When we first began administering JA questionnaires online (over a decade ago) we compared data from both types of administration and saw no substantive difference in job analysis outcomes.  However, we have experienced a much higher response rates from the online surveys, primarily due to the ease of submission and our ability to track in real-time who has started the survey, who has completed it, and who has not yet accessed it...which allows us to send targeted reminders to those who have not completed it.

Regards,
Jeff


Jeffrey Crenshaw, Ph.D.
Personnel Board of Jefferson County
jeff.crenshaw at pbjcal.org<mailto:cjeff.crenshaw at pbjcal.org>

From: IPAC-List [mailto:ipac-list-bounces at ipacweb.org] On Behalf Of Brack, Frederick
Sent: Thursday, February 16, 2017 8:26 AM
To: ipac-list at ipacweb.org
Subject: [IPAC-List] Computer or Online Administered Job Analysis Questionnaires

Does anyone have any experience, insight, or lessons learned, with using online or computer-administered Job Analysis Questionnaires?  We are considering using a vendor named Qualtrics based on the results of a pilot study, which seemed to work well and elicited positive reactions from participants.
The way we administered the pilot survey was we had one of our Test & Measurement Specialists administer a modified, shorter version of a typical JAQ survey to a small group of incumbents at their offices using their desktop computers, so we could walk the employees through the Job Analysis process and ensure they understood the purpose of the questionnaires and the end-result, e.g.  a list of KSA's with importance and frequency ratings and a test blueprint.
At times we have had resistance from a highly unionized mostly blue-collar workforce to filling out lengthy, paper-based questionnaires, even when we are paying them overtime to do it. Some of the advantages of computer-administered Job Analysis Questionnaires are easier survey development & revision, user-friendly survey administration (for the employees with computer proficiency), and the ability to compile, analyze and report results quickly.
Thanks,
Fred Brack
MTA New York City Transit
Personnel, Testing, Selection & Classification
180 Livingston Street, Room 4070
Frederick.Brack at NYCT.com<mailto:Frederick.Brack at NYCT.com>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist9.pair.net/pipermail/ipac-list/attachments/20170217/3bf26797/attachment.html>