Warning: in_array() expects parameter 2 to be array, string given in /home/freerang/public_html/wp-content/plugins/wordpress-mobile-pack/frontend/sections/show-rel.php on line 37

how to write a survey question that’ll tell you nothing

October 1, 2010

in communication

yesterday i popped open SHRM’s healthy food and drinks in the workplace poll and stopped at the first question:

“Does your organization have a formal or informal practice and/or policy in place that promotes healthy food and drinks for work-related functions where food is served, in on-site vending machines, in the company cafeteria, etc?” (Y or N)

yes or no to what?! to whether my company has an informal policy or an informal practice (ad hoc and driven by those who care)? or is that a formal policy or a formal practice (sanctioned by leadership)? and are they asking which of these four options applies to our functions? our cafeteria? our vending machine? our etc? because, you know, my answer may be different for our cafeteria than for our “etc.”

i’m not picking on SHRM. poor survey question design is not unique to them. but this question is a great example of survey design that yields nothing insightful. it asks too many questions in one sentence and doesn’t accommodate all of the potential answers. when you do this, respondents:

a. answer only part of the question, based on what applies to them.

b. select which answer puts their company in the best light.

c. give you a mishmash, not actionable data.

d. all of the above.

f

Leave a Comment

{ 4 comments… read them below or add one }

R. J. Morris October 1, 2010 at 10:48 am

Fran–Great post highlighting an issue with a lot of surveys. I am a little surprised that SHRM would fall into this trap. Like you, I have no idea what they would do with data from this question. In a survey class I took a long time ago, the professor drilled into us that simplicity and clarity were the two most important elements in question design. Tough to do sometimes, but critical if you want to get good results.

Reply

fran October 4, 2010 at 9:21 am

R.J., thanks. i think i’ll have to write a parallel post that talks about how to write a survey question that’ll tell you everything…well, maybe not everything.

f

Reply

Bob Merberg October 2, 2010 at 3:02 pm

Great post, Fran. A lot of organizations do, indeed, design surveys poorly. I think, also, some conceive poor surveys. The biggest problem in survey conception being collection of data that isn’t actionable. Any time I hear someone talking about a new survey and saying, “I’d be curious to know…” or “Wouldn’t it be interesting to find out why…” I’m tempted to cover my ears. Who cares about curiosity?! There’s no point in committing the resources to a survey (including the time employees spend completing it) that collects data that isn’t actionable — with a readiness to actually act on it.
BTW, I also don’t want to pick on SHRM, but can’t help but comment on this question: “Do you agree that it is HR’s responsibility to regulate the type of food/beverages provided in the workplace in order to encourage employees to consume healthy food and drinks at work?” This question has potential bias, because it’s posed as “Do you agree…” rather than “Is it HR’s responsibility…” Further, the word “regulate” seems quite loaded. The question also suffers from the complexity issue you’ve already addressed. Am I being asked about regulation, or about encouragement? And, finally… I’m nitpicking, but the choices — strongly agree, agree, disagree, etc. — don’t match the question. The question is a yes or no question. The “strongly agree” -type choices only make sense if they are offered in response to a statement, such as “It is HR’s responsibility to promote healthy eating at work.”

Reply

fran October 4, 2010 at 9:23 am

hey bob.

being willing to act on the data is key to me. that and the commitment to telling participants what you learned, what you will act on and what you won’t…or can’t.

f

Reply

Previous post:

Next post: