I recently came across an odd question in a conference survey. It read: Will you attend the topical dinner? One respondent replied, "Topical dinner? Is that something I can smear all over my body?" It was then I knew data analysis would be a problem.
There's a science to writing surveys. Well-written surveys can yield powerful findings. But poorly written surveys are a big waste of time.
How you ask
Sometimes it's not about what you ask, but how you ask. Say you and a friend go to dinner. You think your friend didn't like their entree but you're not sure. How do you find out? You could ask, "Did you enjoy your entree?" But the response would likely be "yes" regardless of the truth. "Yes" is the polite response. Most friends don't want to ruin dinner. But asking the question in a different way can change everything. Asking, "Would you order that entree again?" could uncover an entirely different answer.
Questions aren't the only issue. Response sets can do just as much damage.
I was once called in to investigate survey findings from a Fortune 500 company. Executives couldn't understand why thousands of customers called to express their dissatisfaction about store service when customers consistently reported being satisfied in store surveys.
One look at the survey responses explained everything. Customers could select "satisfied," "very satisfied" or "extremely satisfied" to rate store service. It was impossible for customers to report that they were dissatisfied or even neutral. So, quite naturally, they didn't.
Writing good questions and response sets is really only half of what you need to collect good data. Survey distribution is equally important. I once spoke to a call center rep who asked me if I would rate her performance on a survey. I agreed. She proceeded to administer the survey herself. My ratings were quite positive but, to be honest, I didn't feel that way. I'd have been more comfortable reporting what I really felt to someone else. But because she administered her own evaluation, her company will never know my true feelings.
Doing something about it
Once good data have been collected, perhaps the most important step is turning your findings into actionable strategy. Far too often, findings documentation is a place where survey results go to die. And that's a shame.
When asked to provide suggestions on a "topical dinner," one survey respondent wrote "Butter, please." Surveys are one of the most common ways of collecting customer feedback. Unknowingly, many companies design surveys that produce worthless findings. And that means that many of us are completely in the dark about our customers.
Melissa Read is the director of behavioral research at Spunlogic, an interactive marketing and technology agency. She can be reached at firstname.lastname@example.org.