About Surveys

By Scott M. Smith Ph.D.

Online survey software and data collection, once seen as a supplement, is now poised to supplant many traditional research methodologies. This tool is empowering to all those who use it…but how can one ensure that it is used effectively?

1. Be Well Thought-Out. Avoid placing survey questions out of order or out of context. In general, a funnel approach is advised. Place broad and general questions at the beginning of the questionnaire as a warm-up, followed by more specific questions, followed by more general easy to answer questions (like demographics) at the end of the questionnaire.

2. Be Unbiased. Slight wording changes can produce great differences in survey results. “Could”, “Should”, and “Might” all sound almost the same, but may produce a 20% difference in agreement to a question:

DNA modification of human food supplies could.. should.. might.. be allowed in the United States.

Strong words that represent control or action, (such as “prohibit”) produce similar survey results: “Do you believe that congress should prohibit gasoline refiners from raising prices?”

Biased wording or statements should also be avoided: “You wouldn’t want to go to Rudolpho’s Restaurant for the company’s annual party would you?”

3. Be Specific. Unclear survey questions produce answers that lack meaning. The question “Do you like orange juice?” produces a less than meaningful answer, but is the respondent referring to taste, texture, nutritional content, vitamin C, the current price, concentrated, or fresh squeezed? Again, “Do you watch TV regularly?” begs for a definition of “regularly”. Specific questions produce specific understanding.

Questions must be specific. Though intended as a question about taste, the question “What suggestions do you have for improving energy drinks?” may produce suggestions about texture, the type of can or bottle, mixing juices, additives, or something related to use as a mixer or in recipes.

4. Be Exact.

Avoid Double Barreled Questions. The question “What is the fastest and most convenient Internet service for you?” is problematic because the fastest is certainly not the most economical. Two questions should be asked.

Avoid Overly Compressed Answers. Make sure answers choices are independent and cover the landscape of possible answers. For example the question “Do you think of basketball players as being independent agents or as employees of their team?” Some believe that yes, they are both.

Response categories for multiple choice survey questions should be mutually exclusive so that clear choices can be made. Likewise, questions that do not provide all acceptable or meaningful answers frustrate the respondent and make interpretation of results difficult at best. If you are unsure, conduct a pretest using the “Other (please specify)” option. Then revise the question making sure that you cover at least 90% of the respondent answers.

5. Be Considerate.

Avoid intrusive questions. Respondents may not have access to, remember, or want to provide the information requested. Likewise, privacy is an increasingly important issue. Questions about finances, income, occupation, family life, personal hygiene and beliefs (personal, political, religious) can be too intrusive and rejected by the respondent.

Avoid techno-babble. Caloric content, bits, bytes, MBS, and other industry specific jargon and acronyms produce confusion. Your audience must understand your language level, terminology and above all, the question being asked.

Avoid long questions. Multiple choice questions are the longest and most complex to write. Free text answers are the shortest and easiest to answer. Increasing the length of questions and surveys decreases the chance of receiving a completed response.

Avoid impossible questions on future intentions. Yogi Berra (Famous New York Yankees Baseball Player) once said that making predictions is difficult, especially when they are about the future. Predictions are rarely accurate more than a few weeks or in some case months ahead.

Remember: Quality in survey research is a state of “BE-ing”

Scott Smith is the founder of Qualtrics.com. He is the James Passey Professor of Marketing and Director of the Institute of Marketing at Brigham Young University. He received his Ph.D. in Marketing and Quantitative Methods from Pennsylvania State University.

One of the biggest challenges I've seen corporations face as they have begun to adopt survey software is the time to implementation. Beyond learning the survey software system, of which the majority really do have a steep learning curve, is actually adjusting to the new responsibilities in the organization. Whether the corporation has chosen to deploy the software to all employees or a select few, the following items need to be kept in the back of your mind:

• Who will be the administrator — currently a lot of corporations have a department using various platforms
• Reporting procedures and processes will change
• You may have to involve the IT department more than you wish
• The politics of another department competing for a different survey software
• How does the survey software let me control and administer survey privileges to maintain HIPAA compliance–let's face it, if you can comply with HIPAA you can comply with any HR security issues.

Even that list is not fully exhaustive. When an organization is capable of effectively bringing in online survey software to their corporation, they should expect it to take about 1 to 3 months and increase their ability to gather information and give them a very adequate ROI within about 2 months. Ideally, it should be working like a well oiled machine after 6 to 9 months.

When used correctly, every organization I have dealt with has been able to see an ROI very quickly.

Just like any change project in an organization it is essential to map out the political and social changes that will occur in an organization. The most effective way to make a change project succeed is openness from the very beginning. Incorporating survey software into your organization will create changes that threaten individual and will create new political battles. As cool as it is, get ready for a wonderful ride when incorporating survey software in your organization!!

Online Survey Software—A unique blend of maximizing the gathering of information in a tightly controlled or loosely monitored environment for the explicit purpose of coming to understand a phenomena or event that is elusive, unknown, or just plain interesting.

What is an online survey? Well, for starters it is an online form - nothing more than a series of questions asked to a particular individual by another individual/company/organization. Over the past 10 years Online Survey Software has developed into a powerful information gathering tool. When it was first conceived it was intended to let individuals quickly build forms without a computer programmer. Today, it is a tool that not only replaces one programmer, but even replaces entire consulting organizations and outsourcing solutions. To save money and speed up the process a competent Online Survey Software product will immediately give you a Return on Investment.

As people use tools they expand their paradigm of what that tool can be used for. The same is occurring with Online Survey Software.

Fleeting are the days when a bunch of computer programmers could monkey around and build an Online Survey Software tool. Today, organizations are looking for a corporate tool that will allow them the opportunity to collaborate, share, view, and build surveys while maintaining security; with the ability to grow in features and uses as fast as they are capable of innovating.

Even now that is changing. Corporations are looking to integrate survey software with the numerous corporate functions available. Today, Online Survey Software is so powerful that organizations are building Six Sigma processes around it. In reality, any organization that does not have an Online Survey Software tool will be left in the dark ages. It will be the necessary tool of the future, and business schools not educating their students on the uses, applications, and abuses of it are doing their students a disservice.

Why do I say a business school is doing its students a disservice by not teaching about Online Survey Software? "Time is Money." Before you can manage your time you need to be able to make a decision on what to do with your time. In order to do that, you need information. Information is the precursor to any decision you might make. Therefore, the mantra should be "Information is Money." Today, organizations pay big bucks to keep information, but they are missing a very important aspect to information: you can find out anything you want today!!!! Shortly, and you can quote me on this, organizations will be paying big bucks to quickly gather information. To do that they need an effectively designed Survey Software that will meet all of their internal and external needs. Even if you did put a bunch of monkeys in a room together you'd never come up with an effective solution. The dimensions are far too broad and too many experts are needed. It is no different than building a house - get the foundation right and you'll be able to move any wall you want.

Currently, organizations are spending heavy amounts of cash on storing data, all types of data; but unfortunately, good tools for getting that data in a very user-friendly, decision-making environment are missing from the equation.

Well, welcome to the next wave. It is coming in two parts: 1) linking historical data to current phenomenological data (people's minds), and 2) seeing that data visually. [There are a few organizations currently working on that issue and I'll leave that for another blog.]

With technology, a business or organization can get an answer to any question they want. If you want information today, you should have it today. Know what? Investing time in picking the right survey software is one of the best things your organization could be doing right now.