About Surveys

Often when working online, you are in need of a spell-check feature. Let's face it, if you're like me, a spell-check feature comes in handy when you're typing faster than you're thinking. I create a lot of surveys and I've found, thankfully, there are some very simple options that are very helpful when creating survey questions. 

The latest versions of Mozilla's Firefox and Apple's Safari browsers both provide built-in spell-check features. If you are using Microsoft's Internet Explorer 7, then you can install a simple add-on that will accomplish this for you (IE7Pro and ieSpell both provide a nice option for spell checking as well as instructions on how to use them).Now, any time you are entering text for a survey question, you will see the red underline when you have a questionable spelling. Hopefully you find these alternatives as helpful as I do!

By Scott M. Smith Ph.D.

Measuring satisfaction and building a satisfaction survey requires at least a basic knowledge of the satisfaction measurement literature, combined with your own customer satisfaction experiences. This brief tutorial provides such an introduction to the theoretical and methodological underpinnings of satisfaction research.

Customer satisfaction is the most common of all marketing surveys and is part of the "big three" research studies in marketing that include market segmentation and concept testing.

What Is Customer Satisfaction?

Customer satisfaction measures how well a company's products or services meet or exceed customer expectations. These expectations often reflect many aspects of the company's business activities including the actual product, service, company, and how the company operates in the global environment. Customer satisfaction measures are an overall psychological evaluation that is based on the customer's lifetime of product and service experience. "

Why is Customer Satisfaction So Important?

Effective marketing focuses on two activities: retaining existing customers and adding new customers. Customer satisfaction measures are critical to any product or service company because customer satisfaction is a strong predictor of customer retention, customer loyalty and product repurchase.

Satisfaction Measurement: Overall Measures of Satisfaction

Satisfaction measures involve three psychological elements for evaluation of the product or service experience: cognitive (thinking/evaluation), affective (emotional-feeling/like-dislike) and behavioral (current/future actions).

Customer satisfaction usually leads to customer loyalty and product repurchase. But measuring satisfaction is not the same as measuring loyalty. Satisfaction measurement questions typically include items like:

1. An overall satisfaction measure (emotional):
   Overall, how satisfied are you with "Yoni fresh yogurt"?

   Satisfaction is a result of a product related experience and this question reflects the overall opinion of a consumer's experience with the product's performance. Note that it is meaningful to measure attitudes towards a product that a consumer has never used, but not satisfaction for a product or brand that has never been used.

2. A loyalty measure (affective, behavioral):
   Would you recommend "Yoni" to your family and friends?

3. A series of attribute satisfaction measures (affective and cognitive):
   How satisfied are you with the "taste" of Yoni fresh yogurt?
   How important is "taste" to you in selecting Yoni fresh yogurt?

   Satisfaction and attitude are closely related concepts. The psychological concepts of attitude and satisfaction may both be defined as the evaluation of an object and the individual's relationship to it. The distinction is that satisfaction is a "post experience" evaluation of the satisfaction produced by the product's quality or value.

4. Intentions to repurchase (behavioral measures):
   Do you intend to repurchase Yoni fresh yogurt?

   Satisfaction can influence post-purchase/post-experience actions other than usage (such as word of mouth communications and repeat purchase behavior). Additional post-experience actions might include product or information search activity, changes in shopping behavior and trial of associated products.

As shown in Figure 1, customer satisfaction is influenced by perceived quality of product and service attributes, features and benefits, and is moderated by customer expectations regarding the product or service. Each of these constructs that influence customer satisfaction need to be defined by the researcher.

Satisfaction Measurement: Affective Measures of Customer Satisfaction

A consumer's attitude (liking/disliking) towards a product can result from any product information or experience whether perceived or real. Again, it is meaningful to measure attitudes towards a product or service that a consumer has never used, but not satisfaction.

Satisfaction Measurement: Cognitive Measures of Customer Satisfaction

A cognitive element is defined as an appraisal or conclusion that the product was useful (or not useful), fit the situation (or did not fit), exceeded the requirements of the problem/situation (or did not exceed). Cognitive responses are specific to the situation for which the product was purchased and specific to the consumer's intended use of the product, regardless if that use is correct or incorrect.

Satisfaction Measurement: Behavioral Measures of Customer Satisfaction

It is sometimes believed that dissatisfaction is synonymous with regret or disappointment while satisfaction is linked to ideas such as, "it was a good choice" or "I am glad that I bought it." When phrased in behavioral response terms, consumers indicate that "purchasing this product would be a good choice" or "I would be glad to purchase this product." Often, behavioral measures reflect the consumer's experience individuals associated with the product (i.e. customer service representatives) and the intention to repeat that experience.

Satisfaction Measurement: Expectations Measures

Many different approaches to measuring satisfaction exist in the consumer behavior literature. Leonard Berry in 2002 expanded previous research to refine ten dimensions of satisfaction, including: Quality, Value, Timeliness, Efficiency, Ease of Access, Environment, Inter-departmental Teamwork, Front line Service Behaviors, Commitment to the Customer and Innovation. Berry's dimensions are often used to develop an evaluative set of satisfaction measurement questions that focus on each of the dimensions of customer satisfaction in a service environment.

A diagnostic approach to satisfaction measurement is to examine the gap between the customer's expectation of performance and their perceived experience of performance. This "satisfaction gap" involves measuring both perception of performance and expectation of performance along specific product or service attributes dimensions.

Customer satisfaction is largely a reflection of the expectations and experiences that the customer has with a product or service. However expectations also reflect that influences the evaluation of the product or service. When we make major purchases, we research the product or service and gain information from the advertising, salespersons, and word-of-mouth from friends and associates. This information influences our expectations and ability to evaluate quality, value, and the ability of the product or service to meet our needs.

Types of Customer Expectations that Influence Satisfaction

Customer performance expectations for attributes, features and benefits of products and services may be identified as both explicit and implicit expectation questions.

Explicit expectations are mental targets for product performance, such as well identified performance standards. For example, if expectations for a color printer were for 11 pages per minute and high quality color printing, but the product actually delivered 3 pages per minute and good quality color printing, then the cognitive evaluation comparing product performance and expectations would be 11 PPM — 3 PPM + High — Good, with each item weighted by their associated importance.

Implicit expectations represent the norms of performance that reflect accepted standards established by business in general, other companies, industries, and even cultures.

Static performance expectations address how performance and quality for a specific application are defined. Each system's performance measures are unique, though general expectations relate to quality of outcome and may include those researched by Berry, or others such as: accessibility, customization, dependability, timeliness, and accuracy, tangible cues which augment the application, options, cutting edge technology, flexibility, and user friendly interfaces. Static performance expectations are the visible part of the iceberg; they are the performance we see and — often erroneously — assume are all that exist.

Dynamic performance expectations are about how the product or service evolves over time and includes the changes in support and product or service enhancement needed to meet future business or use environments. Dynamic performance expectations may help to "static" performance expectations as new uses, integrations, or system requirements develop.

Technological expectations focus on the evolving state of the product category. For example, mobile phones are continually evolving. Mobile service providers, in an effort to deal with the desire to switch to new technology phones, market rate plans with high cancellation penalties. The availability of low profile phones with email, camera, MP3, email, and blue tooth technology changes technology expectations as well as the static and dynamic performance expectations of the product. These highly involving products enhance perceptions of status, ego, self-image, and can even invoke fear when the product is not available.

Interpersonal expectations involve the relationship between the customer and the product or service provider. Person to person relationships are increasingly important, especially where products require support for proper use and functioning. Expectations for interpersonal support include technical knowledge and ability to solve the problem, ability to communicate, time to problem resolution, courtesy, patience, enthusiasm, helpfulness, understood my situation and problem, communication skills, and customer perceptions regarding professionalism of conduct, often including image, appearance.

For each of these types of expectations that when fulfilled result in customer satisfaction (or when not delivered, result in dissatisfaction and complaining behavior), the perceived quality and value are critical and directly influence intention to repurchase and loyalty.

Satisfaction Measurement: Perceived Quality Measures

Perceived quality is often measured through three measures: overall quality, perceived reliability, and the extent to which a product or service meets the customer's needs. Customer perceptions of quality are the single greatest predictor of customer satisfaction.

Satisfaction Measurement: Perceived Value Measures

Perceived value may conceptually refer to the overall price divided by quality or the overall quality divided by price. Perceived value is measured in many ways including overall evaluation of value, expectations of price that would be paid, and more rigorous methodologies including the Van Westendorp pricing analysis, and conjoint analysis (other Qualtrics white papers and tutorials are available on these topics).

The consumer behavior literature shows that price is a primary indicator of quality when other attributes and benefits are relatively unknown. However when repeat purchases are made in some product categories, price may be reduced in importance.

Satisfaction Measurement: Customer Loyalty Measures

Customer loyalty reflects the likelihood of repurchasing products or services. Customer satisfaction is a major predictor of repurchase, but is strongly influenced by explicit performance evaluations of product performance, quality, and value.

Models of Expectations and Customer Satisfaction

Expectations are beliefs (likelihood or probability) that a product or service (with certain attributes, features or characteristics) will produce certain outcomes (benefits-values). These expectations are based on previous affective, cognitive and behavioral experiences. Expectations are seen as related to satisfaction and can be measured in the following ways:

1. Importance-Value of the product/service fulfilling the expectation;
2. Overall Affect-Satisfaction Expectations: The (liking/disliking) of the product/service;
3. Fulfillment of Expectations: the expected level of performance vs. the desired expectations. This is "Predictive Fulfillment" and is a respondent specific index of the performance level necessary to satisfy.
4. Expected Value from Use: Satisfaction is often determined by the frequency of use. If a product/service is not used as often as expected, the result may not be as satisfying as anticipated. For example a Harley Davidson motorcycle that sits in the garage, an unused year subscription to the local fitness center/gym or a little used season pass to the local ski resort or amusement park may produce more dissatisfaction with the decision to purchase than with the actual product/service.

Expectancy Value Measures of Behavioral Intention (BI), Attitude (A) and Satisfaction (SAT)

Expectancy value models have been found to perform well in predicting both satisfaction/dissatisfaction and behavioral intention (intention to try, purchase, recommend, or re-purchase a product or service).

The Expectancy value model using attitudes and beliefs reads:

where:

• w₁, w₂ = weights that indicate the relative influence of the overall attitude toward the object and the normative influence to purchase the product
• A_o = Attitude toward the object (brand, product, service or company)
• = the overall attitude toward the object. The overall attitude is formed by the multiplicative product of a_i (the person's affective evaluation of attribute i), and b_i (here defined as the importance of attribute i in the purchase decision). The sum is taken over the k attributes that are defined as salient in the purchase decision.
• = The overall normative component of the decision process. This is computed as the multiplicative product of nb_i (the norms governing attitude i), and mc_i (the motivation of the respondent to comply with those norms).

Behavioral Intention (BI)
Behavioral intention is measured using a question such as "Indicate the likelihood of you buying sometime during the next year" with a five or seven-point Likert or semantic differential scale labeled "definitely will purchase" and "definitely will not purchase" at the endpoints.

Satisfaction
Overall satisfaction or dissatisfaction with an object is often measured using a five-point satisfaction scale. As an example, "Overall, how satisfied are you with Sparkle toothpaste?" could be measured with a "Very Satisfied, Somewhat Satisfied, Neither Satisfied Nor Dissatisfied, Somewhat Dissatisfied, Very Dissatisfied" scale. More examples are provided below.

The like-dislike measure is used as an overall measure of respondent satisfaction with a product or service (after purchase). Satisfaction leads to favorable feelings and dissatisfaction leads to unfavorable feelings.

The evaluative dimension may be measured in terms of like-dislike, favorable-unfavorable; approve-disapprove; good-bad; and delight-failure scales.

Attitude (a_i*b_i)
b_i - the probability that attribute i is associated with performing behavior B. The concept "Crest toothpaste prevents decay" could be rated on a seven point scale with endpoints labeled "Very Likely" and "Very Unlikely".

a_i - the evaluation of belief i. A representative measure of ai would be "In terms of buying Crest toothpaste, decay prevention is …" with a five or seven point scale with "good" and "bad"; or "Excellent" and "Poor" at the endpoints.

In building a customer satisfaction survey, it is also helpful to consider reasons why pre-purchase expectations or post-purchase satisfaction may or may not be fulfilled or even measurable.

1. Expectations may not reflect unanticipated service attributes;
2. Expectations may be quite vague, creating wide latitudes of acceptability in performance and expected satisfaction;
3. Expectation and product performance evaluations may be sensory and not cognitive, as in taste, style or image;
4. The product use may attract so little attention as to produce no conscious affect or cognition (evaluation), and result in meaningless satisfaction or dissatisfaction measures;
5. There may have been unanticipated benefits or consequences of purchasing or using the product (such as a use or feature not anticipated with purchase);
6. The original expectations may have been unrealistically high or low;
7. The product purchaser, influencer and user may have been different individuals, each having different expectations.

When to Conduct Customer Satisfaction Surveys

The best timing for measuring customer satisfaction and building customer satisfaction surveys depends on the kind of product or service provided, the kinds of customers served, how many customers are served, the longevity and frequency of customer/supplier interactions, and what you intend to do with the results.

Three very different approaches both produce meaningful and useful findings:

• Post Purchase Evaluation — Satisfaction feedback is obtained from the individual customer at the time of product or service delivery (or shortly afterwards). This type of satisfaction survey is typically used as part of a CRM (Customer Relationship Management System) and focuses on having a long term relationship with the individual customer
• Periodic Satisfaction Surveys — Satisfaction feedback from groups of customers at periodic intervals to provide an occasional snapshot of customer experiences and expectations.
• Continuous Satisfaction Tracking — Satisfaction feedback is obtained from the individual customer at the time of product or service delivery (or shortly afterwards). Satisfaction tracking surveys are often part of a management initiative to assure quality is at high levels over time.

Satisfaction surveys are developed to provide an understanding of customers' expectations and satisfaction. Satisfaction surveys typically require multiple questions that address different dimensions of the satisfaction concept. Satisfaction measurement includes measures of overall satisfaction, satisfaction with individual product and service attributes, and satisfaction with the benefits of purchase. Satisfaction measurement is like peeling away layers of an onion-each layer reveals yet another deeper layer, closer to the core.

All three methods of conducting satisfaction surveys are helpful methods to obtain customer feedback for assessing overall accomplishments, degree of success, and areas for improvement.

Building a Customer Satisfaction Survey

Customer satisfaction surveys often include multiple measures of satisfaction, including:

• Overall measures of customer satisfaction
• Affective measures of customer satisfaction
• Cognitive measures of customer satisfaction
• Behavioral measures of customer satisfaction
• Expectancy value measures of customer satisfaction

General Measures that are part of a customer satisfaction analysis usually involve product fulfillment and will often include product use scenarios where and how is the product used?

Common Ingredients of a Customer Satisfaction Survey

Product Use

• Frequency of product use
• Primary use location
• Primary precipitating events or situations for product use or need
• Usage rates and trends

Product Familiarity

• Degree of actual product use familiarity
• Knowledge (read product information, read product label, etc.)
• Knowledge and Involvement with product and the purchase process
• Awareness of other brands
• Reasons for original product purchase (selection reasons)
• Primary benefits sought from the product

Product Evaluation

• Attribute evaluation matrix: (quality, price, trust, importance, performance, value)
• Perceived benefit associations matrix
• Importance, performance
• Identification of primary benefits sought
• Comparison to other brands (better, worse)
• What is the best thing about the brand, what could be done better

Message and Package Evaluation

• Packaging size, design
• Advertising Promise, message fulfillment evaluation

Value Analysis

• Expectation of price
• Expectation of relative price (full price, on sale)
• Current price paid

Satisfaction Measurements

• Overall Satisfaction
• Reasons for Satisfaction Evaluation
• Satisfaction with attributes, features, benefits
• Satisfaction with use
• Expected and Ideal Satisfaction-Performance Measures
• Likelihood of recommending
• Likelihood of repurchasing

Sample Satisfaction Measures from the Qualtrics Question Library:

By Scott M. Smith Ph.D.

Online survey software and data collection, once seen as a supplement, is now poised to supplant many traditional research methodologies. This tool is empowering to all those who use it…but how can one ensure that it is used effectively?

1. Be Well Thought-Out. Avoid placing survey questions out of order or out of context. In general, a funnel approach is advised. Place broad and general questions at the beginning of the questionnaire as a warm-up, followed by more specific questions, followed by more general easy to answer questions (like demographics) at the end of the questionnaire.

2. Be Unbiased. Slight wording changes can produce great differences in survey results. “Could”, “Should”, and “Might” all sound almost the same, but may produce a 20% difference in agreement to a question:

DNA modification of human food supplies could.. should.. might.. be allowed in the United States.

Strong words that represent control or action, (such as “prohibit”) produce similar survey results: “Do you believe that congress should prohibit gasoline refiners from raising prices?”

Biased wording or statements should also be avoided: “You wouldn’t want to go to Rudolpho’s Restaurant for the company’s annual party would you?”

3. Be Specific. Unclear survey questions produce answers that lack meaning. The question “Do you like orange juice?” produces a less than meaningful answer, but is the respondent referring to taste, texture, nutritional content, vitamin C, the current price, concentrated, or fresh squeezed? Again, “Do you watch TV regularly?” begs for a definition of “regularly”. Specific questions produce specific understanding.

Questions must be specific. Though intended as a question about taste, the question “What suggestions do you have for improving energy drinks?” may produce suggestions about texture, the type of can or bottle, mixing juices, additives, or something related to use as a mixer or in recipes.

4. Be Exact.

Avoid Double Barreled Questions. The question “What is the fastest and most convenient Internet service for you?” is problematic because the fastest is certainly not the most economical. Two questions should be asked.

Avoid Overly Compressed Answers. Make sure answers choices are independent and cover the landscape of possible answers. For example the question “Do you think of basketball players as being independent agents or as employees of their team?” Some believe that yes, they are both.

Response categories for multiple choice survey questions should be mutually exclusive so that clear choices can be made. Likewise, questions that do not provide all acceptable or meaningful answers frustrate the respondent and make interpretation of results difficult at best. If you are unsure, conduct a pretest using the “Other (please specify)” option. Then revise the question making sure that you cover at least 90% of the respondent answers.

5. Be Considerate.

Avoid intrusive questions. Respondents may not have access to, remember, or want to provide the information requested. Likewise, privacy is an increasingly important issue. Questions about finances, income, occupation, family life, personal hygiene and beliefs (personal, political, religious) can be too intrusive and rejected by the respondent.

Avoid techno-babble. Caloric content, bits, bytes, MBS, and other industry specific jargon and acronyms produce confusion. Your audience must understand your language level, terminology and above all, the question being asked.

Avoid long questions. Multiple choice questions are the longest and most complex to write. Free text answers are the shortest and easiest to answer. Increasing the length of questions and surveys decreases the chance of receiving a completed response.

Avoid impossible questions on future intentions. Yogi Berra (Famous New York Yankees Baseball Player) once said that making predictions is difficult, especially when they are about the future. Predictions are rarely accurate more than a few weeks or in some case months ahead.

Remember: Quality in survey research is a state of “BE-ing”

Scott Smith is the founder of Qualtrics.com. He is the James Passey Professor of Marketing and Director of the Institute of Marketing at Brigham Young University. He received his Ph.D. in Marketing and Quantitative Methods from Pennsylvania State University.

One of the most challenging issues behind survey software is understanding question types, their uses, and their analysis. Question types are quite simple to understand, there are only really three types. Beyond that the issue is presentation and the data content that you are validating in them. What follows is a brief description of the three questions types and some different ways of presenting them.

Single Answer –
Use this question type when you want a participant to select 1 and only 1 answer choice. You can display the answer choice text in a horizontal, a vertical, a drop-down, or even a graphical manner. This question type is also commonly known as a radio button.

Multiple Answer –
Use this question type when you want a participant to select more than 1 answer choice. The display of this question type is similar to the single answer question type, except for the drop down menu. This question type is also commonly known as a check box.

Text Entry Box –
Just like it says, use this question type when you want the participant to enter in data. The data can be numbers that rank answer choices, or require a participant to enter numbers that sum to a certain amount, or you can have them tell you their life story.

Wow, that was simple.