— About Surveys

How to Create an Employee Satisfaction Survey

July 10th, 2007

By Scott M. Smith Ph.D.

Employee attitudes, burnout tendencies, passion factors, loyalty, workplace climate, and competitive intelligence are key indicators for employee satisfaction, retention and productivity.

Qualtrics employee tracking will increase employee satisfaction and reduce employee turnover, thereby strengthening your organization. Many companies waste their organizationís HR training and mentoring efforts because employees are dissatisfied and leave. Employee satisfaction can be identified, tracked and improved with timely and accurate survey information.

Key Measures in an Employee Satisfaction Survey

The front line employee is where company meets the customer. The front line is critical to your business. From the customer's perspective, your front line employees are your business. Your organization depends on their service quality, productivity and passion to meet the needs of your customers.

Employee satisfaction surveys help your front line employees to coming together to achieve productivity goals and to provide high quality customer service and help your company achieve excellence.

Employee Satisfaction measures will help craft effective people strategies using our powerful and unique management tools to track indicators of quality, dissatisfaction and customer turnover, and precede actual employee decisions by months. Qualtrics has the most powerful survey software in the world we can help you learn more.

Find Out How to Measure Employee Satisfaction

By conducting an employee satisfaction survey with Qualtrics, you'll gain valuable information from the people most important in your organization — and fast. The Qualtrics do-it-yourself online survey tools are supported by experts in the survey and HR industry. Our experts will help you determine how to best measure employee satisfaction and answer questions like:

What percentage of your employees is happy in their current positions?

What job related issues are most on the mind of your employees today?

What changes are most needed to improve morale in your organization?

Scott Smith is the founder of Qualtrics.com. He is the James Passey Professor of Marketing and Director of the Institute of Marketing at Brigham Young University. He received his Ph.D. in Marketing and Quantitative Methods from Pennsylvania State University.

Posted by admin in Employee Satisfaction, How to, Create Surveys | Comments Off

How to Create a Survey

June 21st, 2007

Building or creating a great survey is a struggle for most researchers. Understanding how to build a survey starts with understanding the research process. The most important step is simply properly formulating the problem that you are trying to understand. Simply stated, if you don't formulate the problem correctly, you can never build the optimal (or even good) survey or questions.

The best approach to build a survey is outlined in the following steps:

Review the basic research objectives of the study.
- What is at the heart of what you are trying to discover?
- What actions do you want to take as a result of the survey?
Visualize all of the relevant information items you need.
- What will the output report look like?
- What charts and graphs will be prepared?
- What information do you need to be assured that action is warranted?
Prepare a written list of the topics in items 1 and 2 and order them according to their value in solving the research problem. List the most important topics first. REvisit items 1 and 2 again to make sure the objectives, topics and information you need are appropriate. Remember, you can't solve the problem if you don't ask the right questions.
Next, ask yourself "How easy or difficult is it for the respondent to provide information on each topic? If it is difficult (they probably don't know, can't remember, can't access the information or won't take time), then ask yourself if there another way to obtain the information. Perhaps asking another question or using another data collection technique.
The fifth step in building a survey is to review the sequence of topics to make sure they are unbiased. Do the questions asked first influence or bias the results of the next questions? Sometimes providing too much information. Perhaps asking another question or using another data collection technique

Determine the type of question that is best suited to answer the question. We must also think about the type of data produced by a given question type.
One easy way to do this is to create a table in MS Word or Excel that has three columns:

Question

Answer Format

Type of analysis to be conducted

Enter the questions

Enter the possible answers and their answer format:
Categorical, Ordered, Ranking, Rating.

open-ended text questions, dichotomous, multiple choice, rank order, multiple choice matrix, Likert or Semantic Differential scales, constant sum, conjoint, side by side

Percentages, frequency counts, means and standard deviations, cross tabulations and statistical tests (chi-square, t-test, ANOVA, regression, multivariate analysis).

In column 1, we enter the questions. In column 2, we indicate the possible answers and their format: open-ended text questions, dichotomous, multiple choice, rank order, scaled or constant sum (ratio scale). Finally, in column 3, we determine the type of analysis (percentages, means and standard deviations, cross tabulations and statistical tests). Do the question and answer formats provide enough robustness to meet analysis requirements.

Write the questions. You will need to write several versions of each question when you are building your survey. Building a great survey and building great questions often take six or more drafts. Select the best one.
Review the question sequence for bias and logical flow.
Repeat all of the steps above to find any major holes. Are the questions really answered? Are the answers what you really need to know? Have someone review it for you.
Time the length of the survey. A survey should take less than five to ten minutes. At six questions per minute, and depending on the question difficulty, you are limited to about 30-40 questions. When building a survey, remember that one open end text question counts for three multiple choice questions.
Pretest the survey to 20 or more people. Obtain detailed feedback . . . critically look at their responses.
- Do they make sense, or do they have a different frame of reference than you had imagined.
- What were they unsure about?
- Did they have questions?
- Did they have trouble understanding what you wanted
- Did they take a point of view not covered in your answers or question?
Revise your questionnaire and pre-test again or begin data collection.

Posted by admin in How to, Create Surveys | Comments Off

How to Conduct a Concept Test

May 25th, 2007

I have a great idea and want to see if it will make it in the market…

By Scott M. Smith Ph.D.

A concept test is the evaluation of a sketchy idea that represents the essence of the product. The concept is in the process of being developed so that it might grow up and become a market strategy. The goal of a market strategy is to convince a target segment of consumers that a particular concept possesses the benefits they desire and to support the claim with evidence.

Another more detailed definition provides insight into consumer product and advertising communications applications for concept tests:

Concept testing is the process of using quantitative methods and qualitative methods to evaluate consumer response to a product idea prior to the introduction of a product to the market. It can also be used to generate communication designed to alter consumer attitudes toward existing products.

These methods involve the evaluation by consumers of product concepts having certain rational benefits, such as “a detergent that removes stains but is gentle on fabrics,” or non-rational benefits, such as “a shampoo that lets you be yourself.” Such methods are commonly referred to as concept testing and have been performed using field surveys, personal interviews and focus groups, in combination with various quantitative methods, to generate and evaluate product concepts…

Advertising professionals have generally created concepts and communications of these concepts for evaluation by consumers, on the basis of consumer surveys and other market research, or on the basis of their own experience as to which concepts they believe represent product ideas that are worthwhile in the consumer market.

From these definitions, we understand that concept testing provides the direction and guidance necessary to evaluate and selectively identify key product and market information that is valued by the potential customer.

Concept tests reshape and redefine ideas to arrive at a basic concept for a product with greater potential for market acceptance. Specifically, concept tests:

Quantitatively assess the relative appeal of ideas or alternative product positions that aim the product at different target segments by highlighting product features that are most desirable by each segment of the population.
Provides necessary information for developing the product and product advertising.
Indicates segments of the population in which the appeal of the product is likely to be concentrated.

We see then, that concept testing provides insight into market potential for a product design project. It tests the success of a new product idea before it is marketed. The concept should be developed to the point that it conveys the product attributes, the desired positioning and the intended brand personality. The actual product concept test is conducted by defining and testing reaction to the core concept through exposure to a story board, sketches, graphics, or even a product mock-up.

The concept test is pre-design, and differs from a pre-test or test market, which are conducted later in the development cycle — based on a finalized product design. Pre-tests and test markets are a final check point at avoiding a major market error.

The term concept analysis is also associated with the process of providing “proof of concept,” which is often required for Venture Funding, and can also apply to communications copy development.

What are the Benefits of a Concept Test?

A concept test can provide a variety of benefits, including:

A product reaction study, including liking and intention to purchase
A tool that can be customized to profile your own product or service idea
Consumers can evaluate and provide feedback early in the development process
A management tool for measuring potential success and to refine the concept
A screen for unproductive ideas and a way to optimize the investment of time and resources

Problems Solved with Concept Tests

New Product Concepts — Identify which benefits are most important to customers and which features are most likely to lead to the fulfillment of that promise. Features can be categorized into those which are “need to haves” vs. “nice to haves.” Customer need must be identified and prioritized for product development and advertising.
Product Modification/Upgrade — modifications and upgrades can reformulate and add new life to existing products and services. Again, identifying the optimal bundle of features is a priority. Differentiating those features that are “need to haves” vs. “nice to haves” is critical in creating products or services that are truly “New and Improved” “New Release” and “Upgrade” worthy.
Migration path — Many products and services offer upgrade or migration paths. For the customer, it is a reflection of a need for the next level of sophistication. Understanding the key features and benefits is critical in mapping consumer needs to the likelihood of upgrading an existing product or adopting a new technology. Our experts will help you answer such critical questions as “Do benefits outweigh the costs and challenges of changing?” Features, Benefits, Brands, Image, Costs and Training are but a few factors that must be considered.
Product Usability — Serviceability — The most effective concept tests assess the use experience of a specific product or service and determine how that experience can be improved. This type of concept test research can focus on a variety of areas — ease of use and similarity to current usage patterns, the ability to adapt and use critical feature implementations, and the congruency with current image, usage patterns and service provisions.
Pricing and Incentives — no one underestimates the importance of price expectations in new product adoptions. Price, incentives, bundling, cross product tie-ins, and cost mitigating factors such as warranties… all change price perceptions and perceptions of value.

How do I Conduct a Concept Test?

Conducting a concept test can be broken into three distinct research tasks:

Pre-Screen concepts that have some potential of being researched.
Refine the concept
Conduct evaluation tests of the final concept, comparing it to other options and the competition

Different Kinds of “Concept” Tests

We identify several different types “concept tests” based on their timing in the product development cycle and the purpose or information that is desired. Concept screenings are conducted early in the product development cycle and are generally followed by positioning — demand analysis as the concept is finalized. Pre-market concept tests involve product comparisons and are conducted to identify competitive challenges and weaknesses in the concept that is being developed.

Type I: Concept Screening Tests

Concept Screenings represent product ideas that are presented to consumers in verbal or visual form and then quantitatively evaluated by consumers to measure degrees of concept believability, personal relevance, purchase intent, likelihood of trial, and similar indicators of product potential. Concept screenings typically identify:

Concept ideas that are sufficiently promising to merit further consideration and development
Believability, Relevance
Perceived Uniqueness, Value
Trial-ability, or the potential for trial
Relative Attribute Performance Advantages
Multiple measures of attribute desirability
Measures of trial potential based on multiple measures that go beyond purchase intent.

Type II: Concept Tests

Concept Test — Positioning — Demand Analysis involves a concept evaluation where concepts within the same functional product class are positioned and evaluated together, to determine relative advantage and potential share.

Positioning — demand analysis will often include selected measures from the Type I Concept Test, plus measures that…

Present consumers with test concepts defined to include the concept and competing brands in product consideration sets.
Identify potential opportunity size for a chosen concept, including preliminary sales estimates.
Provide an in-depth profile of potential adopters for targeting in the product launch.
Strengths/Weaknesses analysis based on attributes/benefits
Awareness/Distribution Impact

Type III: Pre-Market Concept Tests

Pre-Market Concept Tests present concepts to consumers for evaluation. The concept is compared to competing products to determine if the product delivers what is promised by the concept.

The concept test measures attractiveness of a new product or service before its launching into the market by identifying its strengths and weaknesses.
The concept test evaluates the level of agreement between the concept and the specific brand attributes.
The concept test detects communication problems that may interfere with appropriate comprehension by the target segment.
The concept test provides purchase intention indicators, with scenarios varying from most optimistic to most conservative.
The concept test minimizes the risk of failure in the market by allowing product and communications adjustments before launch.

Accuracy Issues in Conducting Concept Tests:

Concept tests provide a top level analysis of overall concept preference and the associated likes and dislikes about a concept. The accuracy of the concept test is affected by many items, including the implementation of the physical concept after testing. The actual service must be accurately reflected in the communication of benefits provided to the consumer. Market communications must provide an accurate mirror image of the benefits provided by the product and those benefits that the potential consumer believes they will be receiving. Otherwise, expectations are “disconfirmed” or not realized and dissatisfaction results.

Concept Screening Test Procedures

Develop A Concept Story Board

Copy and possibly photograph or illustration that describes how the product works and its end benefits.

Interview

Concept testing is usually conducted in a central location (could be done by telephone or mail), but typically done by intercept method (at mall, food store, or other traffic location).
For new product concept screening, 5-8 concepts are often viewed.
For testing alternative products for purchase and incentives, concept screening tests use 10 or more products.

Questions Asked

Purchase intentions
Purchase frequency
Uniqueness of the idea
Believability of the idea
Importance of the message
If alternatives (alternative buying incentive concept screening test) are presented, respondents are asked to evaluate concepts according to how interested they are in purchasing the product

Analysis Approach

Examine intention scores…
Combine “definitely would buy” and “probably would buy”
Classify the concepts in to schema:
- Green: Two “thumbs up” from the consumer: High Potential
  a. Winners — High purchase intent/high uniqueness
- Amber: Acceptable… maybe, but with modifications: Caution
  b. Me too / Generic products — High purchase intent/low uniqueness
  c. Fad / Specialty products — Low purchase intent/high uniqueness
- Red: Two “thumbs down” from the consumer: Rejected
  d. Long shots — Low purchase intent/low uniqueness

Concept Test Evaluation Procedures:
Conducted After Initial Screening and Concept Refinement.

Purpose:

Assess market potential for each product (or product positioning)
Determine a product concept’s strengths and weaknesses
Provide an indication about the market segment in which each proposed product is likely to have the greatest acceptance.
Give an indication of whether to initiate a test market.

How do we Implement the concept test?

Story board, commercial advertisements, actual product prototypes
Concepts have been refined and screened, so that only 2-3 concepts are actually tested

Who will be interviewed?

Concept testing is usually conducted in a central location (could be done by telephone or mail), but typically done by intercept method (at mall, food store, or other traffic location).

Questions asked (see below for further concept test questions)

Purchase intentions
Purchase frequency and volume
Key benefits (open ended questions)
Likes and dislikes (open ended)
Believability (open ended)
Uniqueness
Overall/attribute ratings
Key benefit importance
Demographics

Concept Test Analysis approach

Standard techniques
Combine “definitely would buy” and “probably would buy” and test across alternative concepts.
Classify the concepts in to schema
1. End benefit importance
2. Concept attribute ratings
3. Uniqueness
4. Believability

What to Look For in a Concept Test

Identify the effectiveness of individual concept assets
Identify the barriers to achieving the concept’s full potential
Identify consumer relationships with the concept
Identify the status of the concept in a competitive context
Identify failure of advertising to communicate clearly
Identify failure of packaging that is too non-intrusive
Identify failure through unmemorable product personality
Identify failure through consumer’s lack of value perceptions
Identify the brand equity assets
Identify potential consistencies in brand image and consumer perceptions
Identify potential inconsistencies in brand image and consumer perception
Identify unrecognized product benefits
Identify failures of the brand to integrate into the consumers’ lifestyle
Identify a blurred image
Identify an elusive brand/concept personality

Survey Components for a Basic New Product Concept Study

Concept tests can be easily constructed if the researcher is aware of the key components that should be included in the test instrument or survey. Not that these components will often vary, depending on the purpose of the concept study.

The researcher is urged to carefully consider the objectives of the concept test and to then determine if the measures used in the study will successfully answer these questions. Pre-test the instrument by reviewing the purpose, the results and then making sure that the study purpose is answered with exactness. The major components of concept tests are:

Concept Test Presentation:
Describe the concept completely. Use graphics, video, audio, samples or whatever best portrays the concept and associated attribute and benefit messages that are being tested.
Overall Concept Test Reaction Measurement
Concept need / relative improvement over current method of doing things
Overall reaction to the concept (acceptability, desirability, interest)
Likelihood of purchase of concept
Detailed Concept Test Analysis Evaluation
Likes and Dislikes about the concept
Attribute list evaluation
Awareness of competing products
Awareness of substitute and complementing products
Superiority over other existing products
Use Situation Evaluation
Likelihood of use in specified situations
Current use of similar / competing products
Frequency of product use
Value Analysis Evaluation
Evaluation of product value
Price sensitivity analysis
Preferred method of purchase
Segmentation Analysis
Market segment most likely to use
Type of communications (message, source, media) that would help you decide to purchase

Task Specific Studies Related to Concept Tests (and their components)

Survey Flow for a Typical Concept Test

Introduction and qualify respondents (not part of a disqualification group)
Awareness of product brands
Purchases of product Brands in past 3 months
Introduction of concept description with measures of likelihood of purchase
Purchase dimensions: number of bottles, frequency of purchase
Perceived value of the product concept
Innovativeness of concept measure
Affective evaluation: like-dislike measure
Power to replace: current product you purchase
Frequency of consumption in a day
Believability of concept
Relevance of concept to the respondent
Attribute and benefit evaluations:
- is a good value for the money
- would help me to … Be mentally sharp
- would (benefit, such as: taste good or be something i would eat everyday)
- has attractive packaging
- would provide me with a high level of health
Source of purchase (location)
Demographics

Closely related to the concept test is the habits and uses study. Habits and uses studies are directed at understanding usage situations: how, when and where the product is used. Habits and uses studies sometimes include a real or virtual pantry audit. Understanding current consumer practices goes a long way in understanding preferences in a new concept test. The major components of the habits and uses study include:

General Measures for a Habits and Uses Study

Frequency of Product Use:

Identify usage segments: User/non-user, frequency and amount of product use (Identify Heavy, Medium, Light, Non-user segments)
Comfort with use of product / service (are you a product user; do you currently own…)
Relative use: more or less of a user than other people using the product

Situational Use:

Primary and Secondary Product Use Situations
Primary use location
Critical issues and needs for product / service use
Identification of use situations (+ identify new uses that are currently unknown to you/your company)
How do you use the product?
How have you used the product (in what ways, applications, uses, problems solved, experience)
Reason for not purchasing more of the product / service (current or alternative use situations)
How has the customer studied or learned how to use product more effectively
(is there a user group, a sub-culture group?)

Experiential Use:

Identification of Brand Usage Experience
From whom did you learn to use the product? (sphere of influence)
Which brands have you used?
Which brand are you currently using?
Beliefs about product / service use benefits
Identification of stereotypical user profile

Use Satisfaction ‘ Use Loyalty:

Satisfaction with current use of brand/product/service
Loyalty of use (may be a series of items… see Product Loyalty Measures)

Product Fulfillment Analysis of Attributes, Features, Benefits

Products contain bundles of benefits (both tangible and image) that are promised to the user. The product fulfillment study determines if the expectations created for the product by advertising, packaging and the produce appearance are fulfilled when the product is used.

Product Use Scenario to Understand Fulfillment

Use Scenario Descriptors (where and how is the product used?)
Frequency of product use
Primary use location (home, work, etc.)
Primary precipitating events or situations for product use or need
Usage rate trend (more, same, less than a month ago )

Product Familiarity

Degree of actual product use familiarity
Knowledge (read product information, read product label, etc.)
Knowledge base of product ‘ Are they “hard core” consumers, a “Maven,” the super love group.
Company Contact: Have they called the 800 number , etc.
Awareness of other brands
Reasons for original product purchase (selection of reasons)
Primary benefits sought from the product

Product Evaluation

Attribute evaluation matrix question (quality, price, trust, importance, performance, value)
Perceived benefit associations matrix
Importance, performance
Identification of primary benefits sought
Comparison to other brands (better, worse)
What is the best thing about the brand, what could be done better

Advertising and Packaging Evaluation

Packaging size, design
Advertising Promise, message fulfillment evaluation

Value Analysis

Expectation of price
Expectation of relative price (full price, on sale)
Current price paid
Satisfaction Measurement
Overall Satisfaction

Product Tests and Advanced Concept Tests using Conjoint Analysis

The basic forms of concept tests identify the individual attributes describing the concept, or overall concept preference. However these basic methodologies do not identify the effects of combinations of attributes and how to modify attributes to optimize consumer acceptance and preference. Conjoint analysis is the methodology used to conduct this type of concept or product test.

Introduction

Conjoint analysis answers the question of which attributes are important to consumers and how important they really are. Taken in combination, individual product attributes can be used to describe an entire product. Conjoint analysis determines the combination of product attributes that consumers most prefer. Conjoint analysis, when applied to product, service, and communications projects identifies which product and service attributes, or which communications messages are most preferred and are best combined to produce maximum effect.

Conjoint Analysis originated out of the mathematical psychology research of conjoint measurement¹. Green and Wind² state that conjoint measurement is “concerned with measuring the joint effect of two or more independent variables on the ordering of a dependent variable.

The output of conjoint measurement consists of the simultaneous measurement of the joint effect and separate independent variable contributions to that joint effect, all at the level (asymptotically) of interval scales with common unit.” ” From the standpoint of multiattribute choice making, conjoint measurement can sometimes be used to decompose overall evaluation into implicit utilities for components of the multicomponent alternatives”

In layman’s terms, conjoint analysis (1) identifies the attributes important in a choice decision, (2) identifies the way the attributes are combined to make the decision, and (3) determines the utility value to each of the levels of each of the attributes considered in the decision.

Green and Wind further point out that the method of conjoint analysis used represent the different theories of how people choose between multi-attribute alternatives. Conjoint analysis attempts to jointly identify the composition model for decision choices and at the same time estimate the utility value of the attributes that are important in the choice decision. As the choices are analyzed, the researcher may predict choice share for different product configurations that may be introduced into the competitive marketplace.

Conjoint Analysis in a Nutshell

Conjoint analysis is a methodology for the measurement of psychological judgments, such as consumer preferences. Stimuli (product configurations, advertisements, movie endings, etc.) are presented to the respondent for evaluation.

For example, a respondent may be presented with a set of alternative product descriptions (automobiles). The automobiles are described by their stimulus attributes (level of gas mileage, size of engine, type of transmission, etc.). The respondent views selected alternatives and choice or preference evaluations are made.

From this evaluation or choice information, the researcher determines the respondent’s utility for each attribute level (i.e., what is the relative value of an automatic versus a five-speed manual transmission). Once the utilities for all attribute and all levels are determined for all respondents, the analysis of the utility data can begin.

Preference curves are identified for each attribute so as to show how the market of consumers values each of the different attribute levels. This analysis may be conducted for all respondents or for selected market segments.

Simulations are then run to determine the relative choice share (and thereby estimate market share) of competing sets of new or existing products.

Further explanation of conjoint analysis is found in the Qualtrics tutorial document “Introduction to Conjoint Analysis”.

Concept Tests and Product Tests Using Conjoint Analysis

How does this product perform when evaluated by the consumer?

Basis for evaluation…
Isolation, competitive frame, against its advertising, against formula variation, etc.

Purposes of product tests:

Tests against the competition:
Seek to identify which of many alternative new formulas is best in terms of being most preferred or to measure the performance of the new product relative to other competitive products.
Product improvement tests:
Determine whether an improved formula or construction should replace the current product
Cost-savings tests:
Determine whether a less expensive product should replace the current product
The fit of concept tests:
Determine which of several test-product variants most closely resembles what is being communicated by the selling concept

Types of product tests

Monadic product tests: one product is presented and an evaluation is requested, with no other specific product comparisons.

Comparison product tests: two or more products are presented and a comparison and rating of each is requested:

Paired comparison
Repeat paired comparison
Sequential monadic designs
Round robin designs
Proto-monadic designs
Triangle designs
Paired comparison designs
Duo-trio designs
Discrimination/difference
Choice based conjoint
Choice Based Conjoint
Hierarchical Bayes Conjoint

Posted by admin in Concept Testing, Market Research | Comments Off

Metatheory for Market Researchers

May 15th, 2007

By Scott M. Smith Ph.D.

As market researchers, we are dedicated to a careful appraisal of the market we are surveying. But how much time do we dedicate to the appraisal of our own efforts in this area? Metatheory, or the investigation of investigation, is a critical part of being a superior researcher who understands not only what is being measured, but why.

Metatheory is based upon the organization of the concepts being investigated into models that represent relationships. Survey questions are most effective when they focus on concepts that model a process and confidently represent reality on all significant issues.

How, then, do we measure reality? The two most useful measures of reality to a market researcher are Validity and Utility. The first measure, Validity, refers to the accuracy of the survey model in describing and predicting reality. A sales forecasting model that does not forecast sales with reasonable accuracy, for example, is probably worse than no sales forecasting model at all.

The second measure, Utility, is a different issue. Survey development often mirrors mathematical predictive models that are not incomplete, but too complete. Model developers often try to achieve validity, but are led to include so many variables (with correspondingly difficult data collection problems), that the basic structure of the model becomes buried, input data costs become escalated, and confidence in the model is lost.

To create a survey that meets the standards of Validity and Utility is a daunting task, but absolutely necessary to survey success. For further reading on this subject see Scott Smith’s Metatheory and the Escalating Measurement Quality Issues in Online Survey Research. Delve in, it’s good stuff.

Posted by Scott in Market Research, Metatheory | Comments Off

How to Measure Customer Satisfaction: Satisfaction Measurement and Theory

May 9th, 2007

By Scott M. Smith Ph.D.

Measuring satisfaction and building a satisfaction survey requires at least a basic knowledge of the satisfaction measurement literature, combined with your own customer satisfaction experiences. This brief tutorial provides such an introduction to the theoretical and methodological underpinnings of satisfaction research.

Customer satisfaction is the most common of all marketing surveys and is part of the "big three" research studies in marketing that include market segmentation and concept testing.

What Is Customer Satisfaction?

Customer satisfaction measures how well a company's products or services meet or exceed customer expectations. These expectations often reflect many aspects of the company's business activities including the actual product, service, company, and how the company operates in the global environment. Customer satisfaction measures are an overall psychological evaluation that is based on the customer's lifetime of product and service experience. "

Why is Customer Satisfaction So Important?

Effective marketing focuses on two activities: retaining existing customers and adding new customers. Customer satisfaction measures are critical to any product or service company because customer satisfaction is a strong predictor of customer retention, customer loyalty and product repurchase.

Satisfaction Measurement: Overall Measures of Satisfaction

Satisfaction measures involve three psychological elements for evaluation of the product or service experience: cognitive (thinking/evaluation), affective (emotional-feeling/like-dislike) and behavioral (current/future actions).

Customer satisfaction usually leads to customer loyalty and product repurchase. But measuring satisfaction is not the same as measuring loyalty. Satisfaction measurement questions typically include items like:

An overall satisfaction measure (emotional):
Overall, how satisfied are you with "Yoni fresh yogurt"?

Satisfaction is a result of a product related experience and this question reflects the overall opinion of a consumer's experience with the product's performance. Note that it is meaningful to measure attitudes towards a product that a consumer has never used, but not satisfaction for a product or brand that has never been used.
A loyalty measure (affective, behavioral):
Would you recommend "Yoni" to your family and friends?
A series of attribute satisfaction measures (affective and cognitive):
How satisfied are you with the "taste" of Yoni fresh yogurt?
How important is "taste" to you in selecting Yoni fresh yogurt?

Satisfaction and attitude are closely related concepts. The psychological concepts of attitude and satisfaction may both be defined as the evaluation of an object and the individual's relationship to it. The distinction is that satisfaction is a "post experience" evaluation of the satisfaction produced by the product's quality or value.
Intentions to repurchase (behavioral measures):
Do you intend to repurchase Yoni fresh yogurt?

Satisfaction can influence post-purchase/post-experience actions other than usage (such as word of mouth communications and repeat purchase behavior). Additional post-experience actions might include product or information search activity, changes in shopping behavior and trial of associated products.

As shown in Figure 1, customer satisfaction is influenced by perceived quality of product and service attributes, features and benefits, and is moderated by customer expectations regarding the product or service. Each of these constructs that influence customer satisfaction need to be defined by the researcher.

Satisfaction Measurement: Affective Measures of Customer Satisfaction

A consumer's attitude (liking/disliking) towards a product can result from any product information or experience whether perceived or real. Again, it is meaningful to measure attitudes towards a product or service that a consumer has never used, but not satisfaction.

Satisfaction Measurement: Cognitive Measures of Customer Satisfaction

A cognitive element is defined as an appraisal or conclusion that the product was useful (or not useful), fit the situation (or did not fit), exceeded the requirements of the problem/situation (or did not exceed). Cognitive responses are specific to the situation for which the product was purchased and specific to the consumer's intended use of the product, regardless if that use is correct or incorrect.

Satisfaction Measurement: Behavioral Measures of Customer Satisfaction

It is sometimes believed that dissatisfaction is synonymous with regret or disappointment while satisfaction is linked to ideas such as, "it was a good choice" or "I am glad that I bought it." When phrased in behavioral response terms, consumers indicate that "purchasing this product would be a good choice" or "I would be glad to purchase this product." Often, behavioral measures reflect the consumer's experience individuals associated with the product (i.e. customer service representatives) and the intention to repeat that experience.

Satisfaction Measurement: Expectations Measures

Many different approaches to measuring satisfaction exist in the consumer behavior literature. Leonard Berry in 2002 expanded previous research to refine ten dimensions of satisfaction, including: Quality, Value, Timeliness, Efficiency, Ease of Access, Environment, Inter-departmental Teamwork, Front line Service Behaviors, Commitment to the Customer and Innovation. Berry's dimensions are often used to develop an evaluative set of satisfaction measurement questions that focus on each of the dimensions of customer satisfaction in a service environment.

A diagnostic approach to satisfaction measurement is to examine the gap between the customer's expectation of performance and their perceived experience of performance. This "satisfaction gap" involves measuring both perception of performance and expectation of performance along specific product or service attributes dimensions.

Customer satisfaction is largely a reflection of the expectations and experiences that the customer has with a product or service. However expectations also reflect that influences the evaluation of the product or service. When we make major purchases, we research the product or service and gain information from the advertising, salespersons, and word-of-mouth from friends and associates. This information influences our expectations and ability to evaluate quality, value, and the ability of the product or service to meet our needs.

Types of Customer Expectations that Influence Satisfaction

Customer performance expectations for attributes, features and benefits of products and services may be identified as both explicit and implicit expectation questions.

Explicit expectations are mental targets for product performance, such as well identified performance standards. For example, if expectations for a color printer were for 11 pages per minute and high quality color printing, but the product actually delivered 3 pages per minute and good quality color printing, then the cognitive evaluation comparing product performance and expectations would be 11 PPM — 3 PPM + High — Good, with each item weighted by their associated importance.

Implicit expectations represent the norms of performance that reflect accepted standards established by business in general, other companies, industries, and even cultures.

Static performance expectations address how performance and quality for a specific application are defined. Each system's performance measures are unique, though general expectations relate to quality of outcome and may include those researched by Berry, or others such as: accessibility, customization, dependability, timeliness, and accuracy, tangible cues which augment the application, options, cutting edge technology, flexibility, and user friendly interfaces. Static performance expectations are the visible part of the iceberg; they are the performance we see and — often erroneously — assume are all that exist.

Dynamic performance expectations are about how the product or service evolves over time and includes the changes in support and product or service enhancement needed to meet future business or use environments. Dynamic performance expectations may help to "static" performance expectations as new uses, integrations, or system requirements develop.

Technological expectations focus on the evolving state of the product category. For example, mobile phones are continually evolving. Mobile service providers, in an effort to deal with the desire to switch to new technology phones, market rate plans with high cancellation penalties. The availability of low profile phones with email, camera, MP3, email, and blue tooth technology changes technology expectations as well as the static and dynamic performance expectations of the product. These highly involving products enhance perceptions of status, ego, self-image, and can even invoke fear when the product is not available.

Interpersonal expectations involve the relationship between the customer and the product or service provider. Person to person relationships are increasingly important, especially where products require support for proper use and functioning. Expectations for interpersonal support include technical knowledge and ability to solve the problem, ability to communicate, time to problem resolution, courtesy, patience, enthusiasm, helpfulness, understood my situation and problem, communication skills, and customer perceptions regarding professionalism of conduct, often including image, appearance.

For each of these types of expectations that when fulfilled result in customer satisfaction (or when not delivered, result in dissatisfaction and complaining behavior), the perceived quality and value are critical and directly influence intention to repurchase and loyalty.

Satisfaction Measurement: Perceived Quality Measures

Perceived quality is often measured through three measures: overall quality, perceived reliability, and the extent to which a product or service meets the customer's needs. Customer perceptions of quality are the single greatest predictor of customer satisfaction.

Satisfaction Measurement: Perceived Value Measures

Perceived value may conceptually refer to the overall price divided by quality or the overall quality divided by price. Perceived value is measured in many ways including overall evaluation of value, expectations of price that would be paid, and more rigorous methodologies including the Van Westendorp pricing analysis, and conjoint analysis (other Qualtrics white papers and tutorials are available on these topics).

The consumer behavior literature shows that price is a primary indicator of quality when other attributes and benefits are relatively unknown. However when repeat purchases are made in some product categories, price may be reduced in importance.

Satisfaction Measurement: Customer Loyalty Measures

Customer loyalty reflects the likelihood of repurchasing products or services. Customer satisfaction is a major predictor of repurchase, but is strongly influenced by explicit performance evaluations of product performance, quality, and value.

Models of Expectations and Customer Satisfaction

Expectations are beliefs (likelihood or probability) that a product or service (with certain attributes, features or characteristics) will produce certain outcomes (benefits-values). These expectations are based on previous affective, cognitive and behavioral experiences. Expectations are seen as related to satisfaction and can be measured in the following ways:

Importance-Value of the product/service fulfilling the expectation;
Overall Affect-Satisfaction Expectations: The (liking/disliking) of the product/service;
Fulfillment of Expectations: the expected level of performance vs. the desired expectations. This is "Predictive Fulfillment" and is a respondent specific index of the performance level necessary to satisfy.
Expected Value from Use: Satisfaction is often determined by the frequency of use. If a product/service is not used as often as expected, the result may not be as satisfying as anticipated. For example a Harley Davidson motorcycle that sits in the garage, an unused year subscription to the local fitness center/gym or a little used season pass to the local ski resort or amusement park may produce more dissatisfaction with the decision to purchase than with the actual product/service.

Expectancy Value Measures of Behavioral Intention (BI), Attitude (A) and Satisfaction (SAT)

Expectancy value models have been found to perform well in predicting both satisfaction/dissatisfaction and behavioral intention (intention to try, purchase, recommend, or re-purchase a product or service).

The Expectancy value model using attitudes and beliefs reads:

$B \approx BI \approx A_{o} = w_{1} \sum\limits_{i=1}^k a_{i} * b_{i} + w_{2}(\sum\limits_{i=1}^k nb_{i} * mc_{i})$

where:

w₁, w₂ = weights that indicate the relative influence of the overall attitude toward the object and the normative influence to purchase the product
A_o = Attitude toward the object (brand, product, service or company)
$\sum a_i * b_i$ = the overall attitude toward the object. The overall attitude is formed by the multiplicative product of a_i (the person's affective evaluation of attribute i), and b_i (here defined as the importance of attribute i in the purchase decision). The sum is taken over the k attributes that are defined as salient in the purchase decision.
$\sum nb_i * mc_i$ = The overall normative component of the decision process. This is computed as the multiplicative product of nb_i (the norms governing attitude i), and mc_i (the motivation of the respondent to comply with those norms).

Behavioral Intention (BI)
Behavioral intention is measured using a question such as "Indicate the likelihood of you buying sometime during the next year" with a five or seven-point Likert or semantic differential scale labeled "definitely will purchase" and "definitely will not purchase" at the endpoints.

Satisfaction
Overall satisfaction or dissatisfaction with an object is often measured using a five-point satisfaction scale. As an example, "Overall, how satisfied are you with Sparkle toothpaste?" could be measured with a "Very Satisfied, Somewhat Satisfied, Neither Satisfied Nor Dissatisfied, Somewhat Dissatisfied, Very Dissatisfied" scale. More examples are provided below.

The like-dislike measure is used as an overall measure of respondent satisfaction with a product or service (after purchase). Satisfaction leads to favorable feelings and dissatisfaction leads to unfavorable feelings.

The evaluative dimension may be measured in terms of like-dislike, favorable-unfavorable; approve-disapprove; good-bad; and delight-failure scales.

Attitude (a_i*b_i)
b_i - the probability that attribute i is associated with performing behavior B. The concept "Crest toothpaste prevents decay" could be rated on a seven point scale with endpoints labeled "Very Likely" and "Very Unlikely".

a_i - the evaluation of belief i. A representative measure of ai would be "In terms of buying Crest toothpaste, decay prevention is …" with a five or seven point scale with "good" and "bad"; or "Excellent" and "Poor" at the endpoints.

In building a customer satisfaction survey, it is also helpful to consider reasons why pre-purchase expectations or post-purchase satisfaction may or may not be fulfilled or even measurable.

Expectations may not reflect unanticipated service attributes;
Expectations may be quite vague, creating wide latitudes of acceptability in performance and expected satisfaction;
Expectation and product performance evaluations may be sensory and not cognitive, as in taste, style or image;
The product use may attract so little attention as to produce no conscious affect or cognition (evaluation), and result in meaningless satisfaction or dissatisfaction measures;
There may have been unanticipated benefits or consequences of purchasing or using the product (such as a use or feature not anticipated with purchase);
The original expectations may have been unrealistically high or low;
The product purchaser, influencer and user may have been different individuals, each having different expectations.

When to Conduct Customer Satisfaction Surveys

The best timing for measuring customer satisfaction and building customer satisfaction surveys depends on the kind of product or service provided, the kinds of customers served, how many customers are served, the longevity and frequency of customer/supplier interactions, and what you intend to do with the results.

Three very different approaches both produce meaningful and useful findings:

Post Purchase Evaluation — Satisfaction feedback is obtained from the individual customer at the time of product or service delivery (or shortly afterwards). This type of satisfaction survey is typically used as part of a CRM (Customer Relationship Management System) and focuses on having a long term relationship with the individual customer
Periodic Satisfaction Surveys — Satisfaction feedback from groups of customers at periodic intervals to provide an occasional snapshot of customer experiences and expectations.
Continuous Satisfaction Tracking — Satisfaction feedback is obtained from the individual customer at the time of product or service delivery (or shortly afterwards). Satisfaction tracking surveys are often part of a management initiative to assure quality is at high levels over time.

Satisfaction surveys are developed to provide an understanding of customers' expectations and satisfaction. Satisfaction surveys typically require multiple questions that address different dimensions of the satisfaction concept. Satisfaction measurement includes measures of overall satisfaction, satisfaction with individual product and service attributes, and satisfaction with the benefits of purchase. Satisfaction measurement is like peeling away layers of an onion-each layer reveals yet another deeper layer, closer to the core.

All three methods of conducting satisfaction surveys are helpful methods to obtain customer feedback for assessing overall accomplishments, degree of success, and areas for improvement.

Building a Customer Satisfaction Survey

Customer satisfaction surveys often include multiple measures of satisfaction, including:

Overall measures of customer satisfaction
Affective measures of customer satisfaction
Cognitive measures of customer satisfaction
Behavioral measures of customer satisfaction
Expectancy value measures of customer satisfaction

General Measures that are part of a customer satisfaction analysis usually involve product fulfillment and will often include product use scenarios where and how is the product used?