About Surveys

I have a great idea and want to see if it will make it in the market…

By Scott M. Smith Ph.D.

A concept test is the evaluation of a sketchy idea that represents the essence of the product. The concept is in the process of being developed so that it might grow up and become a market strategy. The goal of a market strategy is to convince a target segment of consumers that a particular concept possesses the benefits they desire and to support the claim with evidence.

Another more detailed definition provides insight into consumer product and advertising communications applications for concept tests:

Concept testing is the process of using quantitative methods and qualitative methods to evaluate consumer response to a product idea prior to the introduction of a product to the market. It can also be used to generate communication designed to alter consumer attitudes toward existing products.

These methods involve the evaluation by consumers of product concepts having certain rational benefits, such as “a detergent that removes stains but is gentle on fabrics,” or non-rational benefits, such as “a shampoo that lets you be yourself.” Such methods are commonly referred to as concept testing and have been performed using field surveys, personal interviews and focus groups, in combination with various quantitative methods, to generate and evaluate product concepts…

Advertising professionals have generally created concepts and communications of these concepts for evaluation by consumers, on the basis of consumer surveys and other market research, or on the basis of their own experience as to which concepts they believe represent product ideas that are worthwhile in the consumer market.

From these definitions, we understand that concept testing provides the direction and guidance necessary to evaluate and selectively identify key product and market information that is valued by the potential customer.

Concept tests reshape and redefine ideas to arrive at a basic concept for a product with greater potential for market acceptance. Specifically, concept tests:

1. Quantitatively assess the relative appeal of ideas or alternative product positions that aim the product at different target segments by highlighting product features that are most desirable by each segment of the population.
2. Provides necessary information for developing the product and product advertising.
3. Indicates segments of the population in which the appeal of the product is likely to be concentrated.

We see then, that concept testing provides insight into market potential for a product design project. It tests the success of a new product idea before it is marketed. The concept should be developed to the point that it conveys the product attributes, the desired positioning and the intended brand personality. The actual product concept test is conducted by defining and testing reaction to the core concept through exposure to a story board, sketches, graphics, or even a product mock-up.

The concept test is pre-design, and differs from a pre-test or test market, which are conducted later in the development cycle — based on a finalized product design. Pre-tests and test markets are a final check point at avoiding a major market error.

The term concept analysis is also associated with the process of providing “proof of concept,” which is often required for Venture Funding, and can also apply to communications copy development.

What are the Benefits of a Concept Test?

A concept test can provide a variety of benefits, including:

1. A product reaction study, including liking and intention to purchase
2. A tool that can be customized to profile your own product or service idea
3. Consumers can evaluate and provide feedback early in the development process
4. A management tool for measuring potential success and to refine the concept
5. A screen for unproductive ideas and a way to optimize the investment of time and resources

Problems Solved with Concept Tests

• New Product Concepts — Identify which benefits are most important to customers and which features are most likely to lead to the fulfillment of that promise. Features can be categorized into those which are “need to haves” vs. “nice to haves.” Customer need must be identified and prioritized for product development and advertising.
• Product Modification/Upgrade — modifications and upgrades can reformulate and add new life to existing products and services. Again, identifying the optimal bundle of features is a priority. Differentiating those features that are “need to haves” vs. “nice to haves” is critical in creating products or services that are truly “New and Improved” “New Release” and “Upgrade” worthy.
• Migration path — Many products and services offer upgrade or migration paths. For the customer, it is a reflection of a need for the next level of sophistication. Understanding the key features and benefits is critical in mapping consumer needs to the likelihood of upgrading an existing product or adopting a new technology. Our experts will help you answer such critical questions as “Do benefits outweigh the costs and challenges of changing?” Features, Benefits, Brands, Image, Costs and Training are but a few factors that must be considered.
• Product Usability — Serviceability — The most effective concept tests assess the use experience of a specific product or service and determine how that experience can be improved. This type of concept test research can focus on a variety of areas — ease of use and similarity to current usage patterns, the ability to adapt and use critical feature implementations, and the congruency with current image, usage patterns and service provisions.
• Pricing and Incentives — no one underestimates the importance of price expectations in new product adoptions. Price, incentives, bundling, cross product tie-ins, and cost mitigating factors such as warranties… all change price perceptions and perceptions of value.

How do I Conduct a Concept Test?

Conducting a concept test can be broken into three distinct research tasks:

1. Pre-Screen concepts that have some potential of being researched.
2. Refine the concept
3. Conduct evaluation tests of the final concept, comparing it to other options and the competition

Different Kinds of “Concept” Tests

We identify several different types “concept tests” based on their timing in the product development cycle and the purpose or information that is desired. Concept screenings are conducted early in the product development cycle and are generally followed by positioning — demand analysis as the concept is finalized. Pre-market concept tests involve product comparisons and are conducted to identify competitive challenges and weaknesses in the concept that is being developed.

Type I: Concept Screening Tests

Concept Screenings represent product ideas that are presented to consumers in verbal or visual form and then quantitatively evaluated by consumers to measure degrees of concept believability, personal relevance, purchase intent, likelihood of trial, and similar indicators of product potential. Concept screenings typically identify:

• Concept ideas that are sufficiently promising to merit further consideration and development
• Believability, Relevance
• Perceived Uniqueness, Value
• Trial-ability, or the potential for trial
• Relative Attribute Performance Advantages
• Multiple measures of attribute desirability
• Measures of trial potential based on multiple measures that go beyond purchase intent.

Type II: Concept Tests

Concept Test — Positioning — Demand Analysis involves a concept evaluation where concepts within the same functional product class are positioned and evaluated together, to determine relative advantage and potential share.

Positioning — demand analysis will often include selected measures from the Type I Concept Test, plus measures that…

• Present consumers with test concepts defined to include the concept and competing brands in product consideration sets.
• Identify potential opportunity size for a chosen concept, including preliminary sales estimates.
• Provide an in-depth profile of potential adopters for targeting in the product launch.
• Strengths/Weaknesses analysis based on attributes/benefits
• Awareness/Distribution Impact

Type III: Pre-Market Concept Tests

Pre-Market Concept Tests present concepts to consumers for evaluation. The concept is compared to competing products to determine if the product delivers what is promised by the concept.

• The concept test measures attractiveness of a new product or service before its launching into the market by identifying its strengths and weaknesses.
• The concept test evaluates the level of agreement between the concept and the specific brand attributes.
• The concept test detects communication problems that may interfere with appropriate comprehension by the target segment.
• The concept test provides purchase intention indicators, with scenarios varying from most optimistic to most conservative.
• The concept test minimizes the risk of failure in the market by allowing product and communications adjustments before launch.

Accuracy Issues in Conducting Concept Tests:

Concept tests provide a top level analysis of overall concept preference and the associated likes and dislikes about a concept. The accuracy of the concept test is affected by many items, including the implementation of the physical concept after testing. The actual service must be accurately reflected in the communication of benefits provided to the consumer. Market communications must provide an accurate mirror image of the benefits provided by the product and those benefits that the potential consumer believes they will be receiving. Otherwise, expectations are “disconfirmed” or not realized and dissatisfaction results.

Concept Screening Test Procedures

Develop A Concept Story Board

Copy and possibly photograph or illustration that describes how the product works and its end benefits.

Interview

• Concept testing is usually conducted in a central location (could be done by telephone or mail), but typically done by intercept method (at mall, food store, or other traffic location).
• For new product concept screening, 5-8 concepts are often viewed.
• For testing alternative products for purchase and incentives, concept screening tests use 10 or more products.

Questions Asked

• Purchase intentions
• Purchase frequency
• Uniqueness of the idea
• Believability of the idea
• Importance of the message
• If alternatives (alternative buying incentive concept screening test) are presented, respondents are asked to evaluate concepts according to how interested they are in purchasing the product

Analysis Approach

1. Examine intention scores…
2. Combine “definitely would buy” and “probably would buy”
3. Classify the concepts in to schema:
   • Green: Two “thumbs up” from the consumer: High Potential
     a. Winners — High purchase intent/high uniqueness
   • Amber: Acceptable… maybe, but with modifications: Caution
     b. Me too / Generic products — High purchase intent/low uniqueness
     c. Fad / Specialty products — Low purchase intent/high uniqueness
   • Red: Two “thumbs down” from the consumer: Rejected
     d. Long shots — Low purchase intent/low uniqueness

Concept Test Evaluation Procedures:
Conducted After Initial Screening and Concept Refinement.

Purpose:

• Assess market potential for each product (or product positioning)
• Determine a product concept’s strengths and weaknesses
• Provide an indication about the market segment in which each proposed product is likely to have the greatest acceptance.
• Give an indication of whether to initiate a test market.

How do we Implement the concept test?

• Story board, commercial advertisements, actual product prototypes
• Concepts have been refined and screened, so that only 2-3 concepts are actually tested

Who will be interviewed?

• Concept testing is usually conducted in a central location (could be done by telephone or mail), but typically done by intercept method (at mall, food store, or other traffic location).

Questions asked (see below for further concept test questions)

• Purchase intentions
• Purchase frequency and volume
• Key benefits (open ended questions)
• Likes and dislikes (open ended)
• Believability (open ended)
• Uniqueness
• Overall/attribute ratings
• Key benefit importance
• Demographics

Concept Test Analysis approach

1. Standard techniques
2. Combine “definitely would buy” and “probably would buy” and test across alternative concepts.
3. Classify the concepts in to schema
   1. End benefit importance
   2. Concept attribute ratings
   3. Uniqueness
   4. Believability

What to Look For in a Concept Test

• Identify the effectiveness of individual concept assets
• Identify the barriers to achieving the concept’s full potential
• Identify consumer relationships with the concept
• Identify the status of the concept in a competitive context
• Identify failure of advertising to communicate clearly
• Identify failure of packaging that is too non-intrusive
• Identify failure through unmemorable product personality
• Identify failure through consumer’s lack of value perceptions
• Identify the brand equity assets
• Identify potential consistencies in brand image and consumer perceptions
• Identify potential inconsistencies in brand image and consumer perception
• Identify unrecognized product benefits
• Identify failures of the brand to integrate into the consumers’ lifestyle
• Identify a blurred image
• Identify an elusive brand/concept personality

Survey Components for a Basic New Product Concept Study

Concept tests can be easily constructed if the researcher is aware of the key components that should be included in the test instrument or survey. Not that these components will often vary, depending on the purpose of the concept study.

The researcher is urged to carefully consider the objectives of the concept test and to then determine if the measures used in the study will successfully answer these questions. Pre-test the instrument by reviewing the purpose, the results and then making sure that the study purpose is answered with exactness. The major components of concept tests are:

• Concept Test Presentation:
  Describe the concept completely. Use graphics, video, audio, samples or whatever best portrays the concept and associated attribute and benefit messages that are being tested.
• Overall Concept Test Reaction Measurement
• Concept need / relative improvement over current method of doing things
• Overall reaction to the concept (acceptability, desirability, interest)
• Likelihood of purchase of concept
• Detailed Concept Test Analysis Evaluation
• Likes and Dislikes about the concept
• Attribute list evaluation
• Awareness of competing products
• Awareness of substitute and complementing products
• Superiority over other existing products
• Use Situation Evaluation
• Likelihood of use in specified situations
• Current use of similar / competing products
• Frequency of product use
• Value Analysis Evaluation
• Evaluation of product value
• Price sensitivity analysis
• Preferred method of purchase
• Segmentation Analysis
• Market segment most likely to use
• Type of communications (message, source, media) that would help you decide to purchase

Task Specific Studies Related to Concept Tests (and their components)

Survey Flow for a Typical Concept Test

• Introduction and qualify respondents (not part of a disqualification group)
• Awareness of product brands
• Purchases of product Brands in past 3 months
• Introduction of concept description with measures of likelihood of purchase
• Purchase dimensions: number of bottles, frequency of purchase
• Perceived value of the product concept
• Innovativeness of concept measure
• Affective evaluation: like-dislike measure
• Power to replace: current product you purchase
• Frequency of consumption in a day
• Believability of concept
• Relevance of concept to the respondent
• Attribute and benefit evaluations:
  • is a good value for the money
  • would help me to … Be mentally sharp
  • would (benefit, such as: taste good or be something i would eat everyday)
  • has attractive packaging
  • would provide me with a high level of health
• Source of purchase (location)
• Demographics

Closely related to the concept test is the habits and uses study. Habits and uses studies are directed at understanding usage situations: how, when and where the product is used. Habits and uses studies sometimes include a real or virtual pantry audit. Understanding current consumer practices goes a long way in understanding preferences in a new concept test. The major components of the habits and uses study include:

General Measures for a Habits and Uses Study

Frequency of Product Use:

• Identify usage segments: User/non-user, frequency and amount of product use (Identify Heavy, Medium, Light, Non-user segments)
• Comfort with use of product / service (are you a product user; do you currently own…)
  Relative use: more or less of a user than other people using the product

Situational Use:

• Primary and Secondary Product Use Situations
• Primary use location
• Critical issues and needs for product / service use
• Identification of use situations (+ identify new uses that are currently unknown to you/your company)
• How do you use the product?
• How have you used the product (in what ways, applications, uses, problems solved, experience)
• Reason for not purchasing more of the product / service (current or alternative use situations)
• How has the customer studied or learned how to use product more effectively
  (is there a user group, a sub-culture group?)

Experiential Use:

• Identification of Brand Usage Experience
• From whom did you learn to use the product? (sphere of influence)
• Which brands have you used?
• Which brand are you currently using?
• Beliefs about product / service use benefits
• Identification of stereotypical user profile

Use Satisfaction ‘ Use Loyalty:

• Satisfaction with current use of brand/product/service
• Loyalty of use (may be a series of items… see Product Loyalty Measures)

Product Fulfillment Analysis of Attributes, Features, Benefits

Products contain bundles of benefits (both tangible and image) that are promised to the user. The product fulfillment study determines if the expectations created for the product by advertising, packaging and the produce appearance are fulfilled when the product is used.

Product Use Scenario to Understand Fulfillment

• Use Scenario Descriptors (where and how is the product used?)
• Frequency of product use
• Primary use location (home, work, etc.)
• Primary precipitating events or situations for product use or need
• Usage rate trend (more, same, less than a month ago )

Product Familiarity

• Degree of actual product use familiarity
• Knowledge (read product information, read product label, etc.)
• Knowledge base of product ‘ Are they “hard core” consumers, a “Maven,” the super love group.
• Company Contact: Have they called the 800 number , etc.
• Awareness of other brands
• Reasons for original product purchase (selection of reasons)
• Primary benefits sought from the product

Product Evaluation

• Attribute evaluation matrix question (quality, price, trust, importance, performance, value)
• Perceived benefit associations matrix
• Importance, performance
• Identification of primary benefits sought
• Comparison to other brands (better, worse)
• What is the best thing about the brand, what could be done better

Advertising and Packaging Evaluation

• Packaging size, design
• Advertising Promise, message fulfillment evaluation

Value Analysis

• Expectation of price
• Expectation of relative price (full price, on sale)
• Current price paid
• Satisfaction Measurement
• Overall Satisfaction

Product Tests and Advanced Concept Tests using Conjoint Analysis

The basic forms of concept tests identify the individual attributes describing the concept, or overall concept preference. However these basic methodologies do not identify the effects of combinations of attributes and how to modify attributes to optimize consumer acceptance and preference. Conjoint analysis is the methodology used to conduct this type of concept or product test.

Introduction

Conjoint analysis answers the question of which attributes are important to consumers and how important they really are. Taken in combination, individual product attributes can be used to describe an entire product. Conjoint analysis determines the combination of product attributes that consumers most prefer. Conjoint analysis, when applied to product, service, and communications projects identifies which product and service attributes, or which communications messages are most preferred and are best combined to produce maximum effect.

Conjoint Analysis originated out of the mathematical psychology research of conjoint measurement¹. Green and Wind² state that conjoint measurement is “concerned with measuring the joint effect of two or more independent variables on the ordering of a dependent variable.

The output of conjoint measurement consists of the simultaneous measurement of the joint effect and separate independent variable contributions to that joint effect, all at the level (asymptotically) of interval scales with common unit.” ” From the standpoint of multiattribute choice making, conjoint measurement can sometimes be used to decompose overall evaluation into implicit utilities for components of the multicomponent alternatives”

In layman’s terms, conjoint analysis (1) identifies the attributes important in a choice decision, (2) identifies the way the attributes are combined to make the decision, and (3) determines the utility value to each of the levels of each of the attributes considered in the decision.

Green and Wind further point out that the method of conjoint analysis used represent the different theories of how people choose between multi-attribute alternatives. Conjoint analysis attempts to jointly identify the composition model for decision choices and at the same time estimate the utility value of the attributes that are important in the choice decision. As the choices are analyzed, the researcher may predict choice share for different product configurations that may be introduced into the competitive marketplace.

Conjoint Analysis in a Nutshell

Conjoint analysis is a methodology for the measurement of psychological judgments, such as consumer preferences. Stimuli (product configurations, advertisements, movie endings, etc.) are presented to the respondent for evaluation.

For example, a respondent may be presented with a set of alternative product descriptions (automobiles). The automobiles are described by their stimulus attributes (level of gas mileage, size of engine, type of transmission, etc.). The respondent views selected alternatives and choice or preference evaluations are made.

From this evaluation or choice information, the researcher determines the respondent’s utility for each attribute level (i.e., what is the relative value of an automatic versus a five-speed manual transmission). Once the utilities for all attribute and all levels are determined for all respondents, the analysis of the utility data can begin.

Preference curves are identified for each attribute so as to show how the market of consumers values each of the different attribute levels. This analysis may be conducted for all respondents or for selected market segments.

Simulations are then run to determine the relative choice share (and thereby estimate market share) of competing sets of new or existing products.

Further explanation of conjoint analysis is found in the Qualtrics tutorial document “Introduction to Conjoint Analysis”.

Concept Tests and Product Tests Using Conjoint Analysis

How does this product perform when evaluated by the consumer?

Basis for evaluation…
Isolation, competitive frame, against its advertising, against formula variation, etc.

Purposes of product tests:

1. Tests against the competition:

   Seek to identify which of many alternative new formulas is best in terms of being most preferred or to measure the performance of the new product relative to other competitive products.

2. Product improvement tests:
   Determine whether an improved formula or construction should replace the current product
3. Cost-savings tests:

   Determine whether a less expensive product should replace the current product

4. The fit of concept tests:

   Determine which of several test-product variants most closely resembles what is being communicated by the selling concept

Types of product tests

Monadic product tests: one product is presented and an evaluation is requested, with no other specific product comparisons.

Comparison product tests: two or more products are presented and a comparison and rating of each is requested:

• Paired comparison
• Repeat paired comparison
• Sequential monadic designs
• Round robin designs
• Proto-monadic designs
• Triangle designs
• Paired comparison designs
• Duo-trio designs
• Discrimination/difference
• Choice based conjoint
• Choice Based Conjoint
• Hierarchical Bayes Conjoint

---

Scott Smith is the founder of Qualtrics.com. He is the James Passey Professor of Marketing and Director of the Institute of Marketing at Brigham Young University. He received his Ph.D. in Marketing and Quantitative Methods from Pennsylvania State University.

This entry was posted by admin on Friday, May 25th, 2007 at 4:47 pm and is filed under Concept Testing, Market Research.