Fostering Sustainable Behavior

Design & Evaluation

If a program is to be effective, careful consideration needs to be given to strategy development. This chapter will clarify how to design, pilot, implement and evaluate a community-based social marketing strategy.

The development of a strategy begins with identifying barriers to the desired activity, using the tools described in Chapter 2. The identification of barriers is followed by strategy. Once the strategy is complete, the next step is to conduct focus groups to obtain reactions to the proposed strategy. If the strategy receives positive reviews, you are ready to pilot. If not, you will want to make further refinements. In the pilot, you test the effectiveness of the strategy with a limited number of people. Essentially, you want to know, before you commit to using the strategy throughout a community, that it will work effectively. If the pilot is successful, you can be much more confident of success when you broadly implement the strategy. If the pilot is unsuccessful, then you need to make further revisions, and pilot again before broad-scale implementation and evaluation.

As can be seen above, the design of a community-based social marketing strategy is pragmatic; each step builds on those that precede it. Effective design will not only help ensure the success of a program, but can also serve one other important purpose; cementing funding support. Increasingly, funders are demanding that projects have a solid research foundation and be piloted before being implemented. The tools introduced in this chapter can help you to persuade your funders that your initiative is worth supporting.

Design and Evaluation: An Example

To introduce the design and evaluation of a community-based social marketing strategy, a hypothetical program to foster the purchase of products with recycled-content will be used. Following this example, critical elements of design and evaluation will be outlined.

Imagine that preliminary research (see Chapter 2) has identified the following barriers to consumers purchasing products that have recycled-content:

• products are viewed as difficult to identify;
• shoppers forget to consider whether a product has recycled-content; and
• buying recycled-content products is not seen as the "right thing to do."

Knowing that recycled-content products are difficult to identify suggests that prompts should be effective in promoting these purchases (see Chapter 4). That consumers forget to consider these properties when making a purchase also suggests that prompts may be an effective tool in promoting the purchase of products with recycled-content. Finally, that buying these products is not seen as the "right thing to do" clarifies that an effective strategy will need to foster supportive norms (see Chapter 5).

What might a community-based social marketing strategy look like which incorporates these behavior change tools? As mentioned in Chapter 4, prompts are most effective when presented at the time the activity is to occur. To encourage the purchase of recycled-content products, prompts would be placed on the store shelves directly below these items. To assist shoppers in easily identifying these products, a graphic design that visually suggests the importance of purchasing products with recycled-content would be used (see Chapter 4 for an example of such a prompt). The prompt would also contain a brief explanation of why buying products with recycled-content is important (remember that for a prompt to be effective, it needs to contain all of the information that is necessary for someone to act appropriately).

Occasionally it is possible to overcome two barriers to a sustainable behavior with one tool. In encouraging shoppers to select products that have recycled-content, the use of prompts makes it significantly easier to identify these products (the first barrier) and increases the likelihood that shoppers will remember to consider these characteristics (the second barrier).

How might community norms be established that foster purchasing recycled-content products? At the beginning of the promotion, asking shoppers to wear a sticker or button that said "I buy recycled" would likely help to establish the community norms discussed in Chapter 5. Asking shoppers to wear a sticker or a button as they enter the store will not only help to establish a norm favouring the purchase of these products and build commitment, but will also serve to highlight the campaign for these shoppers. Nonetheless, the sticker and button will only be worn for a short time before it will be removed. A more permanent way to establish community norms that support the purchase of these products is to ask householders to place a sticker on the side of their blue box that indicates that the household shops for recycled or recyclable products. The development of community norms can also be facilitated through the use of block leaders (see Chapter 6: Communication) who seek commitments from householders to purchase products that favour the environment and indicate how to go about doing so. Asking shoppers to wear a sticker or button not only assists in establishing these norms, but also bolsters commitment (see Chapter 3). Since people wish to behave consistently, agreeing to wear a button or sticker increases the likelihood that they will purchase recycled-content products.

Posters clarifying the meaning of these prompts, or "shelf talkers," would be placed prominently throughout the store (particularly near entrances) In addition, pamphlets at checkouts and potentially a mobile video kiosk would be used to educate shoppers about the importance of selecting products with these characteristics.

The proposed social marketing strategy deals with each of the identified barriers to purchases of products with these characteristics. However, simply selecting and incorporating the tools discussed in this book into a community-based social marketing strategy will not ensure its success. Prior to implementing a strategy throughout a community, it should be tested through focus groups and a pilot.

Focus Groups

While focus groups, as explained in Chapter 2, can be used to explore barriers to a behavior. Focus groups can also provide useful information on the appeal and acceptance of a proposed strategy. To obtain feedback on the above strategy, several focus groups of five to six individuals would be conducted. For each focus group, the purpose of the campaign would be explained and participants would be introduced to drafts of the proposed prompts, stickers (buttons), brochures, posters and video. Focus group participants would be asked whether these materials would capture their attention and if they are clear and easy to understand (see Chapter 6). Once feedback has been received on these characteristics of the materials, participants would be asked if they perceived any difficulties with the proposed strategy and if they had any suggestions for how it could be strengthened.

Following completion of the focus groups, responses to the proposed strategies would be tabulated to uncover any potential themes in participants' responses Where warranted, the strategy would be refined based on the feedback received. After refining the strategy, the pilot is conducted.

Pilot

Think of the pilot as a "test run," an opportunity to work out the "bugs" before committing to carrying out a strategy across a community. To pilot the above strategy, the store managers of two supermarkets would be approached and asked if they would be willing to participate. The two stores would need to be similar both in the demographics of their shoppers as well as in the products available (two stores of the same chain would be good candidates). By the flip of a coin, one of the stores would be randomly assigned to receive the community-based social marketing strategy, while the other would serve as a comparison (what is referred to as a control).

Prior to piloting the strategy, the rate of purchase for recycled-content products would be determined by examining the computerized inventory records for these items. Note that it is important to collect this data for both stores, since they may differ initially from one another. Also it is important to obtain this baseline data for a sufficient period of time (usually a month or more). Following the baseline period, the prompts, posters, buttons (stickers), pamphlets and video kiosk would be introduced in the intervention store. After introducing the strategy, the rate of purchase of targetted items would be monitored for several months to ascertain if the strategy produces a sustained impact upon the purchase of these products.

To determine whether the strategy alters consumer purchases, the purchase of the targetted items during the baseline period is compared to purchases during the intervention (seasonal adjustments may need to be made to these numbers to control for increased purchases around events such as Christmas). The success of the strategy cannot simply be determined by comparing the purchases of the targetted items for the two stores. The following example clarifies how to correctly determine the impact of the strategy. Imagine that after implementing the above strategy, the "intervention" store had sold 5000 units of recycled-content toilet paper, while the control store had sold only 3000. On first glance it appears that the community-based social marketing strategy has brought about a 67% increase in sales for this one item. However, such a conclusion assumes that the stores initially sold an equal amount of recycled-content toilet paper, which is very unlikely. To determine the "real" impact of the intervention, the sales of toilet paper during the baseline period for both stores needs to be considered. Imagine that baseline data revealed that the intervention store had sold in the month prior to the intervention 2500 units of toilet paper and the control store had sold 2000. The real increase in sales that can be attributed to the intervention, 50%, would be determined as follows:

If, when comparing inventory records prior to and following the implementation of the intervention, little or no change in consumer purchases is observed, then the pilot would need to be revised until significant changes in behavior were observed. Since in this proposed initiative the prompts are a central aspect of the campaign, it is natural to start by investigating them. By conducting in-store surveys with a random selection of shoppers, awareness and understanding of the prompts could be probed. If low recognition and understanding of the prompts was observed, then the prompts would need to be redesigned to be more prominent and clear. Further, the placement of the posters which explain the purpose of the shelf talkers should be examined. Did shoppers recall seeing the poster? Did they know what the posters said? If the answer to either of these questions is no, it is possible that simply changing the location and/or number of posters might address this problem. The point of the pilot is to identify and address these problems before launching the campaign throughout the community. You should plan on there being problems and build into your plans the opportunity to refine your strategy until it works well. On one project, I revised a pilot six times before I was able to produce the desired changes in behavior While it was frustrating to have to make this many revisions, I was thankful that I was making the revisions to a pilot rather than to a larger project, for which the problems would have been much more difficult and expensive to rectify. Expect problems, plan for them; in the end, when you implement community-wide you will be rewarded for the time that you took to trouble-shoot.

Community Implementation and Evaluation

When the pilot has successfully demonstrated that the purchase of these products can be substantially increased by the community-based social marketing strategy, it is ready to be implemented across the community. In implementing the initiative, advertising and local media can be used to create additional awareness that would have been undesirable during the pilot In implementing this initiative throughout the community, limited advertising resources could be leveraged by creating public awareness through hosting media events to both launch the campaign and provide feedback on its success. Further, participating retailers could be encouraged to advertise the campaign in their own advertising, greatly increasing exposure (this is a requisite for some levels of involvement in the "Buy Recycled" campaign discussed in Chapter 4).

When implementing throughout the community, it is also important to build in a method to evaluate the impact of the initiative. In the hypothetical project described here, a random selection of retailers would be selected to participate in the evaluation. Baseline data from the electronic inventories of these stores would be obtained and then compared to changes that occurred in the purchase of the targetted products following the launch of the campaign. To provide a stable picture of the impact that this campaign had upon the purchase of these products, the average increase in the purchase of these products across the evaluated stores would be determined. This information not only serves as a critical test of the success of the initiative, but serves two other important functions. First, it is important to provide the community with feedback (see Chapter 6) regarding the impact that their changes in behavior has upon the environment. In other words, an element of a successful community-based social marketing strategy is providing feedback that reinforces changes that people have made. The media will often provide you with a cost-effective way of getting this information back to consumers, though other possibilities exist. One vivid and ongoing form of feedback is to provide shoppers in each retail store with a yardstick of their efforts. By setting up a display in which the percentage increase in the purchase of these products is updated on a regular basis, shoppers can be provided with an ongoing source of feedback and encouragement (the use of feedback can also help to establish a norm that favors this form of shopping). Second, as mentioned at the beginning of this chapter, program evaluation provides evidence of concrete results, which is most convincing to funders that a campaign is worth continued support.

Design and Evaluation Principles

The preceding example demonstrated many of the critical aspects of designing and evaluating a community-based social marketing strategy. This section provides an overview of design and evaluation principles.

Begin with Barriers:

The development of any community-based social marketing strategy begins with the identification of barriers. Using the methods outlined in Chapter 2, identify barriers to the activity you wish to promote prior to giving further consideration to designing a strategy. As you identify barriers, keep in mind that most activities consist of a variety of component behaviors. You need to know the barriers for each of these component behaviors if your strategy is to be effective.

Prioritize the Barriers:

In identifying the barriers, use statistical analysis (multivariate approaches) to help you clarify which barriers are the most significant. It is likely that your research will identify a number of barriers and you will want to ensure that your limited resources are spent on overcoming the most important barriers.

Select Tools that Match Identified Barriers:

To design an effective strategy, it is essential that the tools you select are tailored to the barriers you encounter. For example, if motivation appears to be a problem, consider the use of commitment (Chapter 3) or incentives (Chapter 7). If few people perceive the activity as the "right thing to do," you will likely want to develop community norms using some of the strategies that are discussed in Chapter 5. If there is a lack of awareness or knowledge regarding the activity, you will want to incorporate many of the tools of effective communication that are discussed in Chapter 6.

Scrutinize your Design with Focus Groups:

Prior to piloting your strategy, conduct focus groups to receive feedback on your proposed strategy. The information you obtain from these focus groups will often assist you in designing a more effective strategy.

Use a Minimum of Two Groups to Conduct your Pilot:

When you conduct your pilot, you want to make sure that any changes you observe are the result of your intervention and not other events that are occurring in the community. To be certain that it is your intervention that is bringing about the changes you observe, always include a control group to which nothing is done. By comparing your intervention and control groups, you can be much more confident that your intervention was responsible for any changes you observe.

You may wish to have more than two groups. For example, as in many of the studies described in this book, you may wish to have one group receive a commitment strategy, a second receive feedback, a third receive a combination of the two, and a fourth act as a control. Keep in mind that pilots can often be quite inexpensive to conduct since the size of groups can be kept small (30 to 40 residences each). Including multiple groups in your pilot can help you determine the form that your strategy will take when you implement it across your community. For example, as a result of conducting a pilot on fostering car pooling, you may learn that obtaining commitments provides no additional benefit over assisting employees in identifying others who live in their neighborhood that tyey might drive to work with. As a result, your subsequent program would drop commitment as part of the strategy.

Use Random Assignment:

When you conduct a pilot, you want to know that the group that receives your intervention is as identical as possible to the group that serves as the control. The only way that you can assume this is if the people are randomly assigned to be in one group or another. To randomly assign individuals or households to the groups you plan to use, simply place the names or addresses of all individuals in a hat and then pull them out assigning the first person or address to the first group, the second to the second group, etc.

Make Measurements of Behavior Change a Priority:

In evaluating the effectiveness of a pilot, your primary concern should always be whether you were able to change the behavior that you set out to change. Where possible, don't rely upon people's self reports of their behavior; they can be unreliable. Obtain water records, ask to look in composters, examine weather-stripping, etc. You will also want to examine people's perceptions and attitudes, but don't see these as substitutes for examining actual changes in behavior.

Revise your Pilot Untill it is Effective:

It is tempting when a pilot is ineffective to assume that you know what went wrong and move directly to community-wide implementation. Keep in mind that pilots can often be conducted very quickly. Take the time to run another pilot to confirm that you are actually able to change behavior before you implement across a community. The extra time that you take to run the pilot may save you hundreds of thousands of dollars if your intuition has betrayed you.

Evaluate the Community Implementation

Prior to conducting your community-wide implementation, collect baseline information about the rate at which people are presently engaging in the activity you wish to promote. Where possible, use actual observations of behavior or reliable records (e.g., water meter readings) rather than self-reports to establish this baseline. Once you have implemented your program, begin to collect data to ascertain its impact. Keep in mind that you will want to conduct these evaluations at different time intervals.

Guidelines for Selecting Consultants

You may wish to contract out the design, implementation and evaluation of your program. Here are some suggestions to increase the likelihood that you end up with a consultant who has the necessary skills to use community-based social marketing. In the request for proposals ask that proposals:

• be based upon community-based social marketing methods;
• specify how barriers to the activity will be identified;
• clarify what behavior change tools might be used (e.g., commitment, prompts, norms, social diffusion, etc.);
• indicate how the strategy will be piloted;
• specify how the program will be evaluated once implemented throughout the community;
• provide evidence of competence in survey design, research design and data analysis (at least one member of the research team should have graduate level training in research methods and statistics); and
• provide evidence of familiarity with designing and implementing community-based social marketing strategies.

Public Consultation

Community-based social marketing is based heavily upon public consultation. As explained previously, the process of designing a strategy involves obtaining information from the community at three separate times. First, just after conducting the literature review, focus groups are conducted to obtain in-depth information on perceived barriers to the behavior you wish to promote. Second, this information is supplemented by the phone survey, which provides more information about perceived barriers, attitudes and present levels of involvement in the activity. Third, the social marketing strategy is reviewed by another series of focus groups who provide feedback on the planned strategy. These three steps help ensure that the strategy you devise will be well tailored to your community.

This consultation should be part of the development of any community-based social marketing strategy. However, you may wish to add another opportunity for public involvement - active participation in determining the initial marketing strategy. Some organizations create a stakeholder consultation committee for this purpose. Whether you elect to create a stakeholder consultation committee, and if you do, when they become involved in the process is a matter of personal preference. My own preference is to create a stakeholder committee whenever the planned program is likely to be of special interest or concern (e.g., implementing user fees for garbage disposal), or when the activity you are attempting to promote is not well understood and hence you need input from as many sources as possible. It is often not necessary to create a stakeholder committee when neither of these two criteria are met.

If you decide to form a stakeholder committee, it can be formed at the outset (e.g., prior to the literature review) or after information from the literature review, focus groups and phone survey have been conducted. Once again, when you decide to form the committee is a matter of personal preference. I prefer to create the committee at the outset if the program has any potential to be controversial In order to circumvent concerns about decisions being made without public input. On the other hand, early creation of the committee can make some initial parts of collecting information on barriers, such as the survey, torturous if not well managed. Don't place yourself in the position of writing a phone survey by committee. Do seek suggestions about potential topics that should be addressed in the survey, but avoid having the survey reviewed by the stakeholder committee.

Independent of when you elect to involve a stakeholder committee, you will need to decide beforehand what constraints will be placed upon the committee. For example, if council has made it clear that no subsidies will be provided for the installation of low-flow toilets, your committee needs to know at the outset what limitations have been placed upon the strategies that can be considered. If you are going to be acting as a facilitator for stakeholder meetings, remain impartial when receiving feedback from participants. Your role is to encourage constructive input on the design of a strategy. Remaining impartial will facilitate receiving the broadest feedback.

The Final Report: Getting the Word Out

After conducting a literature review, running focus groups, writing, conducting and analyzing a phone survey, devising a strategy, scrutinizing it with focus groups and a stakeholder committee, piloting the strategy, revising the strategy, implementing it throughout the community and evaluating it, you should be finished, right? Wrong. Community-based social marketing is an emerging field that holds great promise for moving us toward a sustainable future. Take the time to write up a final report and make sure that people know about it (consider adding it to the cases database found at this site). Whether your community-based social marketing strategy was successful or not, others need to learn from your efforts.