Richard A. Krueger

ABOUT US

FOCUS GROUP
INTERVIEWING

EVALUATION

STORY-TELLING

LINKS

HOME

I. Why would you want to develop a Logic Model?

Simply stated, does the program do what it was supposed to do? The logic model helps answer this question. Educational programs vary greatly. Some programs are hastily put together and others go through lots of thought and deliberation. Some are one-shot efforts and others are multi-stage and continue for years. Some are designed to change behavior and others are designed to just create awareness or to perhaps impart new knowledge. In addition, some are successful and others are not. Sometimes it is hard to tell if a program is successful. A program might be popular or entertaining, but lack substance. Just because a program is popular doesn't mean that it meets the goals for which it was intended.

Logic models help educators. The logic model pushes the program designer to be explicit. The logic model increases the odds that the program will be successful---to accomplish the goals that were established for that program.

Logic models improve program quality. The model helps eliminate gaps or logical problems with the program. Also, the logic model clearly identifies the consequences or outcomes of the program.

Logic models build community support. The model helps the public to clearly know what is expected of the program effort. When programs are clear, then the customers know what they can expect and collaborating partners know how they can contribute to the effort.

Logic models enhance accountability. Logic models help professionals be accountable in that it identifies clearly what is expected. The logic model lets others know what the program is supposed to accomplish and give indications of how you will measure that accomplishment.

Logic models improve evaluation. Evaluators use the logic model as a blueprint for the evaluation. Effort is made to measure progress at each step in the model. Often the final goal is difficult to measure, so the evaluation strategy is to document progress toward the final goal.

II. Introduction to Logic Model / Theory of Action

The theory of action or logic model describes how a program works. The theory can be examined for an existing program or for a potential or proposed program. The logic model makes the underlying assumptions explicit and reveals the inner-workings of the program.

Educational programs are based on something. It may be a theory, past experience, tradition, or a creative idea. Each program contains a theory of action which may be implicit (assumed) or explicit (specifically stated). This process seeks to identify an explicit theory of action. The explicit theory of action identifies the sequence of events and the anticipated effect on program participants. This logic model guides the professional in planning the program. Expected changes in participants are usually sequential (i.e. first awareness, then knowledge, and finally changes in behavior) and also logical to the planner. Sometimes the logic for the program is ambiguous, implausible, or even missing altogether.

Here is a simple example of a logic model

1. Educator writes newsletter to improve farm management
2. Educator sends newsletter to farmers
3. Farmers receive newsletter
4. Farmers read newsletter
5. Farmers understand newsletter
6. Farmers believe newsletter
7. Farmers gain knowledge from newsletter
8. Farmers have resources needed to use the knowledge (skills, time, money, support)
9. Farmers want to make changes
10. Farmers make changes
11. Changes made by farmers improve overall farm management

In this logic model each step must be completed for the next step to work. For example, the educator assumes that farmers read the newsletter. If they don't, the following things won't take place.

III. How to Develop a Logic Model

Here are some steps that can guide you when developing a logic model.

1. Begin by doing it yourself. Do it in writing because it leaves a trail that you can later review and investigate. Try to be as clear as you can about each step in the program.

Or

2. Begin with a small group of colleagues. Some people think best when they are in a group and get ideas from others. If so, invite some co-workers to join you for a discussion of how the program works.
3. Build a beginning model of how the program operates. Caution: Don't fall in love with your model at this point. You'll want to be open to how others see the program and make revisions based on feedback from others. Consider using one of the following strategies for building the model.
• The Graphic Model---Draw a picture of the program
• The If-Then Model
• The Inputs-Outputs-Outcomes Model
• The Theoretical Model
These strategies are described below in greater detail. An interesting process with a group of people is to have them work in groups of 2-3 people as they develop the logic model and then have them share their results. Then develop a composite model based on what was learned from the small groups.
4. Get feedback on the model. Share the model with others such as colleagues, (within and outside of your organization), planning committee members, past users of the program, or even friends and neighbors.
5. Continue to revise. You may want to develop several different logic models. For example, you might have a model from the perspective of the designer, a model from the perspective of the user, or a model from the perspective of a non-user, etc.
6. Use the model for evaluation. When you feel that you have a reasonably sound model, use it to determine the evaluation strategy. Identify those places in the model where evaluation evidence can be gathered.

IV. Advice to Consider when Developing Logic Models

• Focus your attention on what is happening to individuals instead of organizations or the community.
• The closer you are to the program the greater potential there is for you to make assumptions and overlook critical parts of the model.
• Work with others to develop the model.
• Begin talking with those closest to the program (such as professionals) and then broaden your listening to other groups.
• Be open to the possibility that you might have more than one logic model operating.
• Begin by anchoring the two ends---the end product and the beginning step. Then, attempt to connect these with activities, actions, events or experiences that might occur. Keep each step specific, clear and measurable.
• Consider the principles of adult learning. Adults are more likely to acquire new knowledge and to use that knowledge if . . .
• They have multiple experiences when receiving the new information (e.g. reading, hearing, seeing, doing, talking)
• They have an opportunity to see a trial or demonstration
• They have an opportunity to ask questions
• They get support from family, friends and neighbors
• They receive confirming information from multiple sources
• They receive knowledge and encouragement from trusted sources
• The barriers or obstacles to adoption are removed
• They have an opportunity to commit to a change in front of friends and neighbors
• They receive reinforcement of the behavior

V. Strategies for Developing a Logic Model

Look over these strategies. Each is slightly different and each has been found to be useful. There is no "ideal" way to develop the model. Each strategy represents a different approach. Possibly use several different approaches and compare results.

Strategy 1: The Graphic Mode---Draw a picture of the program
Some people think visually. They see programs in terms of flow diagrams, analogous objects or mechanical entities. Consider these ideas:

Diagramming: Can you use arrows and boxes and construct a flow chart? This diagram might depict those factors that cause people to change and the stages that participants pass through. Sometimes there are choices and feedback loops.

Analogous Objects: Can you represent the program by using an analogy to an organic object? For example, a plant grows because it receives warmth and light from the sun, nutrients from the soil, water and carbon dioxide from the environment. One educator used the water cycle (vapor, liquid, solid) to illustrate that even though the program changed over time it continued to have influence.

Mechanical Analogies: Can you represent the program by comparing it to some man-made mechanical object? For example, is it like a pinball machine with programs shooting out, then disappearing into holes or getting points for repeat effort.

Strategy 2: The If-Then Model
This model is built by using "if-then" logic. If this happens, then this is the consequence. This model is like a series of steps that leads to the ultimate outcome or program consequence. When using the "if-then" logic it is often helpful to think of it from the perspective of individual participants. If this happened to a participant, then what would he/she do?

1. If I received a flyer describing better nutrition, then I would need to read the flyer.
2. If the flyer was written at my reading level and in my language, then I would read it.
3. If I read the flyer, then I would need to believe it and find the statements relevant, credible and practical.
4. If I found the statements relevant, credible and practical then I might talk to friends and neighbors about making a change.
5. If my family, friends and neighbors encouraged me to make changes then I would try the new behavior.
6. If I felt better or got encouragement to continue, then I would stay with the behavior.

Strategy 3: The Inputs-Outputs-Outcomes Model
Another way of depicting a model is by using the input-output-outcomes model. These are:

1. Inputs---the resources invested in terms of dollars or time
2. Outputs---the activities conducted and a description of the participants
3. Short-term Outcomes---The initial results to the participants. These are things that occur immediately or within a few days. Included are knowledge, skills, attitudes, aspiration, awareness, etc.
4. Medium-term Outcomes---These are changes that occur to people (possibly families and communities as well) that happen after the short-term outcomes. It includes behavior changes, changes in practices, decisions made based on the experiences or policy decisions and social actions.
5. Long-term Outcomes---This is the ultimate impact which is the goal of the program. It could be expressed in outcomes to individuals, to families, to communities or even to a society.

Strategy 4: The Theoretical Model
This model uses a theory to explain how the program unfolds. There are many theories that describe behavior and community changes. Among the more popular are the following:

• Theory of Extension Program Development--"Levels of Evidence"
• Theory of Training Levels
• Theory of Behavioral Change
• Theory of Innovation-Decision Process

These models are described below. Don't feel limited by these theories. Over the past 50 years social scientists have developed a number of theories of how change is made, how individuals are persuaded, how community or organizations change, etc. Incidentally, don't expect that any theory will work all the time. The best you can hope for is a theory that explains most of the behavior most of the time.

VI. Models That Describe Change

Over the years social science researchers, educators and public health researchers have developed models of how programs work and how people change. These models are not perfect, but they do offer insight as to how program events or activities might unfold and help program evaluators as they design an evaluation strategy. These models offer clues as to when, where and how to obtain data for an evaluation. Here are four theoretical models:

1. Theory of Extension Program Development--Levels of Evidence
2. Theory of Training Levels
3. Theory of Behavioral Change
4. Theory of Innovation-Decision Process

1. Theory of Extension Program Development---Levels of Evidence

Claude F. Bennett with the U.S. Department of Agriculture suggests that typical programs go through seven steps or phases. Evaluation evidence can be collected at any one of these levels. The evaluator need not use evidence from every level but rather from levels of interest and concern to decision-makers. The "Levels of Evidence" consist of:

Description of Levels

Inputs are the resource expenditures invested by staff and volunteers consisting of time, money, etc.

Program activities are meetings, workshops, and program events.

People involvement refers to the people who participate in the program. These people can be counted and described by demographic characteristics

Reactions are the immediate responses of the participants. Did they like the program? Were they satisfied?

KASA Change stands for Knowledge, Attitude, Skills or Aspiration Change. This is the degree of change that occurs to an individual in terms of gaining new knowledge, changing one's attitude or feelings about the topic, gaining new skills related to the topic, or changing their aspirations about what might be done in the future about this topic.

Practice Change is when an individual actually makes a lasting change in his or her life. This tends to occur after there has been an earlier change of knowledge, attitudes, skills or aspirations.

End Results are the ultimate objectives. It might be social, economic, environmental or individual consequences. These consequences are expected to occur after a certain number of practice changes have been made.

2. Theory of Training Levels

Donald Kirkpatrick has been an expert in organizational training, particularly for profit-based organizations. When evaluating the success or failure of a training effort, Kirkpatrick suggests using the following four levels:

• Level 1. Reaction: Satisfaction levels
  The level of participant satisfaction with the intervention (activities, materials, instructor, facilities, etc.) End-of-meeting forms are often used to obtain this information.
• Level 2. Learning: Content mastery
  This level seeks to answer how well principles, facts and techniques are understood and absorbed by participants.
• Level 3. Behavior: On-the-job behavioral change
  This level addresses the degree to which the individual or team makes changes.
• Level 4. Organizational Impact: Results
  This level seeks to determine the consequences of the intervention or training on the organization. It could involve such factors as turnover, grievances, product defects, customer satisfaction, reducing costs.

3. Theory of Behavioral Change

This model developed by James Prochaska, John Norcross and Carlo DiClemente, has had wide recognition within health care and prevention efforts. The authors depict the five stages of behavioral change that individuals encounter.

Precontemplation. People at the precontemplation stage are resistant to change; they may not even think they need to change.

Contemplation. Contemplators see the need for change and start wishing things were different, yet they still have some resistance to the idea.

Preparation. People at the preparation stage make specific plans to change within the next few months. That might mean setting a date to begin a weight-loss program, take a fitness class or sign up for a smoking cessation course.

Action. The conditions are right. People who are ready to act not only begin their new behavior, but make efforts to support it. They plan how to deal with barriers and remove temptations. Someone who wants to lose weight and develop low-fat eating habits, for example, should stock up on foods low in fat and include fruits and vegetables as healthy snacks.

Maintenance. Continuing the change takes effort. You need vigilance. Social pressures, personal problems and other stresses can sabotage even long-term success.

Relapses are inevitable at every stage in this process. Still, you can help minimize them with close attention to your feelings and surroundings.

4. Theory of Innovation-Decision Process

Everett Rogers has been a pioneer in helping understand how individuals make decisions about adopting innovative practices. Much of his research was based on how Midwest farmers adopted new agricultural practices. Later, his work was found to apply to a variety of non-agricultural environments as well.

The innovation-decision process is the means by which an individual (or other decision-making unit) passes from first having knowledge of an innovation, to forming an attitude toward the innovation, to a decision to adopt or reject, to implementation of the new idea, and to confirmation of the decision.

The decision to adopt or reject is influenced by five factors, including:

1. Relative advantage---What are the rewards or punishment of adoption?
2. Compatibility---Is it consistent with my existing values, past experiences and needs?
3. Complexity---Is it difficult to understand and use?
4. Trialability---Can I try it on a limited basis?
5. Observability---Can I see it?

VII. For more information

If you would like more information on this topic, consider the following:

Bennett, Claude F. (1977) "Analyzing Impacts of Extension Programs" Washington, DC: Cooperative Extension Service, USDA.

Kirkpatrick, Donald (1998) Evaluating Training Programs: The Four Levels Benett-Koehler Publishers.

Logic Model: University of Wisconsin-Extension and Logic Model: Program Performance Framework. (2000). www.uwex.edu

Measuring Program Outcomes: A Practical Approach. "Step 2: Choose the Outcome You Want to Measure" (1996) United Way of America

Prochaska, James, et al (1994) Changing for Good. New York: Avon.

Rogers, Everett M. (1983) Diffusion of Innovations. New York: The Free Press.

University of Nebraska. Targeting Outcomes of Programs http://deal.unl.edu/TOP/

University of Ottawa. What is a Program Logic Model? http://www.uottawa.ca/academic/med/epid/what.htm

W.K. Kellogg Foundation Handbook . (2000) "Exploring the Three Components of Project-Level Evaluation." Chapter 4. "Measuring Impacts Through the Use of a Program Logic Model" www.wkkf.org

Back to EVALUATION