Logic Models for Program Evaluation: Purpose and Parts

CanMedEd-ipedia: The CORAL Collection. Concepts as Online Resources for Accelerated Learning.

Introduction

A sound and realistic logic model can guide us in developing and implementing program evaluations that produce relevant, evidence-based information about a program’s effectiveness.

Learning Objectives (what you can reasonably expect to learn in the next 15 minutes):

  1. Describe the main purposes of a logic model.
  2. Distinguish among four main components typically included in logic models.

To what extent are you now able to meet the above objectives? Please record your self-assessments. (0 is not at all and 5 is completely)

Describe the main purposes of a logic model.
Distinguish among four main components typically included in logic models.

To get you warmed up … based on your prior knowledge and/or experience, match the terms (1. Inputs 2. Activities 3. Outputs 4. Outcomes) to an example of program evaluation components:

Terms: 1. Inputs 2. Activities 3. Outputs 4. Outcomes

Examples:

a. Number of simulated or volunteer patient encounters delivered in the clinical learning resource centre each year

b. Provide librarian assistance with search strategies to students in all four years.

c. Increase medical student mental health to levels equivalent with those of age-matched peers.

d. Number of half-day workshops organized and delivered by faculty development programs each year.

e. Physicians in the region who are engaged in the medical school in some capacity.

Don’t worry if you did not get them all right. You are just starting out! There’s lots to learn. Now read on to get the scoop on components: Inputs, Activities, Outputs, and Outcomes.

Logic Models for Program Evaluation: Purpose and Parts

Purpose: A program logic model is a visual representation of how we anticipate that the resources invested in a program will lead to the results we intend to achieve.

Logic models illustrate our logical assumptions about how the resources invested in a program will be used to carry out activities and produce products or services that will contribute to achieving a chain of expected results (e.g., “If we deliver x then y will occur, which should then result in z”). These “if-then” assumptions, as depicted in a logic model, represent the basic components of a program’s theory of change – that it, our theory about how a program will effectively lead to a desired change.

By illustrating our expectations about how a program will be effective, logic models allow us to:

  • Verify that our expectations for the program’s effectiveness are sound and realistic.
  • Ensure that there is clarity and consensus among those who fund and deliver the program about the resources available to deliver the program, the key program activities and what they produce, and the program’s expected outcomes.
  • Communicate information about our program to internal and external stakeholders in a reader-friendly, visual manner.
  • Select relevant performance measures, aligned with our expectations of the program as depicted in the logic model.
  • Interpret program data and their implications regarding the extent of program effectiveness.

Components: Logic models come in various formats: some contain more components than others and the labels used to describe the components vary, as does the direction in which the logical model is presented (e.g., left-to-right, top-down).

Keeping this flexibility in mind, logic models typically depict the relationships between the following program components in a logical or reasonable sequence:

  • Inputs: The resources invested/available to deliver the program.
  • Activities: The key activities/actions carried out, using the inputs.
  • Outputs: The products and services that are delivered as a result of the program activities.
  • Outcomes: The changes that are expected to occur over time, in the short-, medium-, and long-term, as a consequence of the program’s activities and outputs.

Outcomes are often grouped according to the order in which we expect the results of the outputs to occur.

    • Immediate outcomes result directly from the delivery of outputs.
    • Intermediate outcomes are the changes anticipated after immediate outcomes are achieved.
    • Ultimate outcomes are the furthest reaching changes that can reasonably be attributed to the program after the immediate and intermediate outcomes have been achieved.

Logic Model Components

Logic Model Components Example 1: Simulation Lab Training Program Example 2: Student Recruitment Diversity Program
Inputs
Resources invested
money, human resources, facilities, and equipment money and human resources
Activities
Actions carried out
Revise and deliver training to students
Develop and administer assessment tools
Develop and implement student recruitment strategy for under-represented demographic groups
Outputs
Products and services delivered
Training delivered to the targeted students
Assessments conducted
Promotional materials and information sessions delivered to under-represented demographic groups
Outcomes
Expected changes
Immediate
Direct result of outputs
Increase students’ engagement / knowledge / skills Increase awareness of program among under-represented demographic groups
Intermediate
Changes anticipated after achievement of immediate outcomes
Improve quality of students’ work in applying what they have learned Increase number of applicants from under-represented demographic groups
Ultimate
Furthest reaching changes attributable to the program
Enhance quality of patient care provided by students in clinical learning environment Increase number of enrolled students from under-represented demographic groups

Check for Understanding

1. Which of the following statements express the purpose of logic models, as presented in this CORAL cell? (True/False?) 

1. Program stakeholders can gain a better understanding of the program from its logic model.


2. Logic models can help us identify which types of information are most relevant for evaluating a program’s effectiveness.


3. Unrealistic expectations about how program outcomes will be achieved can be flagged by developing a logic model.


4. Once a logic model has been developed, the evaluation of a program is complete.


5. Logic models are a tool for evaluating the efficiency with which a program is delivered.


6. When there is evidence that a program is not achieving its expected outcomes, a logic model can help us interpret this information and understand why.


Which of the following statements about the components of a logic model are True/False?

1. The expected outcomes must first be achieved before the program’s products and services can be delivered.


2. Activities are the actions taken to produce products and services.


3. The components of a logic model illustrate our expectations regarding how the invested resources (inputs) are used (activities) to produce products and services (outputs) that result in intended changes (outcomes).


4. (BONUS) Immediate outcomes are more directly within the control of the program than are intermediate and ultimate outcomes.

Self-assessment

To what extent are you NOW able to meet the following objectives? (1 is not at all and 5 is completely)

Objectives
Describe two main purposes of a logic model
Distinguish among four main components typically included in logic models.

To what extent WERE you able just before beginning this Cell to meet the following objectives? (1 is not at all and 5 is completely)

Objectives
Describe two main purposes of a logic model.
Distinguish among four main components typically included in logic models.
Concept map of related CORAL cells: Program Evaluation
Concept map of related CORAL cells: Program Evaluation

Thank you for completing this CORAL Cell. We are interested in improving this and other cells and would like to use your answers (anonymously of course) along with the following descriptive questions as part of our evaluation data.

Provide feedback on module

Thanks again, and come back soon!

The CORAL Cell Team

Credits:

Author: Vanessa Anastasopoulos, McGill University
Series Editor: Marcel D’Eon, University of Saskatchewan
Reviewer/consultant: Derek Wilson, University of British Colombia

Back to the CORAL collection