Harrel, A.  (n.d.). Evaluation Strategies for Human Services Programs.  Washington DC: The Urban Institute.

View entire document

Developing a Logic Model
It is impossible to interpret evaluation findings without a clear understanding of program goals, implementation sequences, and the expected links between them and expected program benefits. Expectations about these linkages are made explicit by developing a logic model. Such a model is developed by discussing with service providers and funders the goals of and rationales behind program organization and content, examining planning documents and program reports, and reviewing research findings on similar programs or problems. The literature review may be particularly helpful in identifying plausible causal links and any factors other than the program which should be considered in the evaluation.

The logic model provides a simplified description of the program, the intended outputs, and the intended outcomes. Program characteristics include the population to be reached, the resources to be used, and identification of the types and levels of service elements. Outputs are immediate program products resulting from the internal operations of the program, such as the delivery of planned services. Examples of output indicators in the area of programs for vulnerable children and youth might include the numbers of children immunized, home visits by case managers, or youth completing a job training program. These program outputs are, in turn, the vehicle for producing the desired program outcomes, for example, decreases in childhood illnesses, decreases in abuse and neglect cases, or increases in youth employment. Careful attention must be paid to when the anticipated outcome should be expected to occur. For this reason it is often useful to divide outcomes into intermediate versus longer term. For example, improved school attendance in early grades might be an intermediate outcome associated with the longerterm outcome of dropout prevention. Care must be given to focusing on outcomes which will occur within the study period.

A classic failure in selecting an outcome that is expected to occur within the time frame of the study occurred in evaluations of the DARE drug prevention program, an educational program for fifth and sixth graders designed to prevent drug use. Evaluation results showed no significant prevention of drug use at the end of the program. This result should have been anticipated, since drug use does not typically begin among youth in this country until the mid-teen years (14 to 17). An age-appropriate intermediate outcome should have been selected as the primary outcome measure, such as improved peer resistance skills and changes in beliefs about the risks of drug use.

The logic model should also include explicit mapping of the conditions present in the program environment or characteristics of the target group or community that may affect the program's ability to achieve its goals. Non-program characteristics of the program organization, community or target population that are likely to influence the outputs and outcomes and/or use of program services are called antecedent variables. Conditions or events in the program, target population, or community that may limit or expand the extent to which program outputs actually produce the desired outcomes are called mediating variables. For example, a drug abuse prevention program may be less effective if the program staff are inexperienced, or if the local community offers fewer recreational alternatives to substance abuse and/or more active open drug markets (antecedent variables). Offering other support services in combination with the program may enhance its impact (a mediating variable).

In impact evaluations the logic model is used to spell out how, and for whom, certain services are expected to create specific changes/benefits. For example, if the program includes parenting classes, the logic model will identify this activity as a key program component and show the types of changes in parenting that will be used to measure program outcomes (e.g., by improving parental assistance with homework or helping parents communicate more effectively with adolescents).

In performance monitoring, the logic model is used to focus on which kinds of output and outcome indicators are appropriate for specific target populations, communities, or time periods. For example, among indicators of child improvement in school, one might expect attendance to improve in the first semester of a program, but academic test score improvement only after a significant period of program participation-with the timing possibly varying by the age and developmental stage of the children.

In process evaluation, the logic model is used to identify expectations about how the program should work-an "ideal type"-which can then be used to assess the deviations in practice, why these deviations have occurred, and how the deviations may affect program outputs. This assists program managers (and evaluators) to identify differences (including positive and negative unintended consequences), consider possible mechanisms for fine-tuning program operations to align the actual program with the planned approach, or re-visit program strategies to consider alternatives.(2)

Logic models are constructed to show temporal sequences, building left to right, and they typically diagram relationships with arrows. An example of a logic model is shown in Exhibit A. It was developed by the Urban Institute during the planning of the evaluation of the Children At Risk program (CAR). CAR is an intensive intervention program designed to prevent involvement in drugs and crime, and to foster healthy development among adolescents ages 13 to 15 who exhibit serious risk indicators and live in severely distressed inner-city neighborhoods.

Exhibit A
Logic Model Used in Evaluation of the Children At Risk Program