U.S. Department of Housing and Urban Development. Program Evaluation and Analysis: A
Technical Guide for State and Local Governments. Washington, DC: Prepared for the U.S.
Department of Housing and Urban Development by Public Technology, Inc.; 1978.   pp.12-15.

View entire document

TASK 4- ESTABLISHlNG GOALS OBJECTIVES AND EVALUATION CRITERIA
With the project team selected, the next major step is to investigate the program's goals, objectives, and evaluation criteria. A program goal is a broad statement of intended accomplishments or a description of a general condition deemed desirable. Goal setting should be primarily the responsibility of elected officials and public administrators. A program objective is a specific, well defined and measurable condition that must be attained in order to accomplish a stated pal. Objective setting should be primarily the responsibility of public administrator and their staffs. or operational personnel. Evaluation criteria are the actual instruments used to measure progress toward objectives. Evaluation criteria are normally formulated by project personnel.

Ideally, all government programs should have goals and objectives explicitly stated as part of the program planning process. In reality, very few government programs have explicit, meaningful goal and objective statements. This poses a problem in program evaluation, since there is no clearly stated direction with which actual performance can be compared. Often Often and objectives are scattered throughout program documentation. In such cases, the analyst must identify goal and objective statements and phrase them clearly.

Many times, the program documentation contains much of the information to compose goals and objectives. In such instances, the evaluator is best advised to assist program personnel, management, and elected officials to establish meaningful goals and objectives to be used in future evaluations and to guide program activities. In the latter case, evaluative effort should not be as rigorous and critical as in cases where goals and objectives clearly exist and are recognized by the program staff.

Since program analysis concerns future program activities, it is always appropriate to establish goals and objectives for a new program and redefine them for existing programs. It will usually be necessary to establish evaluation criteria for all programs, even those that already have effective goals and objectives.

The above discussion should make it obvious that the analyst's role in goal and objective setting will not be the same in every project. The analyst may need to do nothing more than make sure that the set of clearly stated goals and objectives is the most current available or he may need to participate in a full-blown goal- and objective-setting process. In most cases, he will assemble and restate goals and objectives drawn from program documentation and other sources. The steps outlined in this task are designed to provide some guidance to the analyst for each of the cases discusses above. The four major steps are (1) Review program material, (2) define program goals, (3) define objectives, and (4) establish performance criteria.

Step 1-Reviewing Program Material
The first task for the analyst is to review source material relating to the program and to get a general idea of the overall purpose behind the program. Some suggested sources for leads to program goals ant objectives are:

Budget document-The program agency's annual budget request and justification will often have statements of program goals and objectives. Such statements may appear under other names, such as program purpose or program scope.

Program personnel-Perhaps the most important source is the program agency personnel themselves. Their knowledge of program operations and history as well as access to records containing policy memoranda etc., make them the prime source. This activity should be the first concern of the analyst during the agency orientation task of either evaluation or analysis.

Enabling legislation-Many jurisdictions include an indication in their budgets as to the legal basis for tine program. Examination of the charter or applicable statutes will frequently give insights as to the intent and scope of the program.

Policy messages of elected officials-Such documents as"state of-the-city" messages frequently provide insight into what elected officials perceive to be the functions of various key programs.

Expressions made by legislators, citizen groups or individual citizens at hearings before a local council or in the press-Testimony before committees considering a bill to create, expand, abolish, or evaluate a program may contain useful discussions of both explicit and implied objectives.

Minutes of boards and commissions-Many local government programs have some policy-wide oversight body to give overall direction. The minutes or annual reports of such bodies will frequently provide insights into the goals of the program

Study of these sources may revert clear goal and objective statements that fit the characteristics listed in Steps 2 and 3 in which case the analyst should proceed to develop evaluation criteria as outlined in Step 4.

Step 2- Defining Program Goals
A goal statement should describe in general terms something to be accomplished. A goal statement should be written with several factors in mind:

A goal covers long time spans relative to objectives:
A goal can be either intangible or tangible:
A goal should be people- or community oriented:
A goal should not predetermine the details of program activities;
A goal should reflect the direction daired by the general public, elected officials, and public administrators- not staff personnel: and
A goal should be expressed as a desired outcome or condition to be achieved rather than as an action or process.

Examples of program goal statements include:
Traffic Engineering-Safe, efficient, and convenient movement of people and goods.
Fire Department-The highest level of public physical safety with the resources available.
Economic Development-Economic opportunities for persons who have not enjoyed economic equality.

Many programs have several related activities, each of which may have one or more subgoals. For example, a fire department will usually have separate organizational activities for fire suppression, training, fire prevention, ambulance services, and administrative support. Subgoals for these activities might be:

Fire Suppression-Rapid suppression of fire.
Fire Prevention-Reduction in incidence of fires.
Training-More effective and efficient fire department personnel.

As the examples show the subgoals support the general program goal and address a segment of the program mission.

If the analyst cannot find or derive goal statements such as those listed above, it will be necessary to establish goals from scratch. Ideally, goal setting should be directed by the chief administrative officer of a jurisdiction with the direct input of elected officials. As a matter of practicality the analyst may find it more efficient to draft goal Statements in conjunction with the agency head and present these draft goals to the chief administrator and elected officials to stimulate discussion.

The chief administrator and elected officials can be expected to take a greater interest in the goal-setting process as they begin to grasp the importance of goals and objectives in the management of government programs. This means that the goal-setting discussions may be relatively brief for the first several programs studied but may increase in length and intensity for subsequent programs. The analyst should keep this factor in mind, as it can affect the length of time necessary to conduct a study ant therefore should influence workplan preparation.

Step 3-Defining Program Objectives
Once consensus has been reached on the more general goal statement, the analyst's next job is to review, redefine, or define specific and measurable objectives. As a matter of practicality, much of the groundwork for the formulation of objectives will have been done during the development of goal statements. The analyst may even wish to develop the goals and objectives at the same time and to present both to the agency head, top management, and elected officials through the procedure presented above. This consolidated effort will work best when there appears to the analyst to be little question or disagreement on the goal statements as drafted. However, when the program goals seem to be controversial, the analyst should make sure that the goals are agreed upon before attempting to develop objectives.

An objective should describe something to be accomplished in specific, well-defined and measurable terms. Objectives are derived from goals by, first, formulating a strategy for reaching the goal and, second, establishing one or more objectives necessary to make this strategy work. In the case of the fire department example, the subgoals represent an expression of the chosen strategy. That is, in order to achieve the overall program goal of maintaining public physical safety, the strategic elements of fire prevention, suppression, training, and medical assistance are necessary. Specific objectives are then developed for each subgoal.

An objective should be written with these factors in mind:

An objective is something that must be accomplished in order to achieve a goal:
An objective is not a program or project function, activity, task, or step:
An objective should not predetermine in any fashion the solution to a problem or way to do something:
An objective should relate to the needs of groups of citizens or the community as a whole:
An objective should explicitly consider unintended or negative effects:
An objective should be achievable within a specific time frame: and
An objective should be expressed as a desired outcome or condition to be achieved rather than as an action or process.

Figure 5 presents Several examples of effective objectives that follow the above guidelines. If effective objectives cannot be found in or derived from program documentation the analyst will have to develop them from scratch.

In developing objectives, the analyst should take into consideration the effects the objectives have on various population or clientele groups. Different groups may be affected by a program in different degrees. It is important to identify such groups and to collect data reflecting program impacts on them. An "average" crime rate or "average" family income for a jurisdiction will not adequately reflect possible major differences that may exist among segments of the population. The following points should be considered:

Each program will have some groups that are intended beneficiaries; i.e., clients of the service.
Each program is likely to significantly affect certain other groups that are not intended beneficiaries. These effects may be detrimental or beneficial.
The citizens of the community or state considered as a whole often make up a category that should be explicitly identified.
In some cases, future citizens may be an important group to consider explicitly because their interests are closely related to the program.

Figure 5. EFFECTIVE OBJECTIVES. Below are examples of program objectives determined according to me guidelines presented in Step 3.

The analyst will find that the preparation of a clientele group profile will help to develop objectives that are people-oriented by creating a picture of the group that is the target for the program. Figure 6 contains a suggested list of characteristics for inclusion in such a profile. Most of this information can be obtained from census data. Each program is likely to have at least some unique clientele groupings.

It is important that the program objectives be developed in close cooperation with program personnel, especially for programs of long standing, because the analyst is developing the standards against which programs will be measured and it is only fair that everyone agree on the essentials at the outset. Also, should an analyst attempt to develop objectives from the other cited sources alone, it is entirely possible that the analyst might develop a set of obsolete objectives.

The objectives of a program frequently shift with the passage of time; the longer a program has been in operation, the greater the chances that such a change has occurred. The objectives used should be those that the program agency personnel agree are current.

The analyst should get most of the information needed to formulate objectives by interviewing program agency personnel. In addition to the agency head and appropriate division directors, the analyst should also interview first-line supervisors and program workers to learn their perspective and to find out whether they are familiar with existing objectives. While analysts should develop their own specific questions for the interviews, Figure 7 lists some suggested questions that can form the basis for an effective interview.

Figure 6. CLIENTELE GROUP CLASSIFICATION. The analyst should know what population or clientele are affected by program goals and objectives. This classification scheme offers some assistance in developing profiles on population or clientele groups.

The analyst should always have specific questions composed in advance for these interviews. This helps to assure that all necessary information is obtained and to avoid wasting the time of program personnel with inefficient often offensive "fishing expeditions." The analyst should, of course, be prepared to diverge along a promising line of inquiry that emerges during the interview.

The analyst's list of program objectives should be presented to the agency head for discussion and approval before being transmitted to top management. While some jurisdictions may wish to do so, it isn't necessary to have objectives approved by elected officials. Many public administrators feel that overall guidance by elected officials in the form of goal statements is an adequate level of involvement.

An implicit assumption in program evaluation is that the objectives are practical. If objectives are too easy to attain they offer no real incentive for the program staff to strive for greater ach~ievement. It is probably best to set objectives that make program personnel reach a bit. On the other hand, care must be taken not to set objectives that are too ambitious lest employees become frustrated by unreasonable performance targets and cease trying their best.

Analysis of an ongoing program can raise some special problems in establishing objectives. Since the objectives of most programs shift over time, the analyst must be careful not to accept "prepackaged" objectives set down when the program started without some investigation of their relevance. Since analysis is intended to shape the future conduct of program activities, objectives that describe past practices may hamper a thorough search for alternatives. The analyst should make sure that the objectives, criteria, and clientele groups are what local policy makers intend them to be for future program operations.

Figure 7. SUGGESTED INTERVIEW QUESTIONS. These questions should help the analyst to gather information about a program's objectives during interviews with agency personnel.