Office of Juvenile Justice and Delinquency Prevention. Evaluating Juvenile Justice
Programs: A
Design Monograph for State Planners. Washington, DC: Prepared for the U.S. Department of
Justice, Office of Juvenile Justice and Delinquency Prevention by Community Research
Associates, Inc.; 1989. pp. 22-23.
Varieties of Outcome Measures
In reality, outcome is not an "end process" issue; that is, outcome evaluations need not, and probably should not, be concerned only with what occurs at the end point of a particular process. All program activities have outcomes, whether intended or not. Some are quantifiable and some are not. Some are easily observable and comprehensible and some are not. Additionally, all different types of programs have outcomes, and these same issues apply as much to legislative initiatives as they do to direct service provision programs. In this section, we review types of outcomes to dramatize this important point. It is important to think of outcome issues in a more comprehensive context. This will enhance evaluation activities generally, and improve the information that goes to policymakers.
Knowledge Production Outcomes
In many instances, for particular programs or for program plans in general, a major goal is the production of knowledge about juvenile justice issues such as juvenile crime, effective prevention and treatment strategies, current legislation, available services, and so on. Evaluation of knowledge production efforts, or identification of knowledge production outcomes, generally receive low priority from evaluators. The frequent assumption is that more instrumental outcome measures-criminal behavior, test scores, and the like-are more desirable. This is not necessarily an appropriate assumption. Knowledge production is a stated goal in the federal Juvenile Justice Delinquency Prevention law. Acceptance of programs at state and local levels depends on the availability of information about the programs and acceptance of them, goals that cannot be achieved without the production of new knowledge in the general and criminal justice communities.
There are ways to monitor and evaluate knowledge production, and they are addressed later in this monograph. The important point here is that knowledge production be considered a valid process and outcome, worthy of evaluation from the start.
Consensus Building Outcomes
A similar argument applies in the case of consensus building outcomes-the need for production of common understanding and efforts among various constituents in the juvenile justice arena, especially where they may not have existed before. Attaining instrumental goals (see below) often depends on significant consensus building around an issue. Jail removal and de-institutionalization are two primary examples. Such programs cannot guarantee success even if they are well-conceived, well-managed, and adequately funded; they also must enjoy the support of various components of the state and local criminal justice and general communities.
Evaluation efforts tend to ignore this consideration. Consensus building efforts and achievements do not lend themselves to measurement and scientific analysis. However, they provide significant and valid qualitative, or contextual, information, which are worthy of more serious consideration in evaluation efforts. Production of information about the political and social psychological aspects of program implementation would be invaluable.
Instrumental Outcomes
These are the most commonly discussed outcomes in evaluation research. Being directly or indirectly related to a funded program's goals and objectives they, if observed and measured properly, will indicate the program's level of success or effectiveness. Typical areas of measurement include recidivism, educational attainment, self-esteem, and community values or citizenship. These are usually given the highest priority by evaluators and decisionmakers, and for good reason. If a program is not producing the promised results, and evaluation confirms this, then it is time to reconsider program goals, objectives, and methods. Instrumental outcomes are important. They are even more valuable if they are presented with evaluation information about knowledge and consensus building; where appropriate, to give decisionmakers the maximum amount of useful information, in the proper doses.