Planning to Evaluate a Community Justice Initiative? What Are Some Challenges?

Using control groups or comparison groups can be problematic when evaluating community-wide initiatives.

When interventions are implemented that affect the entire community, random assignment of cases is usually not possible. In this case, neighboring localities can be used for comparison purposes. If the initiative involves only part of the community, like one particular court or one police district, then other courts or districts can be used for comparison purposes. Regardless of whether the comparison is a neighboring locality or another court or district, the evaluator must be concerned about group differences: that is, differences that exist between localities, other than the initiative, that might account for any observed differences in outcomes. Randomized control trials are more easily implemented when the initiative being evaluated is an intact entity, such as a specialized court.

Program implementation issues may affect outcome evaluations of community justice initiatives.

Successful community justice programs must incorporate a degree of flexibility in order to accommodate changing conditions in the community as well as the feedback of clients and community members. This means that the nature of the services being provided may change as the evaluation proceeds. Evaluators should allow community initiatives to become established and stabilized before undertaking outcome evaluations. Process evaluations and evaluability assessments are also important elements of an evaluation plan for a community justice initiative.

Evaluations should include "quality of life" and other measures of the health of the community.

Many community justice initiatives have as goals improvements in the health and well-being of community residents via activities designed to make justice more visible to the community and to increase collaboration among local residents, businesses, and community organizations. So, it is important for evaluations to measure what impacts the initiative had on the community. Evaluators should consider qualitative methods such as observations of the community, and interviews and focus groups with community leaders, residents, and offenders in order to assess perceptions of change in these quality of life issues.

| | |
| | |