Chapter 4: Conducting Evaluations In-house or Under Contract
Developing an evaluation capability in the state office may or may not be productive, depending in part on the level of resources that are devoted to evaluations and the kind of expertise that is required. If a central evaluation or research capability already exists in state government, adding staff to evaluate the projects supported by the Drug Control and System Improvement Grant Program may be accomplished relatively easily. If no such central evaluation capability currently exists, creating one has numerous fiscal implications that the state administrator will recognize. (Will, for example, the amount of work to be demanded of evaluations be sufficient to build an entirely new staff with supporting equipment?)
A decision to hire evaluators or to contract for their services should be governed by a desire to maximize several values: (1) the technical skills of the evaluators; (2) the evaluator's familiarity with the details of the criminal justice system, including sensitivity to the political/bureaucratic tensions that prevail; (3) the disinterestedness of the evaluator; and (4) the utility of the evaluation for the decision makers.
Whether the state can attract staff with the experience and training needed for evaluations, especially evaluations of a project's impact, depends in large part on local market conditions. In many parts of the country, it may be difficult to hire persons with sufficient evaluation experience or training. In these instances, contracting with professionals at a university or research organization may be desirable.
In many instances, hiring evaluators into the state office may be the most efficient way to develop expertise that is tuned to the idiosyncracies of the local justice system agencies. In-house evaluators are also more likely, because of proximity and on-going working relationships, to develop close communication with the consumers of evaluation information, thereby producing evaluations that are suited well to the decision maker's needs. These close ties may, however, undercut the evaluator's ability to maintain a disinterested stance vis-a-vis the projects being assessed. How well this tension is balanced depends in large part on the state administrator's willingness to receive objective reports about projects' performance, including reports that find projects failing to accomplish their mission.