What Have We Learned From Evaluations of Information Sharing/Integration Initiatives?

Most studies of information sharing/integration initiatives have either been usability evaluations or case studies. Usability evaluations assess the degree to which users can operate the data system, the efficiency of the data system, and/or satisfaction with the data system by users. Usability evaluations are similar to process evaluations. To date, most work to collect information on implementation and outcomes has used feedback surveys, focus groups, interviews, and direct observations. Few outcome evaluations of information sharing/integration initiatives have been conducted that would provide information concerning the impact of the new technology. The relative newness of work in this area likely explains the focus on implementation and usability.

Depending on the parties involved in the information sharing/integration initiatives, there may be different challenges that affect the implementation and accomplishments of the initiative. Initiatives that include parties from different parts of the criminal justice system within one jurisdiction may face different issues than those that include multiple jurisdictions or those that include parties from the same part of the criminal justice system (e.g., all law enforcement agencies) from multiple jurisdictions.

Several factors are associated with the implementation success of information sharing/integration initiatives:

Encouraging top-level commitment to the initiative
Promoting active participation of stakeholders/users in the initiative
Properly training stakeholders/users on the initiative
Developing a comprehensive planning process for the initiative
Modeling and testing current processes to determine the type of initiative needed and adjustments required once the system has been implemented
Clearly defining performance measures at the start of the project
Staying informed about other information sharing/ integration initiatives that may affect the proposed initiative

Results from usability evaluations show that the technology used in new information sharing/integration initiatives typically results in more accurate and complete data than in the old approach. For example, accuracy is demonstrated by the ability of new systems to detect inconsistently entered data. More complete data exists, for example, with the use of real-time data rather than data that are days or weeks old. Evaluation results show success in merging and translating data stored in the old record systems to the new systems. Preliminary results show increased productivity for users of new systems as a result of reducing the amount of time needed to search for data.

Case studies and anecdotal information suggest some positive outcomes resulting from the implementation of new information sharing/integration initiatives. For example, it has been noted that improved criminal history data reporting has contributed to increased public safety. In addition, the adoption of standard definitions for terms and database fields has increased both the reliability and validity of data, and the ability to more accurately measure justice system policy changes.

| | |
| | |