Comparisons are made between the ten pilot and ten comparison districts using data from cases terminated in 1991 before CJRA and separately using data from cases filed in 1992-93 after implementation of the pilot program plans. Because of differences between our pre-CJRA and post-CJRA data that are unrelated to CJRA and that are difficult to properly account for, we focus on separate pre- and post-CJRA analyses. The results of our qualitative analysis, combined with our separate pre- and post-CJRA quantitative analyses, provide ample evidence concerning the effects of the act.
The evaluation also uses quantitative analyses to compare cases managed in different ways to determine how such management practices affect litigants' costs, time to disposition, participants' satisfaction with the process, and views of fairness. The quantitative analyses exploit natural variation in judges' management practices, rather than an experimental random assignment of management practices to cases.
We used CJRA advisory group reports, documents, and meeting minutes to assess the advisory group process and findings; we used the districts' plans and proposed local rule changes to assess what the district said it would do under CJRA; we used the dockets for a large sample of cases to help us understand what was actually done on cases and when (such as schedule setting, assignment to management tracks, or referral to ADR); we used court records to assess the basic characteristics of the cases and court actions, such as referral to ADR, that were not always on the court docket; we used the judicial surveys on our sample of cases to get judges' views on whether they had changed how they manage cases as a result of CJRA; we used extensive mail surveys of thousands of lawyers and litigants on our sample of cases to get their views on how the case was managed and information on litigation costs, satisfaction, and views of fairness; and we used extensive semi-structured interviews with judges, court staff, advisory group members, and lawyers to better understand both the implementation of CJRA and case management in the districts before and after CJRA.
In total, more than 10,000 cases were selected for intensive study, and more than 60,000 people were to be surveyed. We received completed survey responses from judges on 3,280 cases (about two-thirds of those closed in our post-CJRA sample), from about 9,000 lawyers (about one-half of the lawyers surveyed), and from about 5,000 litigants (about one- eighth of the litigants we attempted to survey). Because of the low litigant response rate, we were limited in our ability to analyze litigants' hours spent, satisfaction, and views of fairness.
We analyze time to disposition, rather than delay, since the latter cannot be defined without reference to some currently unavailable standard of how long civil cases should take to resolve.
We present information on two types of cost: those borne by the litigants, measured in both monetary and work hour terms, and those borne by the federal court system, measured in terms of judicial work hours. Our full report provides data on monetary costs to litigants, litigant hours spent, and lawyer work hours spent. However, we consider lawyer work hours to be the best available measure of how case management affects litigation costs because it has uniform meaning regardless of attorney fee structure[1] or geographic variations in attorney fee rates and can be used consistently for both in-house lawyers and outside counsel. Consequently, in the statistical analyses we use lawyer work hours as our measure of costs.
Our assessment of satisfaction and views of fairness is drawn from the results of our surveys.[2]
Given the observational nature of our data, one should not treat our statistical results as exact estimates of causal effect. Rather, our statistical analyses summarize the differences observed in our sample of cases. We have made every attempt to ensure that our estimates clearly represent effects in our observed data, but since the pilot program did not randomly assign case management procedures to cases using an experimental design, we cannot say definitively that our observed effects correspond to causal effects among the studied cases and districts. Thus, interpretation of our statistical results should take place only in the context of an understanding of how the judicial system functions in practice.
We base our assessment of case management policies and procedures on data from general civil litigation cases[3 ] with issue joined.[4] We also analyze the subset of these cases that took longer than nine months to disposition.
One issue that has been raised regarding the CJRA concerns the appropriateness and effectiveness of national uniform standardized rules and procedures. Some people see CJRA as a "top down" reform started by Congress. Others see CJRA with its local advisory groups and local rule revisions as an attempt to tailor management to the local legal needs and culture. Our research design did not address the debate over national versus local rules and procedures. Instead, we analyze and report what happened as a result of CJRA and the application of management principles and techniques identified in the act; we leave it to others to draw conclusions on the issue of uniformity of rules and procedures.
[2] Satisfaction and views of fairness were measured by responses to the following questions: How satisfied were you with the court management and procedures for this case for your party or parties? How fair do you think the court management and procedures were for this case for your party or parties?
[3] In practice, federal district courts split the civil caseload into two categories--those types of cases that usually receive minimal or no management, and those general civil litigation cases to which the district's standard case management policies and procedures apply (and which are of primary concern for evaluation of CJRA case management principles and techniques). Minimal management is usually applied to prisoner cases (other than death penalty cases), administrative reviews of Social Security cases, bankruptcy appeals, foreclosure, forfeiture and penalty, and debt recovery cases.
[4] Issue is considered joined after the defendants have answered the complaint in accordance with F.R.Civ.P. Rule 12(a) or as mandated otherwise by the court (Administrative Office of the United States Courts, Guide to Judiciary Policies and Procedures, Volume XI, Statistics Manual, Chapter 5, p. 15, updated as of 1995).