Summary

The United States spends considerable sums on international cooperation in research and development (ICRD). In fiscal year 1995, for example, the U.S. government spent more than $3.3 billion, which is more than 4 percent of the annual federal research and development budget. The U.S. government maintains 26 bilateral "umbrella" agreements and hundreds of agency-to-agency agreements to support and encourage international cooperation in science and technology. Policymakers have expressed concerns about this cooperative research. Some fear that the United States is paying more than its fair share of the work's cost. Others worry that the country is giving away critical know-how to potential foreign competitors. Additional concerns have been voiced that cooperative programs subordinate the interests of true science to strategic or political ends. These claims are difficult to test for a number of reasons: the large number, varying goals, and long timelines of projects; the underdeveloped nature of tools for measurement; and the lack of a system for linking international science and technology agreements with actual spending on cooperative R&D.

CTI set out to determine if the benefits of ICRD could be measured in real-time in a way that does not require explicit reporting by individual projects. After an extensive database search complemented by agency interviews, we identified some 3,000 projects to use as a basis for an approach to quantifying benefits. Our analysis led us to conclude that the key to identifying benefits lay in understanding the relationship between the purpose of the project and the type of project undertaken. Knowing the rationale behind each type project enables us to predict what kinds of benefits are likely to accrue, which in turn enables us to identify applicable metrics. After constructing an assessment framework and identifying metrics, we applied the framework to a case study--earthquake sciences and seismology--to test its applicability.

Purposes of ICRD

We identified four broad reasons for government to fund an ICRD project:

• If the scale of the equipment or investment required to conduct the project is large.
• If the nature of the subject is global.
• If unique expertise or natural resources involved have a remote location.
• If the mission of the agency involved is to support international cooperation.

We found that the majority of ICRD projects are conducted for the first two reasons: because large-scale investments are required, thus making cost-sharing arrangements with other countries desirable, and because the global nature of the subject lends itself to international cooperation. We identified eight specific activities involved in these types of projects: collaborative research, conferences, contracts (if not classified), database development, operational support, standards development, technology transfer, and technical support. The overwhelming majority of U.S. spending supports the first of these activities, collaborative research (see Figure S.1).

Figure S.1--U.S. International Cooperative Research and Development Spending by Nature of Activity

Identifying ICRD Spending

The U.S. government committed at least $3.3 billion in federal R&D spending to projects involving some form of international cooperation in fiscal year 1995. In addition, federal governments agencies spent as much as $1.5 billion in other activities that were not tagged as R&D funds but that constitute scientific or technical activities involving significant international cooperation. NASA leads U.S. agencies in committing funds to ICRD, followed by the Department of Defense, the Agency for International Development, the National Science Foundation, the Department of Energy, and the Department of Health and Human Services. Multinational activities accounted for more than half of all the spending identified. For projects in which the United States works with just one other nation, the largest share of the dollars were claimed by projects with Russia. When parsed by field of science, aerospace and avionics was by far the largest area of dollar concentration in a field of science, followed by earth sciences and environment, physics, and health.

Measuring Benefits

Measuring the benefits of these activities requires making the expected benefits explicit and then crafting measures to help enumerate these benefits. At the intersection of the reasons for conducting cooperative research (e.g., the global nature of the research) and the type of research activity chosen (e.g., collaborative activities), expected benefits can be identified. Measures of the results and outcomes of research can provide insight into how well a program is meeting its goals and thereby producing a benefit to the United States. Using available tools to measure outputs and outcomes--bibliometric measures, milestones, surveys, and expert judgment--can provide usable information to help policymakers track the benefits the United States is receiving from ICRD. We describe a framework for applying these measures to different types of ICRD activities.

Seismic Research: A Case Study

In a case study examining cooperation in earthquake sciences and seismology, we tested the ability of the above measures to provide feedback on benefits. Based on the framework of measures developed for this project, we identified three applicable measures: bibliometrics, a survey of research participants seeking information on leveraging foreign research dollars, and expert judgment on standards development. Using these measures, we found that papers co-authored jointly by at least one U.S. national and one foreign researcher had nearly doubled over a 10-year period, with the largest growth being in multinational authorship. The survey found that, on average, the foreign financial contribution equaled the U.S. contribution. The expert judgment standards survey found that the U.S. companies were setting the standard for 80 percent of the essential research equipment used in this field. The measures indicated that the United States is receiving benefits from these activities.

The results of the case study show that the framework can point to possible useful measures. However, improved data on both ICRD spending and outcomes would greatly enhance the ability of decisionmakers to monitor these activities in real time. Government agencies may wish to take advantage of new electronic networking technologies to flag and share data on ICRD activities. In addition, agencies may wish to map ICRD activities against the existing international science and technology agreements to gain a clearer picture of where and why the United States cooperates with other countries on these projects.

---

Contents