Translational Impacts of World Trade Center Health Program Research

Overview

The World Trade Center (WTC) Health Program (hereafter Program) provides medical monitoring and treatment for its Members: the responders and survivors who were at the WTC and related sites or located within the New York City disaster area during and after the terrorist attacks of September 11, 2001. Scientific research supported by the Program provides an evidence base for the health monitoring and clinical care of Members. In 2017, the National Institute for Occupational Safety and Health (NIOSH) contracted with RAND to conduct a four-year study to inform research program planning in the Program. The purpose of this assessment is to support Program planning, not to audit the Program or evaluate whether the Program is “succeeding” according to its legislative mandates.

Methods

In the first months of this study, the RAND team and Program leaders jointly developed four research questions to guide the analyses. The following research questions reflect our intent to support the planning work of the Program and its stakeholders:

• What topics and questions have been addressed in WTC health-related research?
• To what extent is WTC health-related research translational?
• What are the facilitators of and barriers to translation of WTC health-related research?
• To what extent has WTC health-related research had impact?

We integrated five quantitative and qualitative data sources to answer the research questions through a mixed methods analysis: (1) peer-reviewed publications through October 31, 2020, (2) Program documentation of research activities through February 2021, (3) transcripts of 12 structured focus group discussions with research stakeholders from December 2019 through February 2020, (4) transcripts of individual stakeholder interviews from October through November 2020, and (5) gray literature publications through November 30, 2020.

To examine the first research question (RQ1: What topics and questions have been addressed in WTC health-related research?), we collected data from nearly 1,000 peer-reviewed publications using an electronic abstraction form. We charted these data into categories using several descriptive taxonomies for research studies. These include the Population, Intervention/Exposure, Comparator, Outcome, Timing, Setting, and Study Design (PI/ECOTSS) approach to classifying research; a classification of studies into three categories (exposure characterization, exposure-outcome linkage, and outcome modification); and the National Institutes of Environmental Health Sciences (NIEHS) Translational Framework, among others. Finally, we assessed the range and extent of research using these classifications.

To answer the second research question (RQ2: To what extent is WTC health-related research translational?), we synthesized qualitative data from gray literature, Program documentation, focus groups, and interviews. From focus group and interview transcripts, we examined definitions of translational research in respondents’ own words. We also identified stakeholders’ views on the uses of existing research in future research, planning, policy, and care. Finally, we applied a machine-learning algorithm with these same words and uses of research to identify related text passages within thousands of pages of Program documentation and gray literature. Using these sources, we examined whether and how WTC health-related research is translated into use.

To answer the third research question (RQ3: What are the facilitators of and barriers to successful translation of WTC health-related research?), we again synthesized qualitative data from gray literature, Program documentation, focus groups, and interviews. We applied a priori codebooks to all sources and sorted data into tables by code to facilitate comparisons. Next, we deductively identified themes, or concepts that describe patterns in the data. Among these themes were the factors that might positively or negatively influence research translation.

To answer the fourth research question (RQ4: To what extent has WTC health-related research had impact?), we ran several analyses. Our a priori coding approach enabled the research team to view information from all five data sources across the same condition or population of interest (e.g., WTC responders, WTC survivors, all Program-covered conditions, and a preselected set of other conditions). We selected three of these topics to explore in case studies on research translation: research involving depression and anxiety, women's health, and WTC-exposed youth. We applied the NIEHS Translational Research Framework to describe how research on these topics followed various translational pathways and achieved impact by directly informing Program planning and policy.

Findings

Our analyses resulted in the several key findings, organized by research question.

RQ1: What Topics and Questions Have Been Addressed in WTC Health-Related Research?

• Finding 1. WTC health-related research addresses a wide variety of health conditions and is balanced across Member populations.
• Finding 2. There is a high concentration of research that addresses fundamental questions about the health effects of 9/11 and a lower concentration of research on health care interventions for affected populations.

RQ2: To What Extent Is WTC Health-Related Research Translational?

• Finding 3. Program leaders, researchers, clinicians, and Members understand the fundamentals of translation and can describe, in their own words, how WTC health-related research is used. However, almost all stakeholders struggled to give a comprehensive definition of translation.
• Finding 4. The Program has a strong record of documenting and disseminating Program-funded research, but the range of stakeholder preferences for receiving information about WTC health-related research is broad and presents challenges.

RQ3: What Are the Facilitators of and Barriers to Translation of WTC Health-Related Research?

• Finding 5. Facilitators of research translation include a focused mission to improve care through research; research infrastructure, such as the colocation of researchers and clinicians at Clinical Centers of Excellence and the collection, management, and analysis of data by the WTC Data Centers and WTC Health Registry; and the communication and dissemination of findings. Barriers to research translation include methodological limitations, fragmentation of the health care system, funding constraints, variable definitions of translation, and misaligned expectations about research.

RQ4: To What Extent Has WTC Health-Related Research Had Impact?

• Finding 6. Program research is used in decisions about care, planning, and policy.
• Finding 7. Stakeholders want to know how they can contribute to maximizing the impacts of research. Some stakeholders may need information and guidance to support their involvement in Program planning, policy, and research.

These findings serve as the basis for recommendations that NIOSH could implement to reinforce and strengthen the translational impact of Program research.

Recommendations

The mission of WTC Health Program–funded research is to inform improved care for Program Members. To support this work, we offer the following recommendations to reinforce and strengthen the translational impacts of Program research:

• To ensure the Program is able to maintain a balanced portfolio in terms of topics and populations (Finding 1), the Program can build on and further develop existing research infrastructure and processes for conducting timely, objective, and continuous reviews of the Program research portfolio.
• To come to a shared understanding of translation and align expectations about what such research can achieve for Members (Findings 3 and 4), the Program can gather input from stakeholders on a novel translational research framework.
• To guide a more systematic approach to activities of the research program, the Program can pilot the translational framework in four key areas: research planning, monitoring, evaluation, and communication about research impacts (Findings 2, 3, 5, and 6).
• To be responsive to stakeholders’ interest in contributing to maximizing the impacts of research (Finding 7), the Program can implement structured processes for stakeholder engagement in program planning, policy, and research activities.

Limitations

This study had a few limitations. First, in the scoping review, we included articles published up to and including October 31, 2020, and excluded conference abstracts. Consequently, this review did not capture research that was presented in conferences but not published, nor did it capture new research that was completed after October 2020. Also, because of this review's large scope (consisting of nearly 1,000 publications), it covers a variety of populations and conditions. The scope and heterogeneity prevented a focused look within population and condition groups for summary effect estimates or strength of evidence assessments. Finally, as with all scoping reviews, some data items may be subject to misclassification.

Second, in qualitative data collection and analysis, we were unable to disentangle the degree to which certain stakeholders or views are overrepresented or underrepresented. We sought to examine the full range of stakeholder views on a given topic, but the qualitative findings may not be generalizable. As in all qualitative research, the views expressed by the stakeholders in this study (who, for the most part, are affiliated in some way with the Program) are opinions that may or may not be factually accurate.

Third, the search strategy we used to find gray literature involved a novel but unvalidated use of several search engines. As with the other qualitative data sources used in this study, the goal of examining gray literature was to ensure that no significant views of stakeholders were missed. Relatedly, the machine-learning algorithm we used to identify relevant text passages in the gray literature was developed and trained on manually coded data from focus group transcripts and therefore should be interpreted as an extension of those findings.

Finally, the NIEHS framework, although a promising way of examining a body of research, is novel and may need extensive testing and validation. It is unknown whether this framework on its own can capture all the distinctiveness and complexities of the Program's research activities.

Conclusion

More than 20 years after the attacks on 9/11, the Program continues to carry out its focused research mission: to improve health care and outcomes of its Members. In support of this mission, the Program engages in ongoing planning activities that are grounded in rigorous assessments of existing research and future needs. This study is intended to support NIOSH in program-planning activities by critically examining the past decade of Program-supported research and looking ahead to the near- and longer-term research activities of the Program.