Understanding the Theoretical Limits of AI-Enabled Pathogen Design: Insights from a Delphi Study

This study presents the findings from a structured expert elicitation designed to assess the near-term limits of artificial intelligence–(AI)-assisted and AI-driven biological design, focusing on the feasibility of engineering novel pathogens. Motivated by increasing concern about the dual-use potential of AI in bioengineering, we sought to distinguish among impossible threats, theoretically possible but implausible threats, and threats that warrant concrete risk mitigation now or in the near-term future. Rather than forecasting specific risks, our central objective was to identify which biological and AI-related constraints might serve as hard or persistent barriers to misuse.

To do so, we conducted two parallel Delphi elicitations, engaging experts in biology and in AI to independently and comparatively evaluate the limits that each field faces. We asked the participants to assess the validity and applicability of a set of proposed constraints that included biological trade-offs, such as transmissibility and environmental stability, and technical challenges in data availability and AI model generalization. We determined the time horizon for this assessment to extend from 2025 to 2027, a period in which substantial advances are plausible but full autonomy in biological design by AI is unlikely.

The overarching goal of this effort is to inform policy and risk analysis by clarifying which scenarios fall outside the plausible envelope of near-future capabilities and which will remain impossible indefinitely. By focusing on what might be impossible or unlikely, this work aims to help refine the scope of biosecurity planning and improve the signal-to-noise ratio in discussions about AI-assisted biological design.

We employed the Delphi method to elicit expert judgment on the limits of AI-enabled pathogen design, drawing from two distinct panels: one composed of experts in biology and bioengineering and the other in AI and computational biology. One limitation of this work is that the participant groups were small (10 to 12 members per group) and did not include participants from large AI companies. Participants were asked to evaluate a curated list of proposed limits along two dimensions: (1) validity (whether the limit is likely to be real, either in the near term or fundamentally) and (2) applicability (whether the limit, if valid, would meaningfully constrain capabilities). Our Delphi study was designed not to force consensus but to identify areas of agreement and persistent disagreement and underlying rationales. Although small and interdisciplinary by design, the elicitations provided qualitative and semiquantitative insight into the boundaries of what is currently (as of 2025) and plausibly possible in terms of using AI in the domain of biological threat design and what will remain implausible, even in the future.

Research Questions

• What are the key areas of uncertainty or disagreement regarding the limits of engineered pathogens and AI-driven bioengineering?
• What are significant unknowns that emerge from expert elicitation in the fields of biophysics, genetic engineering, and machine learning?
• How do expert opinions converge or diverge regarding the plausibility of different proposed limits to biorisk and to bio-related AI models?
• What are the specific conjectured limits that might make it possible to verify or falsify current assumptions about the scalability and functionality of AI-driven bioengineering systems?
• Which assumptions about the limits of AI systems and bioengineering methods are most likely to be challenged or confirmed? Which types of breakthroughs or work will challenge or confirm these assumptions?

Key Findings

In this section, we summarize our findings based on the responses of the experts who participated in the Delphi study.

Overall Expert Consensus

• Near-term AI is an assistive tool rather than an independent driver of biological design. Participants explained that AI is most effective as a tool to assist experts rather than an independent driver of pathogen optimization or design. However, they also believed that this could change—for example, if generally intelligent systems were created or other advances eliminated current barriers. Still, they agreed that such a change would not be likely until after 2027.
• AI already helps experts do bioengineering and adjacent tasks in various ways. Participants across both elicitations emphasized that existing AI systems augment biological and bioengineering research by optimizing designs of biological materials to be engineered and the research and validation processes, assisting with pattern recognition, and greatly speeding up hypothesis generation. AI might also be able to automate laboratory work in the near term. However, AI remains fundamentally limited by the need for human guidance, interpretation, and validation.
  • A few experts believed that future versions of general language models would have substantially more capability, but how quickly this would occur was disputed.
• The risks posed by AI-enabled or AI-assisted biology (AIxBio) are shifting and could greatly increase. Outside the two-year time frame, experts were very unsure how rapidly capabilities would evolve, and their views varied from expecting slow progress and marginally increased risk to being concerned that models might rapidly gain the ability to autonomously design novel bioweapons at some threshold capability level. Specific enablers of these shifting risks include automation, simulation, progress in clinical applications, and progress toward generally capable AI.
• Limits are interdependent and context dependent. A key takeaway from both the biological and AI elicitations is that many of the identified limits are interdependent, making it challenging to consider any single constraint in isolation. Participants from both studies consistently highlighted how various biological and technological constraints overlap, interact, and mutually reinforce one another, complicating efforts to engineer novel pathogens or enhance existing ones.
• AI effectiveness depends on the quality of biological data. The effectiveness of existing AI systems in biological design is heavily dependent on the availability and quality of biological data. Participants from both elicitations emphasized that data biases, gaps, and inconsistencies remain significant barriers to AIxBio, particularly when it comes to generating novel pathogens or predicting complex biological functions accurately.
• No strong fundamental limit to AI capabilities was found. Despite the practical and significant near-term challenges, the possibility that AI could design pathogens is not outside the fundamental limits of biological systems nor outside the likely future capacity of increasingly more-general AI capabilities.

Expert Consensus About AI Limits to Pathogen Engineering

• In the coming years, AI will be able to assist with designing—but not independently design—pathogens. Existing AI systems help human researchers by speeding up tasks, but they cannot autonomously design novel biological threats or enable nonexperts to do so. Advances in AI that enable more-general intelligence could change this, as could other domain-specific advances.
• The power of AI systems depends heavily on the quality and quantity of biological data used to train them. Poor, incomplete, or biased datasets could somewhat limit what AI can learn, especially for such tasks as predicting how a new pathogen might behave in a real-world setting. Many relevant types of data are not collectible, making this limit hard to circumvent without either reliable simulation or significant progress toward general AI.
• Existing AI systems struggle with complex and long-term predictions. Tasks involving host-pathogen interactions, immune responses, or evolutionary trade-offs are too complex for foreseeable near-term AI models to predict reliably. Computational simulations are not currently sufficient for addressing this, and laboratory validation is at best difficult.
• Experimental validation is a major bottleneck. Even when AI proposes biological designs, testing them in real-world conditions is slow, expensive, and often infeasible without expert oversight. Reliable simulation or AI-assisted lab automation could mitigate but not necessarily eliminate this bottleneck.
• General-purpose models are often less useful than tailored ones. Large language models and other general AI tools are less effective than highly specialized tools designed for narrow biological problems. Unless and until large language models are even more generally capable or can effectively use those tools and integrate them into reliable agentic workflows, the use of AI tools and capabilities will be restricted to experts, making independent design by less sophisticated actors by the route impossible.

Expert Consensus About Biological Limits to Pathogen Engineering

• Pathogen transmissibility has physical and biological limits. A pathogen's ability to spread is capped by how it moves among people, how stable it is in the environment, and how it affects host behavior. These limits are close to immutable but do not critically limit the creation of much more dangerous bioengineered pathogens.
• Environmental stability is hard to engineer. Many pathogens quickly degrade outside a host, but making them more stable often comes with trade-offs (such as reducing their ability to replicate effectively). These trade-offs are difficult to predict computationally, and they limit the viability of engineered changes to pathogens.
• Fitness trade-offs constrain engineered pathogens. When pathogens are modified, they often become weaker in other ways. Thus, designing a virus that is both highly contagious and durable is harder than increasing contagion or durability while decreasing the other fitness factor.
• Limits depend on the type of pathogen. Constraints vary widely among viruses, bacteria, and other organisms—bioengineering that is difficult to achieve with one organism might be easier to achieve with another, but no design is unconstrained.
• Human responses also impose real limits. Social distancing, vaccines, and medical countermeasures can rapidly blunt the spread of even well-designed pathogens, acting as strong constraints. However, historical examples of failures to reliably contain natural disease outbreaks show that human responses might have limited impacts.

Policy Conclusions

Despite growing concerns about AIxBio, expert assessments from our elicitation indicated that many of the most-extreme speculative scenarios—such as autonomous AI systems independently designing radically novel pathogens—remain implausible within the next one to two years, but advances in AI systems and other types of progress could change the risk landscape greatly and should be monitored closely. Current and near-future AI tools are more likely to serve as accelerators for already capable actors rather than as independent agents of risk, primarily enhancing the speed and efficiency of optimizing known biological threats.

Some experts anticipated rapid gains in AI capabilities, but there was widespread uncertainty about the pace and direction of such progress. Most experts emphasized that any projections beyond the near term (i.e., from 2025 to 2027) are highly speculative.

Our results show that the two most-immediate governance needs are prioritizing the monitoring of scientific progress and future risks and mitigating the most-plausible and most-actionable risk pathways—specifically, the use of AI tools to assist in optimizing or modifying existing pathogens.

For monitoring changes to risk, the most-promising areas to monitor are

• capabilities for clinical applications of bioengineering
• laboratory automation that could provide more-rapid feedback for both applications and training of AI systems
• reliable simulations of biological systems that reduce the need for experimental validation or reliance on training data
• generally capable AI systems.

Advances in any of these areas will indicate that current limits have been weakened or removed.

In the near term, researchers should clarify where hard limits exist, with a particular focus on the intersection of biological and AI constraints. Such clarification can significantly sharpen threat models and help policymakers focus on credible misuse cases. In this context, we found that biological data infrastructure emerges as a key leverage point: The quality, accessibility, and oversight of genomic and experimental data will shape both the capabilities and the risks of AI-assisted bioengineering.

In addition, there are immediate and general benefits to reinforcing traditional biosecurity safeguards as AI tools increasingly lower the threshold for sophisticated misuse. Strengthening and globally coordinating gene-synthesis screening can help prevent the malicious or accidental creation of risky sequences, especially considering that participants agreed that AI already accelerates iterative design. As a result of these risks, emerging cloud and automated lab platforms should be subject to oversight frameworks that include identity verification, experiment prescreening, and audit trails to prevent or at least detect misuse. In parallel, increased and sustained investment in core pandemic preparedness—such as rapid diagnostics, scalable vaccine platforms, and public health surge capacity—provides a robust backstop. This investment comes with immediate benefits for reducing the disease burden from nonengineered pathogens and new natural outbreaks, and it could reduce the impact of any AI-assisted biological threat that might bypass upstream controls.

In addition, cultivating a culture of responsible AI use in biological research is essential, particularly among developers and institutions deploying increasingly capable tools. Encouraging interdisciplinary coordination—especially among AI practitioners, biologists, and biosecurity experts—can help bridge gaps in understanding and align technical capabilities with risk awareness. This cross-domain engagement is foundational for building flexible governance frameworks that can adapt to fast-moving developments through such mechanisms as horizon scanning and iterative threat modeling. Although self-governance approaches (such as predeployment review and responsible disclosure) are also important and should play larger roles in fostering safety norms, they are inherently limited in anticipating novel misuse pathways or constraining determined actors, including state-sponsored programs. This underscores the need for complementary regulatory and institutional safeguards to be put in place as quickly as possible.