RAND researchers interviewed data scientists and engineers with experience in building artificial intelligence and machine learning (AI/ML) models in industry or academia to investigate why AI projects fail. They synthesized the experts' experiences to develop recommendations for smart implementation of AI. The lessons from earlier efforts to build and apply AI/ML will be helpful for others to avoid the same pitfalls.

The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed

Avoiding the Anti-Patterns of AI

James Ryseff, Brandon De Bruhl, Sydne J. Newberry

ResearchPublished Aug 13, 2024

To investigate why artificial intelligence and machine learning (AI/ML) projects fail, the authors interviewed 65 data scientists and engineers with at least five years of experience in building AI/ML models in industry or academia. The authors identified five leading root causes for the failure of AI projects and synthesized the experts' experiences to develop recommendations to make AI projects more likely to succeed in industry settings and in academia.

By some estimates, more than 80 percent of AI projects fail — twice the rate of failure for information technology projects that do not involve AI. Thus, understanding how to translate AI's enormous potential into concrete results remains an urgent challenge. The findings and recommendations of this report should be of interest to the U.S. Department of Defense, which has been actively looking for ways to use AI, along with other leaders in government and the private sector who are considering using AI/ML. The lessons from earlier efforts to build and apply AI/ML will help others avoid the same pitfalls.

Key Findings

Five leading root causes of the failure of AI projects were identified

  • First, industry stakeholders often misunderstand — or miscommunicate — what problem needs to be solved using AI.
  • Second, many AI projects fail because the organization lacks the necessary data to adequately train an effective AI model.
  • Third, in some cases, AI projects fail because the organization focuses more on using the latest and greatest technology than on solving real problems for their intended users.
  • Fourth, organizations might not have adequate infrastructure to manage their data and deploy completed AI models, which increases the likelihood of project failure.
  • Finally, in some cases, AI projects fail because the technology is applied to problems that are too difficult for AI to solve.

Recommendations

  • Industry leaders should ensure that technical staff understand the project purpose and domain context: Misunderstandings and miscommunications about the intent and purpose of the project are the most common reasons for AI project failure.
  • Industry leaders should choose enduring problems: AI projects require time and patience to complete. Before they begin any AI project, leaders should be prepared to commit each product team to solving a specific problem for at least a year.
  • Industry leaders should focus on the problem, not the technology: Successful projects are laser-focused on the problem to be solved, not the technology used to solve it.
  • Industry leaders should invest in infrastructure: Up-front investments in infrastructure to support data governance and model deployment can reduce the time required to complete AI projects and can increase the volume of high-quality data available to train effective AI models.
  • Industry leaders should understand AI's limitations: When considering a potential AI project, leaders need to include technical experts to assess the project's feasibility.
  • Academia leaders should overcome data-collection barriers through partnerships with government: Partnerships between academia and government agencies could give researchers access to data of the provenance needed for academic research.
  • Academia leaders should expand doctoral programs in data science for practitioners: Computer science and data science program leaders should learn from disciplines, such as international relations, in which practitioner doctoral programs often exist side by side at universities to provide pathways for researchers to apply their findings to urgent problems.

Topics

Document Details

Citation

RAND Style Manual
Ryseff, James, Brandon De Bruhl, and Sydne J. Newberry, The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed: Avoiding the Anti-Patterns of AI, RAND Corporation, RR-A2680-1, 2024. As of October 10, 2024: https://www.rand.org/pubs/research_reports/RRA2680-1.html
Chicago Manual of Style
Ryseff, James, Brandon De Bruhl, and Sydne J. Newberry, The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed: Avoiding the Anti-Patterns of AI. Santa Monica, CA: RAND Corporation, 2024. https://www.rand.org/pubs/research_reports/RRA2680-1.html.
BibTeX RIS

Research conducted by

Funding for this research was provided by RAND National Defense Research Institute (NDRI) exploratory research funding that was provided through the FFRDC contract and approved by NDRI's primary sponsor. The research was conducted within the Acquisition and Technology Policy Programm of the RAND National Security Research Division (NSRD).

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.