U.S. Marines with 1st Tank Battalion, 1st Marine Division participate in a field exercise (FEX) at Marine Corps Air Ground Combat Center Twentynine Palms, California, Oct. 22, 2019, photo by Sgt. Miguel A. Rosales/U.S. Marine Corps

commentary

(War on the Rocks)

November 4, 2019

First, Manage Security Threats to Machine Learning

U.S. Marines with 1st Tank Battalion, 1st Marine Division participate in a field exercise at Marine Corps Air Ground Combat Center Twentynine Palms, California, Oct. 22, 2019

Photo by Sgt. Miguel A. Rosales/U.S. Marine Corps

by Rand Waltzman and Thomas S. Szayna

This article was submitted in response to a call for ideas issued by the co-chairs of the National Security Commission on Artificial Intelligence, Robert Work and Eric Schmidt. You can find all of RAND's submissions here.

The U.S. Army tank brigade was once again fighting in the Middle East. Its tanks were recently equipped with a computer vision-based targeting system that employed remotely controlled drones as scouts. Unfortunately, adversary forces deceived the vision system into thinking grenade flashes were actually cannon fire. The tank operators opened fire on their comrades two miles away. Although the U.S. brigade won the battle, they lost 6 soldiers, 5 tanks, and 5 fighting vehicles —all from friendly fire. The brigade commander said, “our equipment is so lethal that there is no room for mistakes.”

This story is based on an actual event. The tanks involved did not have automated computer vision systems —but someday they will.…

The remainder of this commentary is available at warontherocks.com


Rand Waltzman is a senior information scientist at the nonprofit, nonpartisan RAND Corporation. He has served two terms as an AI program manager at the Defense Advanced Research Projects Agency (DARPA). Thomas Szayna is a senior political scientist at RAND. His research has focused on strategic planning for the U.S. armed forces and the future security environment.

This commentary originally appeared on War on the Rocks on November 4, 2019. Commentary gives RAND researchers a platform to convey insights based on their professional expertise and often on their peer-reviewed research and analysis.