UN Official Warns AI Is Crowding Out Human Decision-Making in Modern Warfare

Robin Geiss, director of the United Nations Institute for Disarmament Research, stated on Tuesday that artificial intelligence is on track to crowd out human decision-making in warfare. Speaking during a press briefing in New York City, Geiss highlighted that militaries are offloading critical thinking skills onto bots, a trend he described as dangerous for global security and ethical standards.

Geiss questioned whether society will allow machines to target and kill human beings, calling such a development a moral red line. He noted that handing sensitive military decisions to black box AI systems could make it more difficult for humans to prevent costly errors. This risk extends to the most sensitive arena of all, including the potential use of nuclear weapons, where human oversight is traditionally paramount.

International Calls for Ethical Constraints

These warnings echo statements made by UN Secretary General António Guterres regarding fully autonomous weapons systems, which he has referred to as killer robots. During a speech at the first-ever Global Dialogue on AI Governance conference in July, Guterres asserted that some decisions must remain forever human. He specifically emphasized that none of these decisions are more critical than the act of taking a human life, reinforcing the need for human control in lethal operations.

Pope Leo XIV issued a similar plea earlier this year in his Magnifica Humanitas encyclical. The Pope urged the global community to impose the most rigorous ethical constraints around the use of AI in warfare. These high-level statements reflect a growing consensus among international leaders that the rapid integration of AI into military structures requires strict governance to prevent unintended consequences and loss of human agency.

Military Adoption in Active Conflicts

The use of AI in warfare has grown steadily in recent years, driven by rapid technological advancements and a resurgence of military conflict worldwide. These conflicts have provided real-world testing grounds for autonomous weapons and other AI-powered military systems, fueling demand from armed forces. American startups have been quick to share their technology with nations engaged in active combat, accelerating the deployment of these tools in live environments.

According to reports, US companies like Anduril and Palantir shared their technology with Ukrainians following Russia’s invasion of Ukraine in early 2022. This collaboration demonstrated the practical application of AI in modern defense strategies. Additionally, Bloomberg reported in early 2024 that Palantir struck a deal with the Israel Defense Ministry to support war-related missions. Palantir executive vice president Josh Harris confirmed this partnership to the news outlet, highlighting the expanding role of private sector AI in national defense.

What should you do?

Security professionals and policymakers should monitor developments in autonomous weapons regulation and ethical AI frameworks. Organizations involved in defense technology should review their deployment protocols to ensure human oversight remains integral to decision-making processes. Staying informed about international guidelines, such as those from the UN, can help align corporate practices with global ethical standards.

What to do and how to stay safe: AI

• Review internal policies on the use of AI in critical decision-making processes to ensure human oversight is maintained.
• Monitor international regulatory developments regarding autonomous weapons and AI ethics to stay compliant with emerging standards.
• Assess vendor contracts for AI technologies to understand the extent of automation and potential risks associated with black box systems.
• Engage in industry dialogues and conferences focused on AI governance to share best practices and ethical considerations.

Step-by-step guide: Prompt Injection Warning Signs: How to Spot AI Manipulation

General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.

Frequently asked questions

Who is warning about AI in warfare?

Robin Geiss, director of the UN Institute for Disarmament Research, along with UN Secretary General António Guterres and Pope Leo XIV, have issued warnings about AI in warfare.

What is the main concern with AI in military decisions?

The main concern is that AI could crowd out human decision-making, making it difficult to prevent costly errors and raising ethical issues about machines targeting human beings.

Which companies are involved in supplying AI to militaries?

American startups like Anduril and Palantir have supplied AI technology to Ukraine and Israel, with Palantir striking a deal with the Israel Defense Ministry in early 2024.