As frequent visitors to the Ethics Unwrapped website know, one of our focuses is behavioral ethics, the study of the psychology of moral decision making. Defense correspondent Katrina Manson’s new book Project Maven: A Marine Colonel, His Team, and the Dawn of AI Warfare (2026) prompts us to revisit this topic.

As we noted in an earlier blog post (“AI Warfare Is Already Here. We Need the AI Ethics to Go with It”), Manson’s book focuses largely on the efforts of maverick Marine Colonel Drew Cukor to induce various units of the U.S. military to adopt AI warfare tools and the military bureaucracy’s reactions to his efforts. Cukor calls Project Maven “the Defense Department’s effort to embed AI into the world’s most complex and consequential operational workflows.” While Manson’s books is decidedly not about behavioral ethics, in describing how the U.S. military has adopted and used AI warfare over the past several years, Manson unintentionally gives several examples of behavioral ethics principles in action.

For example, there’s the status quo bias, people’s tendency to prefer their current state to alternatives, all things being equal. Psychologist Shane Frederick notes: “Considerable anecdotal and experimental evidence suggests that the option one currently possesses or customarily chooses is preferred over other options to a degree that is difficult to [rationally] justify.” Because of the status quo bias, only 50% or so of Americans are using AI on the job compared to more than 80% in China and several other countries even though the U.S. leads the world in AI research and product development. Several scenarios described by Manson seem to illustrate strong initial resistance by the U.S. military to testing or adopting the tools offered by Project Raven consistent with the status quo bias. Cukor told Manson that the greatest difficulty in transitioning to Project Maven “is just getting people to use it.”

Once soldiers do begin using AI, the danger of ethical failures comes to the fore. UN Secretary General Antonio Guterres has stated the popular view that: “The prospect of machines with the discretion and power to take human life is morally repugnant.” But Project Maven provides tools that can do exactly that. Manson reports that a “single click could send coordinates through a tactical data link to a specific weapons platform so that it could fire at the target.” This will be the ethical focus of this blog.

The military typically promises vaguely to “keep a human in the loop,” but this may not mean much for several reasons based in behavioral psychology. And this ethical challenge is the focus of our discussion.

A significant concern is the automation bias, a well-documented tendency of people to over-rely on automated systems, which can lead to a higher risk of accidents and errors. The bias can lead people to trust information produced by a machine or an algorithm over contradictory information provided by other humans or even by their own physical senses. (Kahn et al.)

Once soldiers have cast their lot with the AI’s judgment, a further problem is that humans are plagued by the confirmation bias, a tendency to interpret new information consistent with conclusions they have already drawn. Manson gives a chilling example of U.S. drones killing ten civilians where the Air Force Inspector General blamed the confirmation bias for “warping operators’ interpretation of what they were seeing.” This particular case seems not to have involved AI, but it is illustrative of the confirmation bias’s impact.

The dangers of the automation and confirmation biases may be ratcheted up in the current environment where AI chatbots are known to flatter users and parrot their conclusions back to them rather than giving them objective advice. “Models often learn to flatter users, a tendency known as sycophancy, and will sometimes prioritize this over honesty.” (Farrow & Marantz)  Manson reports that a military AI advisor

…warned me that he had learned that chatbots could easily reinforce cognitive bias. Depending on the way a question was asked, he said, AI could deliver multiple erroneous reasons to support the questioner’s desired end state, such as undertaking a lethal strike or a raid, spluttering out why something false might be true (“here’s why this strike is valid and legal”).

An additional complicating factor is framing, the idea that people’s judgments, including their moral judgments, are heavily influenced by what is in their frame of reference at the time they make their decision.  Manson notes that some military leaders worry that the AI screens soldiers use to monitor recommendations for strikes very much resembles gaming consoles so “that soldiers might be lulled into using ‘game ethics’ in place of personal ethics.” This is not a frivolous concern. Studies indicate that lawyers engaged in negotiation on behalf of their clients who frame the exercise as simply a “game” to be won or lost act less ethically than do lawyers who resist that “gaming” frame of mind and view negotiation as an attempt to serve the purposes of both sides. (Cohen et al.)

All this adds up to a strong possibility that the human-in-the-loop may forfeit any independent judgment and simply rubber stamp the AI’s recommendations. An Israeli intelligence officer has said: “I would invest 20 seconds for each target this stage, and do dozens of them every day. I had zero added-value as a human, apart from being a stamp of approval.” (Shabibi & Croft). This sounds like a cavalier attitude, but it is consistent with Manson’s observation that humans tend not to “feel as responsible for mistakes when AI is involved.” Manson notes: “[a] former member of JSOC told me that it was easier to take someone’s life watching it on a screen than physically beside them.” This observation, in turn, is consistent with a phenomenon known as the tangible and the abstract, which is the well-established phenomenon that we can become blind to the negative consequences of our actions if we discount the impact of our decisions on people far removed in location and/or time.

In every endeavor of life where there are ethical challenges, including war making, there will almost certainly be principles of behavioral ethics that can help light the way. And just as certainly, we have a moral obligation to learn about the traps our minds can lay for us with AI lest we stumble in the dark and lose our way.

 

Sources:

Peter Asaro, “Autonomous Weapons and the Ethics of Artificial Intelligence,” in Ethics of Artificial Intelligence (S. Matthew Liao, ed. 2020).

Cara Biasucci & Robert Prentice, Behavioral Ethics in Practice: Why We Sometimes Make the Wrong Decisions (2021).

Taya R. Cohen et al., “Honesty Among Lawyers: Moral Character, Game Framing, and Honest Disclosure in Negotiations,” Negotiation Journal, 38(2): 199-234 (2022).

Drew Cukor, “I Helped Build the Pentagon’s AI Transformation. Corporate America is Making Every Mistake We Almost Made,” Fortune, May 11, 2026, at https://fortune.com/2026/05/11/drew-cukor-project-maven-ai-adoption-china-corporate-transformation/.

Ronan Farrow & Andrew Marantz, “Sam Altman May Control Our Future—Can He Be Trusted?” New Yorker, Apr. 6, 2026, at https://www.newyorker.com/magazine/2026/04/13/sam-altman-may-control-our-future-can-he-be-trusted.

Shane Frederick, “Automated Choice Heuristics,” in Heuristics and Biases: The Psychology of Intuitive Judgment (Thomas Gilovich et al., eds. 2002).

Lauren Kahn et al., ÁI Safety and Automation Bias: The Downside of Human-in-the-Loop,” Center for Security and Emerging Technology (Nov. 2024), at https://cset.georgetown.edu/publication/ai-safety-and-automation-bias/.

Daniel Kahneman et al., “Anomalies: The Endowment Effect, Loss Aversion, and Status Quo Bias,” in Choices, Values, and Frames (Daniel Kahneman & Amos Tversky, eds. 2000).

Gideon Lewis-Kraus, “A New Book Charts the Creation of a Secretive System that Automates Warfare for the Military. The Progression from Target Identification to Target Destruction Is Four Clicks,” New Yorker, Apr. 15, 2026, at https://www.newyorker.com/books/under-review/how-project-maven-put-ai-into-the-kill-chain.

Katrina Manson, Project Maven: A Marine Colonel, His Team, and the Dawn of AI Warfare (2026).

David E. Sanger et al., “Trump Lays Out Vision of Power Restrained Only by ‘My Own Morality,” New York Times, Jan. 8, 2026, at https://www.nytimes.com/2026/01/08/us/politics/trump-interview-power-morality.html.

Jason Scholz & Jai Galliott, “The Case for Ethical AI in the Military,” in The Oxford Handbook of Ethics of AI (Markus D. Dubber et al., eds. 2021).

Namir Shabibi & Alex Croft, “’We Want to Use It for Everything’: How Project Maven Became Central to America’s AI-Powered Warfare,” The Independent, March 10, 2026, at https://www.independent.co.uk/news/world/americas/project-maven-ai-us-airstrike-iraq-anthropic-b2929138.html.

Brandi Vincent, “Feinberg’s New Maven Directive Sets AI-enabled Decision-Making as ‘The Cornerstone’ for CJADC2,” Center for Security and Emerging Technology, April 4, 2026, at https://cset.georgetown.edu/article/feinbergs-new-maven-directive-sets-ai-enabled-decision-making-as-the-cornerstone-for-cjadc2/.

Russell Wald & Sha Sajadieh, “Americans Are Down on AI. These Two Caricatures Are to Blame,” Washington Post, April 28, 2026.

Videos

Behavioral Ethics: https://ethicsunwrapped.utexas.edu/glossary/behavioral-ethics.

Confirmation Bias: https://ethicsunwrapped.utexas.edu/glossary/confirmation-bias

Framing: https://ethicsunwrapped.utexas.edu/video/framing

“Running with Scissors: AI and the Race for the Future,” at https://ethicsunwrapped.utexas.edu/video/running-with-scissors-ai-and-the-race-for-the-future.

Tangible & Abstract: https://ethicsunwrapped.utexas.edu/video/tangible-abstract.