Human control of military AI: safeguard or weak point?
Human judgment is the main safeguard in US military AI, and the thing a fast, jammed battlefield wears away. Pentagon policy requires only appropriate human judgment, while Palantir says Maven cuts two weeks of staff planning to one person and four hours.
Listen to this briefing
Human control of military AI: safeguard or weak point?.
Human judgment is the main safeguard in US military AI, and the thing a fast, jammed battlefield wears away. Pentagon policy requires only appropriate human judgment, while Palantir says Maven cuts two weeks of staff planning to one person and four hours.
Palantir's chief technology officer told CBS on 11 October that a day of 1,000 strikes once took two weeks of planning by 50 people and can now be planned by one person in four hours. In the same interview Shyam Sankar said military doctrine keeps a human accountable for every decision and keeps humans in the processes that involve lethal force. Both statements can be true. Together they describe the argument that has divided the Pentagon from Anthropic since February: whether human control is the safeguard that makes military AI legitimate, or the weakness a fast, contested battlefield will punish.
What the rules actually require
The governing policy is looser than the phrase "human in the loop" implies. Pentagon Directive 3000.09 requires that systems allow commanders to exercise "appropriate levels of human judgment over the use of force". The Congressional Research Service explains that this does not require manual control: a human must assess the environment and decide to deploy a weapon, which can then operate autonomously, and the requirement "does not mean that such systems are operating with a human in the loop".
The rules are also in motion. Section 3(a) of the 5 June presidential memorandum, NSPM-11, directs the Secretary of War to update 3000.09 within 90 days, a deadline that fell around 3 September. The only public version McCyber could find is the January 2023 text, so what any rewrite changes is not yet visible. Separately, H.R. 9729, the Human Authority over Autonomous Weapons Act, would write a requirement of human authority over such weapons into statute.
Two serious arguments
Anthropic's position, as the appeals court records it, is that it accepted most uses but kept two contractual exceptions, mass surveillance of Americans and lethal autonomous warfare. Dario Amodei's public case was a reliability one: fully autonomous weapons "may prove critical for our national defense" in future, but today's AI is not "reliable enough" to power them. The Pentagon's reply was that a private company cannot set limits on how government fights, and that its uses would be lawful. A 2-1 appeals panel sided with the Pentagon on the exclusion on 25 September. Its opinion called the contested limits "hardly self-defining", observing that there are "any number of possible scenarios" with greater or lesser human involvement in targeting decisions. That is an argument against vague red lines as much as against Anthropic.
The military-technical argument is older and independent of any company. Remote control depends on a communications link the enemy can attack. P.W. Singer put it in a Brookings essay as the loop being the vulnerability to cut, and a West Point analysis notes that Ukraine has reportedly fielded semi-autonomous strike drones to defeat Russian jamming. Preserving the human link can disable the weapon. These are different decisions, though: remote piloting, autonomous navigation under jamming, automated target recognition and autonomous engagement each place the human differently, and a severed link does not by itself settle where judgment belongs. Both camps therefore have a real point: judgment is what legitimises force, and latency and dependency are what lose engagements.
Where Thiel fits, and where he does not
Peter Thiel gives the clearest public statement of the case that restraint is unaffordable, though not on lethal decisions. In his interview with Walter Isaacson, taped on 16 September and published on 22 October, he said he is "still not in the pause button camp" and that "we need to hit the accelerator as hard as possible", and he broadly agreed that competition with China rules out a pause. He credited effective altruists with seeing that AI was coming, while calling parts of the movement "almost cult-like". Sankar used similar language in the CBS interview, calling effective altruism a "millenarian doomsday cult meets messianic complex".
Nothing in Thiel's interview addresses targeting, lethal force or human control. He said AI companies "have a lot of internal controls" and that government "doesn't really know how to regulate it". Linking "no one can afford to slow down" to pressure on the targeting cycle remains McCyber's inference, not his argument.
The compressed human
The Minab school strike shows what that pressure looks like in practice. Bloomberg reported on 18 September, citing people familiar with an unreleased Pentagon review, that some Central Command personnel relied too heavily on the AI in Maven. Target-list work was compressed to minutes. About three dozen people were in the chain and all answered "yes", and investigators found "high confidence throughout". A UN report, A/HRC/63/61, cites independent information that 157 people were killed, including 123 children. It says the strike "went beyond mere negligence" and hit its intended point of impact, and it does not mention AI. Reported overreliance is not demonstrated causation, and the UN findings cannot corroborate an AI contribution.
Sankar told CBS the reporting was wrong and the likely cause was human error. Both can hold: the school was reportedly listed as a military site on outdated information, which is a human failure, and the open question is whether the system made it harder to catch. Bloomberg adds that civilian-harm-mitigation staff had been cut by roughly 90%, at Central Command from ten to one, that no such review took place for Minab, and that it is unclear whether one would have caught the error. On 22 September Bloomberg quoted an outside analyst saying the "same errors happen faster and at a larger scale", and Stanley saying targeting had "outpaced several other warfighting functions". The Pentagon described a "human-directed process".
Final approval remains human. But Sankar's own description of targeting is a process in which the system takes a desired effect, identifies related targets, runs them against no-strike lists and civilian proximity, and returns options. The machine finds, correlates, ranks and presents; the person authorises. The question that matters is not whether someone clicks, but whether that person has the time, independent evidence and practical freedom to reject the framing they are shown.
The strongest case for fewer humans
The best argument against treating the human as a safeguard comes from two directions. Sankar argues that machine checks are more repeatable than people working against the clock, and Minab itself began with a human-maintained database. A piece for the Australian Army argues the dilemma is false because even autonomous weapons begin with a human setting engagement parameters, so the human "starts the loop". On that view the task is to decide where human judgment is exercised, and to place it where it counts.
That is a serious position, and it could be compatible with the one Anthropic took, depending on the engagement parameters and the precise contractual limit. It also leaves the hardest cases untouched, because it assumes the human who sets the parameters knows what the machine will do with them. When planning for a thousand strikes is compressed into four hours of one operator's work, where does independent scrutiny enter the process?
This is Part 2 of 3 in Maven after Claude. Part 1, Palantir Maven after Claude: the swap, by Palantir's account, covers the replacement of Claude in Maven. Part 3 asks who controls an AI agent inside someone else's system.
Disclosure: Anthropic is a named actor in this essay. McCyber uses Anthropic's Claude models in its editorial workflow, as set out on the About page.
Sources: CBS News (Face the Nation transcript, 11 Oct); Anthropic PBC v. U.S. Department of War, No. 26-1049 (D.C. Cir., 25 Sep 2026); Associated Press (via ClickOrlando); NSPM-11 (White House, 5 Jun 2026); H.R. 9729 (congress.gov); Congressional Research Service (IN12669); Bloomberg (18 and 22 Sep); UN Human Rights Council, A/HRC/63/61; Walter Isaacson, The Builders (Kaleidoscope), interview with Peter Thiel, YouTube auto-caption transcript; Brookings; Modern War Institute at West Point; Australian Army Cove.
Enable JavaScript and use a browser with spoken audio support to listen.
Read aloud by your browser. Voice and availability depend on your device.
From the rule governing human judgment in targeting, to Palantir's claimed compression of planning time, to the Minab strike. Sources: CBS (11 Oct); DoD Directive 3000.09; NSPM-11; CRS IN12669; Bloomberg (18, 22 Sep); UN A/HRC/63/61.
Palantir’s chief technology officer told CBS on 11 October that a day of 1,000 strikes once took two weeks of planning by 50 people and can now be planned by one person in four hours. In the same interview Shyam Sankar said military doctrine keeps a human accountable for every decision and keeps humans in the processes that involve lethal force. Both statements can be true. Together they describe the argument that has divided the Pentagon from Anthropic since February: whether human control is the safeguard that makes military AI legitimate, or the weakness a fast, contested battlefield will punish.
What the rules actually require
The governing policy is looser than the phrase “human in the loop” implies. Pentagon Directive 3000.09 requires that systems allow commanders to exercise “appropriate levels of human judgment over the use of force”. The Congressional Research Service explains that this does not require manual control: a human must assess the environment and decide to deploy a weapon, which can then operate autonomously, and the requirement “does not mean that such systems are operating with a human in the loop”.
The rules are also in motion. Section 3(a) of the 5 June presidential memorandum, NSPM-11, directs the Secretary of War to update 3000.09 within 90 days, a deadline that fell around 3 September. The only public version McCyber could find is the January 2023 text, so what any rewrite changes is not yet visible. Separately, H.R. 9729, the Human Authority over Autonomous Weapons Act, would write a requirement of human authority over such weapons into statute.
Two serious arguments
Anthropic’s position, as the appeals court records it, is that it accepted most uses but kept two contractual exceptions, mass surveillance of Americans and lethal autonomous warfare. Dario Amodei’s public case was a reliability one: fully autonomous weapons “may prove critical for our national defense” in future, but today’s AI is not “reliable enough” to power them. The Pentagon’s reply was that a private company cannot set limits on how government fights, and that its uses would be lawful. A 2-1 appeals panel sided with the Pentagon on the exclusion on 25 September. Its opinion called the contested limits “hardly self-defining”, observing that there are “any number of possible scenarios” with greater or lesser human involvement in targeting decisions. That is an argument against vague red lines as much as against Anthropic.
The military-technical argument is older and independent of any company. Remote control depends on a communications link the enemy can attack. P.W. Singer put it in a Brookings essay as the loop being the vulnerability to cut, and a West Point analysis notes that Ukraine has reportedly fielded semi-autonomous strike drones to defeat Russian jamming. Preserving the human link can disable the weapon. These are different decisions, though: remote piloting, autonomous navigation under jamming, automated target recognition and autonomous engagement each place the human differently, and a severed link does not by itself settle where judgment belongs. Both camps therefore have a real point: judgment is what legitimises force, and latency and dependency are what lose engagements.
Subscribe
New essays and snapshots, by email, roughly weekly.
Where Thiel fits, and where he does not
Peter Thiel gives the clearest public statement of the case that restraint is unaffordable, though not on lethal decisions. In his interview with Walter Isaacson, taped on 16 September and published on 22 October, he said he is “still not in the pause button camp” and that “we need to hit the accelerator as hard as possible”, and he broadly agreed that competition with China rules out a pause. He credited effective altruists with seeing that AI was coming, while calling parts of the movement “almost cult-like”. Sankar used similar language in the CBS interview, calling effective altruism a “millenarian doomsday cult meets messianic complex”.
Nothing in Thiel’s interview addresses targeting, lethal force or human control. He said AI companies “have a lot of internal controls” and that government “doesn’t really know how to regulate it”. Linking “no one can afford to slow down” to pressure on the targeting cycle remains McCyber’s inference, not his argument.
The compressed human
The Minab school strike shows what that pressure looks like in practice. Bloomberg reported on 18 September, citing people familiar with an unreleased Pentagon review, that some Central Command personnel relied too heavily on the AI in Maven. Target-list work was compressed to minutes. About three dozen people were in the chain and all answered “yes”, and investigators found “high confidence throughout”. A UN report, A/HRC/63/61, cites independent information that 157 people were killed, including 123 children. It says the strike “went beyond mere negligence” and hit its intended point of impact, and it does not mention AI. Reported overreliance is not demonstrated causation, and the UN findings cannot corroborate an AI contribution.
Sankar told CBS the reporting was wrong and the likely cause was human error. Both can hold: the school was reportedly listed as a military site on outdated information, which is a human failure, and the open question is whether the system made it harder to catch. Bloomberg adds that civilian-harm-mitigation staff had been cut by roughly 90%, at Central Command from ten to one, that no such review took place for Minab, and that it is unclear whether one would have caught the error. On 22 September Bloomberg quoted an outside analyst saying the “same errors happen faster and at a larger scale”, and Stanley saying targeting had “outpaced several other warfighting functions”. The Pentagon described a “human-directed process”.
Final approval remains human. But Sankar’s own description of targeting is a process in which the system takes a desired effect, identifies related targets, runs them against no-strike lists and civilian proximity, and returns options. The machine finds, correlates, ranks and presents; the person authorises. The question that matters is not whether someone clicks, but whether that person has the time, independent evidence and practical freedom to reject the framing they are shown.
The strongest case for fewer humans
The best argument against treating the human as a safeguard comes from two directions. Sankar argues that machine checks are more repeatable than people working against the clock, and Minab itself began with a human-maintained database. A piece for the Australian Army argues the dilemma is false because even autonomous weapons begin with a human setting engagement parameters, so the human “starts the loop”. On that view the task is to decide where human judgment is exercised, and to place it where it counts.
That is a serious position, and it could be compatible with the one Anthropic took, depending on the engagement parameters and the precise contractual limit. It also leaves the hardest cases untouched, because it assumes the human who sets the parameters knows what the machine will do with them. When planning for a thousand strikes is compressed into four hours of one operator’s work, where does independent scrutiny enter the process?
Disclosure: Anthropic is a named actor in this essay. McCyber uses Anthropic’s Claude models in its editorial workflow, as set out on the About page.
Sources: CBS News (Face the Nation transcript, 11 Oct); Anthropic PBC v. U.S. Department of War, No. 26-1049 (D.C. Cir., 25 Sep 2026); Associated Press (via ClickOrlando); NSPM-11 (White House, 5 Jun 2026); H.R. 9729 (congress.gov); Congressional Research Service (IN12669); Bloomberg (18 and 22 Sep); UN Human Rights Council, A/HRC/63/61; Walter Isaacson, The Builders (Kaleidoscope), interview with Peter Thiel, YouTube auto-caption transcript; Brookings; Modern War Institute at West Point; Australian Army Cove.
