Here’s our initial take on OpenAI’s description of its contract with the Department of Defense. We wish we could trust OpenAI to provide an accurate, spin-free report on the contents of the contract or how it will be implemented, but we don’t. So we’re approaching this skeptically.
It’s important to note up front that we don’t have the full contract, and it’s hard to say much with certainty without it, including about a purported contractual provision quoted in isolation. What OpenAI provided is a small snippet of out-of-context contractual language and a nonbinding account of what activities they claim the contract authorizes and prohibits. Interpreting a contract requires reading the contract as a whole to give effect to all its provisions and to ascertain the mutual intent of the parties. We obviously can’t do that with what OpenAI has provided to date. (We don’t even know how the contract defines the obviously critical term “AI System” used repeatedly in the quoted language.)
Subject to the above, let’s turn to the contractual language, which we’ll quote here in full first, so you don’t have to flip back and forth:
The Department of War may use the AI System for all lawful purposes, consistent with applicable law, operational requirements, and well-established safety and oversight protocols. The AI System will not be used to independently direct autonomous weapons in any case where law, regulation, or Department policy requires human control, nor will it be used to assume other high-stakes decisions that require approval by a human decisionmaker under the same authorities. Per DoD Directive 3000.09 (dtd 25 January 2023), any use of AI in autonomous and semi-autonomous systems must undergo rigorous verification, validation, and testing to ensure they perform as intended in realistic environments before deployment.
For intelligence activities, any handling of private information will comply with the Fourth Amendment, the National Security Act of 1947 and the Foreign Intelligence and Surveillance Act of 1978, Executive Order 12333, and applicable DoD directives requiring a defined foreign intelligence purpose. The AI System shall not be used for unconstrained monitoring of U.S. persons’ private information as consistent with these authorities. The system shall also not be used for domestic law-enforcement activities except as permitted by the Posse Comitatus Act and other applicable law.
As an initial matter, the quoted language from the contract does not purport to prohibit the government from any uses beyond what is already prohibited by law. There is some question, however, about whether it even effectively prohibits the government from illegal uses, and if so, what law applies.
With respect to the first paragraph (on autonomous weapons), mere reference to “all lawful purposes” and “applicable law” cannot be relied on to constrain the government’s actions. Courts have found that similar limiting language does not incorporate the law such that the government committing illegal acts as part of performance amounts to breach of contract. See, e.g., Earman v. United States, 114 Fed.Cl. 81, 103-04 (Fed. Ct. Cl. 2013) (provision stating that the contract “shall be carried out in accordance with all applicable Federal statutes and regulations” “does not refer to any particular statutory or regulatory provision,” and therefore “cannot reasonably be read as incorporating the entire corpus of the [relevant] statute into plaintiff’s contract”), aff’d, 589 Fed.Appx. 991 (Fed. Cir. 2015); Smithson v. United States, 847 F.2d 791, 794 (Fed. Cir. 1988). The second sentence that refers to “law, regulation or Department policy” and later to “the same authorities” has the same problem.
The third sentence at least identifies a specific DoD Directive, but it only purports to describe what the Directive says without expressly incorporating it. Under Federal Circuit common law (which we assume applies given this is a government contract, without having the entire contract to confirm) “the language used in a contract to incorporate extrinsic material by reference must explicitly, or at least precisely, identify the written material being incorporated and must clearly communicate that the purpose of the reference is to incorporate the referenced material into the contract (rather than merely to acknowledge that the referenced material is relevant to the contract, e.g., as background law or negotiating history).” Silver State Land LLC v. United States, 148 Fed.Cl. 217 (Fed. Ct. Cl. 2020) (quoting Northrop Grumman Info. Tech., Inc. v. United States, 535 F.3d 1339 (Fed. Cir. 2008)) (emphasis in original). In other words, naming the Directive and describing it may not be enough to allow OpenAI to rely on it as incorporated into the contract at all.
Take all of this together, and the government could argue that there is nothing in this first paragraph that does anything at all to constrain its use of the “AI System.”
The second paragraph (on surveillance) includes more precise legal references, at least identifying specific statutes, though the language also falls back on generalities like “applicable DoD directives” that are subject to the same line of attack. And what limiting effect the referenced authorities have on the government’s ability to conduct “unconstrained monitoring” is unclear, as we are not aware of that phrase being used in any of the authorities referenced in the prior sentence. The final quoted sentence is more straightforward, but again, merely invoking “applicable law” is insufficient to incorporate it into the contract. Moreover, the sentence leaves open the possibility of the government relying on future “applicable law” to justify expanding use of OpenAI’s technology for domestic law enforcement beyond what would currently be permitted.
In the FAQ, OpenAI claims that the contract “explicitly references the surveillance and autonomous weapons laws and policies as they exist today, so that even if those laws or policies change in the future, use of our systems must still remain aligned with the current standards reflected in the agreement.” (Emphasis in the original.) But the quoted contract excerpt contains no explicit language to this effect. With respect to DoD Directive 3000.09, for example, the contract could have said “[t]his contract expressly incorporates DoD Directive 3000.09 as it exists on the date this contract is signed.” Instead, the contract references and purports to summarize the directive without expressly incorporating it at all, much less explicitly incorporating it as it exists at the time of contracting.
Even assuming the quoted text effectively incorporates the extrinsic sources it references (like DoD Directive 3000.09) as they exist at the time of contracting, we’re not convinced that’s an effective way of enforcing the red lines OpenAI claims to be enforcing. That would assume the incorporated sources durably prohibit the uses OpenAI is claiming to prohibit, and it’s unclear that’s the case. Some commentators are arguing it’s not.
OpenAI’s failure to contract for explicit language ensuring its interpretation in the FAQ is problematic, and stating its interpretation in the FAQ does nothing to fix the problem because a court interpreting the contract must look only at the contract’s own language to determine what the parties agreed to. Most sophisticated contracts include a “merger clause,” which is standard language establishing that the written contract represents the entire agreement between the parties. If the contract includes one (and it almost certainly does), a court would not consider OpenAI’s FAQ as evidence of what the contract means. Even without a merger clause, basic principles of contract interpretation generally prevent parties from using external statements to override clear contract language. In short, what OpenAI says the contract means in a blog post carries no legal weight if the contract language itself doesn’t support that reading.
A charitable reading of the situation is that the technical/structural aspects of the deployment are doing most of the work in what OpenAI describes as a “multi-layered approach.” And perhaps OpenAI is relying more on those non-contract layers than the contract layer to make the claims it makes in the blog post. We’d love to hear from technical experts what they think about the descriptions of the deployment and whether it should assuage our concerns. We hope it does, because based on what we know now, we’d be very concerned if the contract described in OpenAI’s blog post were the only thing standing between the government and unfettered use of OpenAI’s systems for mass surveillance of US citizens and independent control of lethal autonomous weapons systems.