We’re updating our blog post to respond to this post from Sam Altman that includes additional language he says will be added by amendment to OpenAI’s agreement with DoD.
We’ll quote the portion of his post that includes the purported new language here, for reference, and follow with our thoughts:
1. We are going to amend our deal to add this language, in addition to everything else:
“• Consistent with applicable laws, including the Fourth Amendment to the United States Constitution, National Security Act of 1947, FISA Act of 1978, the AI system shall not be intentionally used for domestic surveillance of U.S. persons and nationals.
• For the avoidance of doubt, the Department understands this limitation to prohibit deliberate tracking, surveillance, or monitoring of U.S. persons or nationals, including through the procurement or use of commercially acquired personal or identifiable information.”
It’s critical to protect the civil liberties of Americans, and there was so much focus on this, that we wanted to make this point especially clear, including around commercially acquired information. Just like everything we do with iterative deployment, we will continue to learn and refine as we go.
I think this is an important change; our team and the DoW team did a great job working on it.
2. The Department also affirmed that our services will not be used by Department of War intelligence agencies (for example, the NSA). Any services to those agencies would require a follow-on modification to our contract.
Our thoughts:
1. We still don’t have the whole contract, and that makes it hard to say anything with certainty.
2. It’s not clear what it means to “add this language, in addition to everything else” given the language overlaps with the quoted language from the blog post. Presumably they are not actually duplicating that language.
3. This is definitely an improvement on the mass surveillance language because the terms at least independently address the extent to which the technology can be used for domestic surveillance of U.S. persons or nationals, rather than just attempting (and failing) to punt to whatever is the law.
4. But there are a few things worth mentioning, and we still have very serious concerns.
First, the references to “applicable laws, including the Fourth Amendment to the United States Constitution, National Security Act of 1947, FISA Act of 1978” continue not to incorporate those laws into the actual terms of the contract. The language does not “clearly communicate that the purpose of the reference is to incorporate the referenced material into the contract (rather than merely to acknowledge that the referenced material is relevant to the contract, e.g., as background law or negotiating history).” Silver State Land LLC v. United States, 148 Fed.Cl. 217 (Fed. Ct. Cl. 2020) (quoting Northrop Grumman Info. Tech., Inc. v. United States, 535 F.3d 1339 (Fed. Cir. 2008)) (emphasis in original).
A naive reading of the language might make one assume that by violating one of the referenced sources of law, the government would be in breach of the contract. But that is not the case. However, the fact that the references fail to incorporate law into the contract is less problematic than the prior language because there is independent wording now at least purporting to prohibit some use for domestic surveillance.
Second, the words “intentional” and “deliberate” leave a lot of wiggle room, especially for incidental collection and analysis. If history is any guide, the government is likely to exploit that wiggle room to allow surveillance most people would assume the language would prohibit.
Third, “U.S persons or nationals” almost certainly does not include people in the US on visas, including work visas. So this language would permit DoD to use the AI system to intentionally surveil, e.g., a person living in San Francisco and working at an AI company on an H-1B visa. And given how “domestic surveillance” is used in the new clause, the language likely permits the use of the AI system for surveillance even of U.S. persons or nationals while they are abroad (including US citizens on vacation).
Finally, the new language does nothing to fix the problems with the previously-released language on lethal autonomous weapons systems.
5. On point 2 regarding the use of OpenAI’s technology by the DoD’s intelligence agencies, is there anything in the contract that actually ties the government’s hands? If so, why not quote that, too? Otherwise, this is putting a lot of trust in an unreliable counterparty.