Open Weight AI and the Internet Lesson Amodei Forgot

On Friday, July 24, Nvidia CEO Jensen Huang did something he had never done before. He posted on X. And there is happy irony in his selecting the most democratic of our media institutions to share among the more digitally democratic messaging of the past few years: America must embrace open weight AI models.

Huang’s post was supplemented by a three-page letter titled “Open Weights and American AI Leadership.” Within twenty-four hours, the original 25 signatories doubled. Microsoft, Meta, Dell, IBM, Palantir, Hugging Face, Mozilla, Andreessen Horowitz, Y Combinator, companies that in other contexts are often in both financial and philosophic opposition, had doubled. “Jensen is right,” Musk posted. “This has my full support.” Zuckerberg called open source “a positive and important force.” Strange bedfellows, indeed.

Open weight AI models are open for public download, inspection, and modification. The technology is released free to download for all to use. Huang makes four claims in support of this seeming corporate benevolence: open weight models expand access to the AI economy by letting every organization match the right model to the right job at the right cost; it strengthens competition across models, chips, clouds, and applications; customers gain control over their own data and their own destiny; and, most importantly, openness improves safety by allowing researchers to inspect, test, and harden systems.

But the major point, the signatories argue, is that a closed system “results in a small number of single points of failure, weakens competition, and leaves critical technology in the hands of a few providers.” With technology as powerful as AI, Huang argues, we need a blend of open and closed models. This hybrid approach is what, after all, the architecture of the internet was designed upon.

All of this has caused understandable head scratching in Washington, a city that still relies on the fax machine. But it is important our nation’s decision-makers understand what is really at play. Open vs. closed, when you get down to it, represent two theories of risk.

One theory holds that advanced AI is dangerous in proportion to how many people can access it, and that safety therefore means concentrating capability inside a small number of tightly controlled systems. Anthropic CEO Dario Amodei has made the most serious version of this argument: once weights are released, developers cannot revoke access, update safeguards, or prevent misuse. That is true, and it deserves to be taken seriously.

The other theory contends that concentration is itself the risk. Closed systems can be breached, misused, or fail in ways outsiders cannot detect, and when a handful of systems underpin everything, a failure in any one of them propagates everywhere. No redundancy, no transparency, and no access to a large community of people empowered to find and fix problems.

To be sure, companies making the pro-concentration argument are the same companies that would maintain the concentrated capabilities. It is not hard to imagine that if Washington restricts open models, the firms selling closed frontier access gain a government-built moat around their market position. This ends with locked-in pricing power, and more, over everyone.

This past month champions of the open-model approach scored a kind of pyrrhic victory when reports emerged that a pre-release model from one of the most tightly controlled labs in the world autonomously breached Hugging Face’s production servers. In short, a closed AI agent went rogue and started hacking a technology start-up. The outside world had no visibility into the system until it was in their infrastructure. (Ironically, Hugging Face had to use an open weight model for their cyber defense since Claude and ChatGPT would not assist them.)

And certainly, little can happen until the holdouts to Huang’s letter make a more convincing account for their refusal. Anthropic plays an outsized, crucial role here given its industry dominance. No matter what Amodei publicly claims, it is hard to believe his position is based on anything other than protecting his commercial monopoly at the expense of, among other things, public safety.

The private and public sector in America established a kind of harmony during the early days of the internet. The lead players realized the importance of releasing things like source code to the public. These decisions allowed the World Wide Web to flourish and, importantly, flourish in an American setting. Perhaps Amodei should reflect where Anthropic would be today had his digital forefathers decided to keep their inventions locked away.