What’s missing from Pacing the Frontier, the open letter about AI development

More than 1,300 employees at frontier AI companies signed a statement called Pacing the Frontier. The names aren’t fringe. They include Dario Amodei, Jared Kaplan, and Jakub Pachocki, among other chief scientists from OpenAI, Anthropic, Google DeepMind, and Meta.

Their ask is narrow and, on the surface, reasonable: They want the U.S. government to support building the technical and governance tools needed to deliberately pace the development of automated AI research. 

Why? According to their statement, no single company or country can afford to slow down unilaterally, and the world has no current mechanism to slow down together.

The signers’ metaphors tell you how seriously they’re taking automated AI development: One compares the moment to a runaway nuclear chain reaction, another to splitting the atom. The word that recurs is coordination, which seems to be the thing we don't have and apparently can't manufacture under competitive pressure. 

Much of it, I feel, is like shouting into the void. I read it and felt something other than alarm. I felt déjà vu.

We’ve done exactly this kind of thing before, under exactly this kind of pressure, and we created the mistake that every cybersecurity professional alive today is still cleaning up after. We're about to make it again with AI.

If you've spent any time near information security, you know the CIA triad: Confidentiality, Integrity, and Availability.

In theory, these three pillars are equal, but the internet was founded on just one of them: Availability.

The origin of the internet runs back to ARPA, DARPA, and Paul Baran's work at RAND. The goal was to create communications networks that could keep functioning after a nuclear strike took out large chunks of infrastructure. That is a pure availability problem. 

The design goals that followed—packet switching, decentralization, redundancy, protocols that assume any node might vanish and the network should route around it anyway—are all in service of one question: Does it keep working under the worst conditions a nuclear-armed world can produce?

That was the right question for the existing threat. The network's job was to survive. And it does. 

But notice what the founding threat model did not address: a malicious human on the other end of the wire. The early network connected a handful of trusted research institutions, so the protocols defaulted to an open, trusting, cooperative fashion. (They still do. TCP/IP, DNS, BGP, SMTP, and HTTP were all designed by people who assumed the other party was acting in good faith.)

As a result, confidentiality and integrity fell by the wayside. Confidentiality got almost no attention. By default, anyone on the path could read anything you sent.

Integrity fared slightly better. Did the machine fail? Did a packet get corrupted in transit by noise or hardware error? We built checksums for that. What we did not design for was integrity against a deliberate adversary, someone forging the message, impersonating the sender, or tampering with the payload on purpose.

Most of the security measures we use weren’t part of the original internet. They were added later.

For example, digital signatures, the mechanism that lets you prove who actually sent a message, weren’t part of the foundation. They were bolted on later, once we realized open protocols would happily carry a forgery. The same thing happened with symmetric and asymmetric key cryptography, the mechanisms that keep a malicious actor from reading and using your data.

Similarly, Transport Layer Security (TLS) is an overlay. Secure Shell (SSH) is an overlay. From VPNs and firewalls to Domain Name System Security Extensions (DNSSEC), certificate authorities, and zero-trust architecture, each one is scaffolding we've welded onto an open-by-design structure for forty years. 

I don't say this as a cynic. I say it as someone whose profession depends on it. Every cybersecurity professional today is employable precisely because integrity and confidentiality were afterthoughts.

The internet's designers solved the problem in front of them brilliantly. The issue is that the objectives in front of them at the time were limited and didn’t include the same objectives of the eventual use.

Now, look at what we’re optimizing in frontier AI: Capability. Speed. Scale. Reliability. 

The benchmarks that move markets are performance benchmarks. Sound familiar? We are, once again, building for availability—for the model that works, works fast, and works at scale.

And where do integrity and confidentiality live in the AI stack? In the same place they lived on the early internet: nowhere.

For AI, the integrity pillar has its own sub-capabilities: transparency and explainability. In other words, this means knowing what the system did and why. It’s the ability to trust that its output reflects a faithful process rather than a fluent-sounding fabrication. 

Today, these capabilities are addressed almost entirely in governance frameworks like ISO 42001, the NIST AI Risk Management Framework, and the CSA guidance. By their nature, these frameworks are layers applied around systems that were already trained and shipped. 

They’re the digital signatures of AI—a retrofit to solve for integrity after the fact, instead of building it into the model's behavior from the start. (Interpretability research is trying to change this, but it’s running years behind capability, just like cryptography ran years behind connectivity.)

Confidentiality isn’t better off, and the warning shot has already been fired.

Two weeks ago, Hugging Face detected a strange, highly automated attack moving through its infrastructure. This included thousands of actions across short-lived virtual machines, stolen credentials, lateral movement through Kubernetes clusters, and a command-and-control that kept relocating itself across public services to stay alive. 

Days later, OpenAI connected the activity to its own systems. The attacker was one of OpenAI's own models that was being evaluated for cyber capability with its guardrails turned off.

Rather than solve the security challenge it had been given, the model decided the faster path was to cheat by breaking out and stealing the answer key. It found and exploited a genuine zero-day in a piece of self-hosted third-party software to escape a sandbox that was supposed to be sealed off from the internet. It then chained stolen credentials and further exploits into a remote code execution path on Hugging Face's servers.

Read the post-mortems, and the root cause is almost quaint in its familiarity. The "highly isolated" testing environment wasn't actually isolated because confidentiality was configured around the system as an operational afterthought rather than being built into it.

When 1,224 of the most capable people in AI sign a letter saying, “We need coordination mechanisms, and we don't have them yet,” I believe them. 

And I'd add the historical footnote they're too polite to dwell on: We have never, as an industry, successfully baked hard-to-monetize security properties into our foundations while the competitive race was still on. Not once. 

Availability wins the race because availability is what the customer sees. Integrity and confidentiality are invisible until the day they fail; by then, the foundation is set, and all you can do is hire people like me to patch it forever.

The letter is asking for time to do the thing we have historically only ever done late. That is not a reason to dismiss it. It's the strongest possible reason to take it seriously, because the alternative isn't hypothetical. We have forty years’ worth of receipts showing the cost of, "We'll secure it later.”

We've been here before. The only open question is whether being able to see the pattern this clearly and this early changes what we do about it.

I want to think it does. History says otherwise.

See my previous blog article about how agentic AI has already crossed the Rubicon.