Open models have real benefits. Do they outweigh the risks?

We have entered a weird era of defensive research. Anthropic recently dropped a report that feels more like a security briefing than a technical paper. They are worried about GLM-5.3, an open-source model, and they are claiming it lowers the barrier for sophisticated cyberattacks to a dangerous degree. As someone who builds in this space, I see this as a pivotal moment for the industry. It is no longer just about who has the best LLM; it is about who gets to decide who can access them.

The Argument Against Open Weights

The core of the argument against open-weights models like GLM-5.3 is the lack of a kill switch. When you use a closed API from a company like OpenAI or Anthropic, they have multiple layers of safety filters. If you ask a model to help you find an exploit in a critical piece of infrastructure, the server-side monitoring is supposed to catch you and shut the conversation down. With an open model, that safety layer is whatever the developer baked into the weights, and those weights can be fine-tuned away by anyone with a few GPUs.

Anthropic is signaling that we are reaching a point where these models are too capable to be left out in the wild without a leash. They are pointing to specific capabilities in coding and vulnerability discovery that could, in theory, let a script kiddie do the work of a nation-state actor. It sounds terrifying on paper, but we have to look at the motivation behind these warnings.

Building in the Shadow of Giants

For founders and builders, open models are the lifeblood of innovation. If you are building a specialized tool, you cannot rely on a black-box API that might change its pricing or terms of service overnight. You need to own your stack. Open models allow us to iterate, keep data private, and avoid the platform risk that killed so many startups in the early days of social media.

The push to label open-source AI as a security threat feels, to many of us, like a sophisticated form of regulatory capture. If the industry agrees that powerful models are too dangerous to be open, then the only people allowed to build them are the few companies with multi-billion dollar compute budgets. That creates a permanent oligarchy in the AI space. We have to ask: are these models actually dangerous, or are they just a threat to the market share of the incumbents?

The Reality of Cyber Risk

Let's talk about the actual risk. Yes, an LLM can help write code. Yes, it can help find bugs. But so can a thousand other tools that have existed for decades. Hackers have been using automation and specialized scripts long before GPT was a thing. The leap from "this model can help me write a script" to "this model just took down the power grid" is a massive one that hasn't been proven yet.

The real danger isn't necessarily the model itself, but the lack of defense on the other side. If our infrastructure is so brittle that a refined open-source LLM can break it, the problem isn't the AI—it's the infrastructure. We should be using these same open models to patch vulnerabilities and build better defenses, rather than trying to ban the math that makes them work.

The Founder's Dilemma

If you are a founder right now, you are caught in the middle. On one hand, you want the freedom of open weights. On the other, you don't want to be building on technology that might be regulated into oblivion because it's deemed a safety hazard. The narrative is shifting toward a world where "open" means "unsafe."

• Open models provide transparency: You can audit what is happening inside.
• Closed models provide a false sense of security: They are just as capable of being used for harm if the safety filters are bypassed.
• Regulatory pressure is mounting: Governments are listening to these reports, and they aren't tech-literate enough to see the nuance.

We are seeing a repeat of the encryption debates of the 1990s. Back then, the government argued that strong encryption was a weapon and shouldn't be available to the public. Today, we know that without strong encryption, the modern internet wouldn't exist. AI is facing its PGP moment.

The True Cost of Control

What happens if the gatekeepers win? We get a sanitized, expensive, and limited version of the future. Innovation moves to jurisdictions that don't care about these warnings, and we lose the competitive edge. The benefit of open models is that they democratize intelligence. They allow a kid in a garage to build something that rivals what a Google engineer can do. That is a feature, not a bug.

The risks Anthropic pointed out are not zero, but they are often overstated. Every new technology that increases productivity also increases the productivity of bad actors. That was true for the printing press, the steam engine, and the internet. We didn't ban those; we built systems to manage them. We didn't stop printing books because someone might print a manifesto.

Takeaway for the Ecosystem

We need to be skeptical of safety reports that conveniently benefit the companies writing them. The "security" argument is the strongest weapon big tech has to shut down open-source competition. As builders, we have to advocate for the right to own our models. If we allow the narrative to be dominated by fear, we will end up in a world where we don't own our intelligence—we just rent it from a few corporations in San Francisco.

The real risk isn't that models like GLM-5.3 are open. The risk is that we let a handful of companies define what safety looks like for the rest of us. We should be focused on building better shields, not trying to break everyone else's swords.

Read the original at MIT Technology Review →