Anthropic is still grappling with the economics of compute

We have been told for three years that scaling is the law of the land. The theory was simple: build bigger, spend more on chips, and eventually, the efficiency of these models would outpace the cost to run them. The reality hitting the pavement right now suggests otherwise. Anthropic, the supposed silver medalist in the LLM race, is currently staring at a balance sheet that doesn't make sense for a software company.

New data regarding Anthropic's internal economics shows that their compute margins are actually getting worse, not better. In a typical tech lifecycle, as you scale and optimize, your unit costs drop. In the world of frontier AI, the opposite is happening. Every time Claude gets smarter, the cost to keep the lights on spikes faster than the revenue generated by its users. This isn't just a business problem for Dario Amodei; it’s a warning sign for every founder building on top of these APIs.

The Margin Trap

For those of us building products, we usually think of software as having 80% or 90% gross margins. You build the code once, and then you sell it a million times. The marginal cost of adding one more user is effectively zero. AI flips this model on its head. Every time a user asks Claude a question, a massive cluster of H100s has to spin up to generate those tokens. That costs real money in electricity, hardware depreciation, and cloud rent.

Anthropic is caught in a cycle where they have to keep pushing the frontier to stay relevant against OpenAI and Google. But each new model generation—Claude 3 to 3.5 and beyond—requires an exponential leap in compute. The efficiency gains they find in the software aren't enough to offset the sheer physical cost of the hardware required. When your margins are shrinking during your fastest growth phase, you aren't building a software company; you're building a high-end utility with massive overhead.

Why Scaling Laws Are Failing the Accountants

The technical term for this is the "compute wall." We know that more data and more compute generally lead to better performance. But we are reaching a point of diminishing returns on the dollar. If it costs ten times more to make a model only 10% better, the economics eventually break. Anthropic’s current struggle suggests that the industry hasn't found the "magic bridge" to sustainable profitability yet.

This is particularly dangerous because Anthropic doesn't have the diversified revenue streams that Microsoft or Google enjoy. They are a pure-play AI company. If the cost to serve a query stays high while competition forces prices down, the gap is filled with venture capital and cloud credits from Amazon and Google. It is a circular economy where the money goes from the investor to the startup, then immediately back to the investor's cloud division. It looks like growth, but it feels like a subsidy.

What This Means for Builders

If you are a founder building an AI-first startup, you need to be watching these numbers closely. Here is the blunt reality: you cannot rely on the cost of intelligence dropping forever. We have spent the last eighteen months assuming that GPT-4 level intelligence would be free by 2025. If Anthropic can't fix their margins, they will eventually have to raise prices or limit usage.

• Stop competing on model cost: If the providers themselves are struggling with margins, your business model shouldn't rely on being the "cheapest" wrapper.
• Optimize for token efficiency now: Don't wait for the providers to make it cheaper. Use prompt caching, smaller models for simple tasks, and only call the heavy hitters when necessary.
• The "Moat" is not the API: If Anthropic is struggling, your reliance on their API is a risk. You need to own the data or the distribution, because the underlying compute is currently a volatile commodity.

The Infrastructure vs. Application Divide

There is a growing divide in the ecosystem. On one side, you have the "God Model" builders like Anthropic who are essentially gambling that they can reach AGI before they run out of cash. On the other side, you have the application layer. The application layer assumes that the God Models will always be available and getting cheaper. Anthropic's margin squeeze proves that the second assumption is a dangerous one.

We are seeing a shift where the "smartest" model might not be the most successful one. A model that is 90% as good as Claude but costs 10% as much to run is a far better business partner for a developer. Anthropic's obsession with the frontier is noble, but it is creating a massive overhead that they are passing down—or eating at a loss.

The dream of infinite, cheap intelligence is hitting a physical wall made of silicon and power lines.

The Founder’s Perspective

I’ve talked to dozens of founders who are terrified of their API bills. Seeing a company as well-funded as Anthropic struggle with these same economics should be a reality check. We are currently in the "subsidized ride-share" era of AI. Just like Uber was cheap because VCs paid for your ride, Claude and GPT-4 are currently cheaper than they probably should be because investors are subsidizing the compute. Eventually, the bill comes due.

For a founder, the takeaway is clear: build for a world where compute stays expensive. If your margins only work if OpenAI or Anthropic cuts their prices by 50% next year, you don't have a business; you have a prayer. Focus on vertical integration. Look at how you can use fine-tuned smaller models (like Llama 3) to handle the bulk of your work. Relying on the giants is becoming a high-beta bet.

Final Thoughts

Anthropic isn't going away—Amazon won't let them. But the dream of them becoming a high-margin software powerhouse is fading in favor of them becoming an essential, but low-margin, infrastructure provider. As builders, we need to stop looking at these companies as miracle workers and start looking at them as utility companies. And right now, the cost of the utility is going up, not down. Adjust your roadmaps accordingly.

Read the original at TechCrunch AI →