Inference Is the Most Important Market in Software

For the last two decades, software was a game of storage and access. We built digital filing cabinets, called them databases, and charged people a monthly fee to look inside them. That era is ending. The new software powerhouse isn't where you keep your data, but how you process it. Specifically, it is the inference market.

We are looking at a projected $350 billion market by 2027. To put that in perspective, that is larger than the entire global database market. If you are a builder, you need to stop thinking about your app as a wrapper for a database and start seeing it as a conduit for inference. This shift changes everything about how we build, price, and scale companies.

The Death of the Fixed Margin

In the classic SaaS model, the math was simple. You built the software once, hosted it for cheap, and every new customer was nearly pure profit. Gross margins were consistently in the 80% to 90% range. Infrastructure was a rounding error. That was a comfortable life for founders, but inference is about to set that model on fire.

When you integrate AI, your Cost of Goods Sold (COGS) isn't flat anymore. Every time a user interacts with your product, you are burning compute. You aren't just serving up a static page; you are renting a small slice of a massive GPU cluster to perform complex math in real-time. This means your margins are no longer protected by the efficiency of your code alone; they are tied to the raw cost of compute power.

Infrastructure is the New Product

In this new landscape, infrastructure becomes the dominant component of your costs. We are moving toward a reality where gross margins might compress to 50% or 60%. For a traditional software investor, that looks like a disaster. For a builder, it means you have to be much smarter about your unit economics from day one.

If you build a product that relies on heavy inference without a clear strategy for compute efficiency, you aren't building a software company. You are building a pass-through entity for Nvidia. You are essentially taking money from your customers and handing a majority of it directly to the cloud providers and chip makers.

Why Inference Dominates

Why is this market becoming so much larger than the data storage market? Because storage is passive, but inference is active. A database just sits there. You pay for the space it occupies. Inference, however, is the act of generating value. It is the writing of the email, the coding of the script, the diagnosis of the medical image.

We are moving from a world where software helps us organize our work to a world where software does the work. When the software is the labor, the market size is capped only by the amount of work humans need to get done. That is why the $350 billion estimate feels conservative to me. If inference becomes the engine of global productivity, the scale will be unprecedented.

The Skeptical Take on Subscriptions

Here is the hard truth for founders: the flat-fee subscription model is probably dying. If your costs are variable based on how much a user interacts with your AI, a $20-a-month seat price is a gamble. One heavy user can turn your profitable customer into a net loss overnight.

We are likely headed toward a usage-based or value-based pricing world. Builders who resist this and try to stick to the old SaaS playbooks will find themselves squeezed. You cannot scale a business where your primary cost is a volatile commodity like GPU time unless your pricing reflects that volatility.

What This Means for Builders

If you are starting a company today, your first priority shouldn't just be product-market fit. It has to be architecture-market fit. You need to ask yourself if your product actually requires a massive LLM for every task, or if smaller, specialized models can do the job at a fraction of the cost.

• Efficiency is a feature: In the old days, messy code just meant a slightly slower app. In the inference era, messy code means you go broke. Code efficiency now translates directly to your bottom line.
• Own the workflow, not just the model: Models are becoming commodities. The value is in the proprietary data you feed them and the specific workflow you solve. Don't compete on the quality of the inference itself; compete on how that inference is applied to a specific problem.
• Infrastructure strategy is a core competency: You can't outsource your understanding of the stack. You need to know exactly how much every query costs and where that money is going.

The transition from a database-centric software market to an inference-centric one is the biggest structural shift we have seen since the move to the cloud. It is exciting because it means software can do more than ever before. But it is dangerous because the old rules of profitability no longer apply.

The shift to inference means your margins are no longer protected by your code, but by your ability to manage compute as a raw material.

We are entering a phase where the winners won't just be the ones with the best AI, but the ones who figure out how to build a sustainable business around it. The $350 billion is there for the taking, but it won't be handed to people running 90% margin SaaS companies. It will go to the builders who treat compute like the precious, expensive resource it is.

Read the original at Tomasz Tunguz →