Stop counting tokens, start counting what ships

We have spent the last two years obsessed with the cost of a thousand tokens. We track the price drops of GPT-4o, Claude 3.5 Sonnet, and Llama 3 like they are commodity oil futures. For a while, that made sense. If you were building a basic wrapper or a simple chatbot, your margins were directly tied to that unit cost.

The End of the Linear Prompt

But the game has changed. We are moving away from simple request-response loops and toward autonomous agents. If you are a developer building an agentic workflow—something that actually writes, tests, and deploys code—you have probably noticed that the math no longer adds up the way it used to. When you give an AI agent a coding task, it doesn't just give you an answer. It builds a plan. It generates a draft. It runs a test suite. It sees the errors, loops back, and tries again.

A single task that used to look like one prompt is now a recursive cycle. Data is starting to show that a single useful output from an agent can consume 10 to 50 times the tokens of a standard chat interaction. We are entering the era of the high-compute output. If you are still counting tokens as your primary success metric, you are essentially trying to measure the efficiency of a factory by how much electricity it uses rather than how many cars it ships.

The Inference Trap for Founders

For those of us in the founder seat, this presents a massive architectural challenge. In the old world, a user query was a predictable expense. In the agentic world, a user query is a variable expense that could spiral if your agent gets stuck in a logic loop or tries to solve a particularly thorny bug. This is what I call the inference trap. The more capable and autonomous you make your agent, the more expensive it becomes to run, and not in a linear way.

I’ve seen builders get paralyzed by this. They spend weeks optimizing prompts to save a few cents on token costs while their actual product remains unreliable. We need to shift the focus. The cost of the token is becoming secondary to the value of the completed task. If an agent successfully automates a task that would have taken a senior developer three hours, does it really matter if it cost $5 in tokens instead of $0.50? Probably not. But if it costs $5 and still fails, you have a fundamental engineering problem, not a pricing problem.

Measuring What Matters: The Ship Rate

We need to start counting what ships. In the context of AI-driven development, that means looking at the success rate of autonomous tasks. If your agent is burning 50,000 tokens to fix a bug, the metric that matters is whether that bug actually stayed fixed in production. We are seeing a shift toward 'compute-over-time' as the real cost of doing business.

• Task Completion Velocity: How fast can the agent go from a Jira ticket to a PR?
• Loop Efficiency: How many recursive attempts does it take to reach a passing test state?
• Self-Correction Accuracy: Does the agent learn from the test failure, or does it repeat the same mistake?

If you focus on these, the token costs eventually become an optimization problem for later. Right now, we are in the efficacy phase. If you can’t get the agent to ship reliable code, the cost of the tokens is irrelevant because the product has no value.

The Skeptical View on Agentic Hype

Now, let's be honest. A lot of the current 'agent' craze is just marketing fluff for complex scripts. There is a lot of talk about agents that can do everything, but when you look under the hood, they are often just burning tokens in a circle without making progress. This is where the founder-perspective needs to kick in. You have to be skeptical of your own logs.

Just because an agent is 'thinking' (consuming tokens) doesn't mean it is working. We are seeing a lot of 'agentic drift' where the model gets lost in the context window. It starts off strong, but by the tenth recursive loop, it has forgotten the original constraints of the task. As a builder, your job isn't to give the agent more tokens to play with; it's to build better guardrails and state management so it doesn't have to burn those tokens in the first place.

Why Token Subsidies are a Distraction

We are currently living in a period of subsidized intelligence. The big labs are fighting a price war, and we are the beneficiaries. But relying on token prices to keep dropping is a lazy strategy. Eventually, these companies will need to find margins, or the sheer volume of agentic traffic will force a rethink of how they charge.

Smart builders are already looking at small, specialized models (SLMs) to handle the repetitive parts of the agentic loop. Use the big, expensive models for the initial reasoning and planning, but use a local or smaller fine-tuned model for the constant syntax checking and linting. This is how you scale an agentic product without going broke.

What This Means for the Next Six Months

If you are building in AI or crypto right now, the signal is clear: move up the stack. Stop competing on who has the cheapest wrapper. Start competing on who has the most reliable autonomous outcomes. The market is going to stop caring about how many tokens your LLM can handle and start caring about whether the job actually got done.

The most expensive token is the one that leads to a failed task. The cheapest token is the one that results in a shipped feature.

We have to stop treating AI like a search engine and start treating it like a specialized employee. You don't measure an employee's value by how many words they speak in a meeting; you measure it by what they deliver at the end of the sprint. It’s time we applied that same logic to our code.

Final Takeaway for Builders

Don't let the fear of high token usage stop you from building complex, multi-step agents. However, don't let the novelty of agents mask poor engineering. Focus on the success rate of the final output. If you can ship value, the market will find a way to pay for the compute. The winners won't be the ones with the lowest API bill; they'll be the ones whose agents actually work when the human isn't looking.

Read the original at MIT Technology Review →