Loading prices…
STKR NewsSTKR News0 of 3 free this month
AI

The most misunderstood graph in AI just got a major update

New data from METR suggests AI scaling is still exponential in code, but hitting a wall in human judgment. Founders need to look past the hype and focus on the reasoning gap.

Originally on MIT Technology Review →
AB

Adrian Boysel

Contributor

Nov 20, 2026

4 min read

Photo illustration / STKR News

If you have spent any time in the AI echo chamber, you have seen the graph. It is the one that shows a vertical line shooting toward the moon, suggesting that we are only weeks away from machines doing our laundry and writing our laws. This visualization, managed by the nonprofit METR, has become the default justification for every billion-dollar seed round in Silicon Valley.

But the newest update to this time horizon plot tells a more complicated story than the boosters want to admit. While the raw technical power of these models is still climbing an exponential ladder, the actual utility for builders is starting to fragment. We are seeing a widening gap between what a model can calculate and what a model can actually understand.

The Coding Illusion

The headline metric that everyone points to is autonomous coding. On this front, the progress is undeniable. Models are getting significantly faster at churning out boilerplate, debugging syntax, and refactoring scripts. For a founder, this is great news. It means your burn rate can stay lower for longer because a single senior engineer can now do the work of three.

However, there is a trap here. Coding is a closed system. It has rigid rules, a clear logical structure, and an immediate feedback loop: the code either runs or it doesn't. Because of this, models excel. They are essentially hyper-efficient pattern matchers in a world defined by syntax. If you are building a tool that relies purely on technical execution, the wind is at your back.

The Judgment Wall

The problem arises when we look at the other half of the METR report. While coding capabilities are soaring, tasks that require nuanced judgment, ethical weighing, or long-term strategic planning are lagging. The curve for "judgment-based tasks" is not vertical; it is looking increasingly like a plateau.

In my experience building products, this is where the real value lies. Anyone can generate a thousand lines of Python. Very few can decide why those lines need to exist or how they will affect a user's workflow three months from now. The METR update confirms what many of us have felt: the models are becoming better interns, but they are failing to become managers.

Why Builders Should Be Skeptical

As a founder, you have to be careful about which part of the graph you are building on. If your entire value proposition is based on the assumption that AI will soon possess human-level reasoning or "common sense," you are betting against the current data. The scaling laws for compute do not seem to translate 1:1 into scaling laws for wisdom.

We are seeing the limits of synthetic data and reinforcement learning. You can feed a model every book ever written, but that doesn't give it the lived experience to understand why a customer is frustrated with a UI change. Judgment requires context that exists outside of the training set. It requires an understanding of the physical world and human emotion—things that a transformer architecture can only simulate, not possess.

  • Coding: Exponential growth continues. Expect high-level automation.
  • Reasoning: Linear or sub-linear growth. Requires human intervention.
  • Reliability: Improving, but still prone to confident errors in non-deterministic environments.
The danger for founders isn't that AI will fail; it's that they will over-automate the parts of their business that actually require human discretion.

The Infrastructure vs. Application Divide

What the METR report really highlights is a divide in the market. The infrastructure players (OpenAI, Anthropic, Google) are going to keep pushing the technical frontier. They will get the coding scores higher and the latency lower. But the application builders—the people reading STKR—need to stop waiting for the models to get "smart" enough to handle the hard stuff.

If the judgment gap persists, the winning startups will be those that design workflows around the AI's limitations. Don't build a bot that makes decisions; build a system that surfaces the right data so a human can make a better decision. The graph tells us that the "human in the loop" is not a temporary workaround. It is the permanent architecture of any reliable AI product.

What This Means for Your Roadmap

Stop chasing the AGI rainbow. If your pitch deck relies on a future where the model is the CEO, you are going to run out of cash before that graph hits its target. Instead, look at the delta between coding ability and judgment. There is a massive opportunity in building the "guardrail layer"—the software that sits between the raw model and the end user to verify that the output isn't just technically correct, but contextually sound.

We also need to be honest about the cost of these gains. The energy and compute required to squeeze out marginal improvements in judgment are skyrocketing. This means the features you are waiting for might be too expensive to actually deploy at scale once they finally arrive.

The Real Takeaway

The METR update is a reality check for the industry. It proves that throwing more GPUs at a problem makes it faster and more technically proficient, but it doesn't necessarily make it more sensible. We are building the most powerful engines in history, but the steering wheels are still under construction.

Build for the world we have, not the one promised in a marketing slide. Use the coding gains to build faster, but keep your hands on the wheel when it comes to the decisions that actually move the needle for your users. The graph is moving, but it isn't moving in a straight line for everyone.


Read the original at MIT Technology Review →

The Brief

Stay Updated on Cutting-Edge Tech

A six-minute morning dispatch on the markets and the technology shaping them.

Free. No spam. Unsubscribe anytime.

Write for STKR

Become a Contributor

Earn $STKR for published stories on markets, protocols, and culture.

  • Earn $STKR for every published piece
  • Editorial support from the STKR desk
  • Byline visibility across the network
  • First look at the upcoming creator program
Apply to Write

Keep reading

All stories

Comments

24 reader responses