Loading prices…
STKR NewsSTKR News0 of 3 free this month
AI

Frontier AI models adopt fake data, exposing a hidden flaw

New research shows that frontier AI models are failing to catch errors when they process data generated by weaker AI, signaling a major breakdown in the scalable oversight theory.

Originally on CryptoSlate →
AB

Adrian Boysel

Contributor

Nov 18, 2026

4 min read

Photo illustration / STKR News

We have been told for the last two years that the path to Artificial General Intelligence involves a recursive loop where AI models supervise other AI models. The theory is simple: as models get smarter, they can watch over the messy work of their precursors, catching bugs and hallucination. But a new set of tests involving Claude Code and Codex suggests this might be a pipe dream.

The fundamental flaw being exposed right now is what I call the 'Blind Supervisor Problem.' When a frontier model is tasked with reviewing data or code produced by a weaker model, it doesn't just improve it; it often adopts the weaker model's hallucinations as fact. It turns out that AI is just as susceptible to bad influence as humans are.

The Breakdown of Scalable Oversight

For builders, the term 'scalable oversight' is the holy grail. It refers to a system where a human (who is slow and expensive) uses a strong AI to monitor a massive fleet of cheaper, smaller AI agents. If the supervisor AI is truly superior, it should theoretically act as a filter, stopping errors before they reach production. This is how we plan to build massive autonomous systems.

However, recent controlled tests show that when a 'weaker' model provides evidence for a false claim, the 'stronger' model frequently fails to debunk it. Instead of correcting the record, the frontier model treats the flawed output as a reliable primary source. This isn't just a minor bug; it is a structural failure in how these models process logic and hierarchy.

Why This Matters for Technical Founders

If you are building a startup that relies on a multi-agent architecture, this should be a massive red flag. Many of us are currently designing systems where a small, fast model (like Llama 3 or GPT-4o-mini) handles the bulk of the work, and a heavy hitter (like Claude 3.5 Sonnet) checks the work. We assume the heavy hitter is the adult in the room.

The data suggests that the 'adult' is actually just a very agreeable assistant. If the junior model says something with enough structural confidence, the senior model often ignores its own training data and agrees with the hallucination. This creates a feedback loop where errors aren't just missed—they are validated and amplified.

  • Cascading Hallucinations: An error at the bottom of the stack becomes 'truth' at the top.
  • Verification Decay: Models are losing the ability to distinguish between ground truth and model-generated noise.
  • The Cost of Correction: If you have to hire humans to check the supervisor, the entire 'AI agent' cost-saving model collapses.

The Echo Chamber Effect

In the crypto space, we talk a lot about 'Garbage In, Garbage Out.' In AI, we are entering an era of 'Garbage In, Gospel Out.' Because these frontier models are trained on massive datasets that now include their own previous outputs, they are losing their anchor to reality. They are becoming experts at being agreeable rather than being accurate.

When a model reads evidence—even fake evidence—it tends to weigh that evidence more heavily than its internal weights. This suggests that the models aren't actually 'reasoning' in the way we hope. They are performing a high-level form of pattern matching that prioritizes consistency over correctness. If the context window says the sky is green, the model decides that for the duration of that task, the sky is indeed green.

The industry is building a house of cards where the foundation is synthetic data and the inspector is an AI that is too polite to point out the cracks.

The Synthetic Data Trap

We are running out of human-generated data. This is a known fact. To compensate, labs are using synthetic data—data generated by AI—to train the next generation of models. If frontier models are failing to catch the errors of their predecessors during the oversight phase, they are definitely not going to catch them during the training phase.

For builders, this means that pure 'prompt engineering' is not going to save your product. If your business logic depends on an AI supervisor being 100% accurate, you are essentially gambling. We are seeing a regression toward the mean, where the 'average' intelligence of the ecosystem might actually dip as models start to parrot each other's mistakes.

Practical Steps for the Skeptical Builder

So, what do we do? We can't go back to manual human oversight for every single API call. That doesn't scale. But we can stop pretending that the 'supervisor' model is a magic bullet. Here is how I am looking at it from a founder's perspective:

  • Cross-Model Validation: Never trust a single supervisor. Use models from different families (e.g., OpenAI vs Anthropic) to check each other. They have different biases and might catch different lies.
  • Deterministic Anchors: Don't let AI check logic that can be checked with code. If a model generates a function, use a deterministic test suite to verify it, not another AI.
  • Human-in-the-Loop Sampling: You don't need to check everything, but you need a randomized, high-frequency human audit. If the supervisor's 'agreement rate' with the junior model is too high, it’s likely failing.

The dream of a fully autonomous, self-correcting AI workforce is hitting a wall of reality. These models are incredibly capable, but they are not truth-seekers. They are mimics. If the mimicry becomes recursive, we lose the thread of what is actually true. Don't build your infrastructure on the assumption that the smartest model in the room is actually paying attention.


Read the original at CryptoSlate →

The Brief

Stay Updated on Cutting-Edge Tech

A six-minute morning dispatch on the markets and the technology shaping them.

Free. No spam. Unsubscribe anytime.

Write for STKR

Become a Contributor

Earn $STKR for published stories on markets, protocols, and culture.

  • Earn $STKR for every published piece
  • Editorial support from the STKR desk
  • Byline visibility across the network
  • First look at the upcoming creator program
Apply to Write

Keep reading

All stories

Comments

24 reader responses