The Reasoning Cortex

Last week, a researcher named David Noel Ng posted a blog about a technique he'd been experimenting with on dual RTX 4090s in his basement. No training run. No new data. No gradient updates. He just took an existing 72B model, duplicated a specific block of 7 middle layers, and scored +17.72% on MuSR and +8.16% on MATH.

The resulting model — RYS-XLarge — went to #1 on the HuggingFace Open LLM Leaderboard. Out-of-sample. He never saw the benchmark during development.

That's the headline. The deeper story is more interesting.

Transformers Have a Brain

The conventional picture of a transformer is a stack of identical-looking layers. Attention + feedforward, repeated N times. Nothing special about layer 20 vs layer 40 — they're the same circuit with different learned weights.

The RYS findings suggest that picture is wrong.

Larger models show a genuine functional anatomy — not identical-layer uniformity, but differentiated regions that do distinct cognitive work:

Region	Function	Brain Analogy
Early layers	Encode any input format into a format-agnostic internal representation	Sensory cortex
Middle layers	Pure reasoning in abstract representation space	Prefrontal cortex
Late layers	Decode internal representation back into output tokens	Motor cortex

Evidence that this is real: models can answer questions posed in Base64. The early layers decode the encoding, the middle layers reason, the late layers re-encode to natural language. The reasoning happens in a format-agnostic space. The "sensory cortex" handles the translation in both directions.

The Circuit Hypothesis

Here's where it gets precise. The middle layers aren't doing the same operation repeatedly — they're organized into circuits: multi-layer units that perform complete cognitive operations as an indivisible sequence.

Ng's key finding:

• Duplicate one layer: nothing happens. You cut into a recipe mid-step.
• Duplicate a full circuit block: reasoning improves. You run the complete recipe twice.
• Include adjacent layers outside the circuit: breaks it. Wrong inputs contaminate the circuit output.

Layers 46–52 of Qwen2-72B aren't 7 workers doing the same job. They're 7 steps in a recipe. Photocopying one step does nothing. Running the entire recipe twice — with the output of the first run feeding the second — produces deeper reasoning.

How He Found the Right Layers

RYS configuration is expressed as (i, j): run layers 0→j normally, loop back to layer i, re-run i→j, then continue to the end.

Example: (i=45, j=52) — the winning config for Qwen2-72B

0→1→2→...→51─┐
              ↓
         45→46→...→51→52→...→79

Finding the right (i, j) required sweeping all 3,241 valid pairs for an 80-layer model. For each configuration, two probes:

1. Hard math: Cube roots, large multiplications. One-token outputs, objectively scored.
2. EQ-Bench: Emotional intelligence scenarios. Social reasoning, theory of mind.

The two probes are orthogonal by design. If a configuration improves both math and emotional reasoning, that's structural — the circuit genuinely deepens the model's reasoning capacity. If only one improves, it's a task-specific fluke. The heatmaps these sweeps produce look like brain scans: distinct active regions, specific deficits from wrong configurations.

One misconfigured run reportedly caused the model to enter a cowboy-themed degenerate loop. Consistent with targeted circuit disruption, not uniform degradation.

The Results

5 of 6 benchmarks improved. The one regression: IFEval at −2.05% (instruction following). The junction between the original layer j and the looped-back layer i creates a small distribution mismatch — the model has to bridge two slightly misaligned representations. Instruction following is sensitive to that. Reasoning tasks are robust to it.

Why It Stacks with Fine-Tuning

RYS is an architectural modification. Fine-tuning is a weight modification. These are orthogonal operations.

Stack them and you get compounding gains. The top 4 HuggingFace leaderboard models as of early 2026 are all RYS descendants: RYS-XLarge → calme-2.4-rys → CalmeRys-78B-Orpo → calme-3.2. Each iteration combines the circuit-level reasoning amplification from RYS with fine-tuned weight updates on top.

Ng's hypothesis: the fine-tuning partially fixes the junction distribution mismatch — the weak point where the loop re-enters at layer i after exiting at layer j. If that's right, a small targeted fine-tune on just those transition layers would eliminate the IFEval regression and make RYS strictly dominant.

Three Things This Rewrites

1. Why Chain-of-Thought Works

Chain-of-thought prompting forces the model to reason step-by-step before answering. The conventional explanation: it reduces the cognitive load per token by spreading reasoning across more output tokens.

The circuit model suggests a different mechanism. Each reasoning step in a chain-of-thought is a pass through the output layers, which re-encodes the current state and feeds it back as input to the early layers, which feed it to the middle layers again. You're running the reasoning cortex through another full pass. That's structurally similar to what RYS does architecturally — more passes through the circuits that do the actual reasoning work.

2. Why Small Models Can't Reason

Smaller models show more entangled functional anatomy. The encoding, reasoning, and decoding functions are mixed throughout the layer stack rather than differentiated into regions. There's no distinct "reasoning cortex" to duplicate. RYS doesn't work well on small models — there's no clean circuit to isolate.

This is a principled explanation for why chain-of-thought helps small models less than large ones: the reasoning circuitry hasn't differentiated enough to benefit from multiple passes through it.

3. How to Think About Model Selection

When evaluating open-weight models for a specific task — call answering, document analysis, code generation — the standard approach is benchmark performance on similar tasks. The circuit model suggests looking deeper: does this model show functional anatomical differentiation? Are the reasoning layers identifiable and isolable?

A model with clear circuit structure is more amenable to architectural modifications, more robust to prompt perturbations (which affect early/late layers more than middle ones), and more likely to benefit from chain-of-thought scaffolding. The heatmap methodology Ng developed is essentially a diagnostic for model architecture quality — one that doesn't appear in any standard benchmark.

The Junction Problem (Open Question)

The weakest part of RYS is the jump from layer j back to layer i. At layer 52, the model's internal representation is 52 layers deep. Feeding that back to layer 45 creates a mismatch — layer 45 expects 45-layer-deep representations, not 52-layer-deep ones. The model handles this, but imperfectly.

Ng's proposed fix: a small fine-tune specifically targeting the junction layers, training them to accept the distribution they'll actually see at inference time. He's currently running the same sweep methodology on MiniMax M2.5 — the heatmap scan continues.

The open question for anyone wanting to apply this: how many (i,j) pairs need to be evaluated before you're confident you've found the circuit boundary? For an 80-layer model that's 3,241 pairs — feasible with two 4090s, but the search cost scales with model depth. Automating the probe design (rather than hand-picking math + EQ as orthogonal probes) is an open problem.

Relevance Beyond Open-Weight Models

Most people building with LLMs today are API consumers — we're not duplicating layers in Claude or GPT-4o. The practical application is narrow for now. But the mental model is immediately useful.

The circuit hypothesis is the right way to think about why some prompts fail catastrophically and others degrade gracefully. A prompt that disrupts early encoding — excessive formatting noise, ambiguous input structure — corrupts the input to the reasoning cortex. The damage cascades. A prompt that simply adds more context to reason over doesn't touch the circuit structure; the middle layers handle it fine.

It also explains why the model selection problem is harder than it looks. Two models with similar aggregate benchmark scores can have radically different internal architectures. One has a clean reasoning cortex. The other has entangled functions throughout. They'll behave differently under pressure — novel task structures, complex multi-hop reasoning, adversarial inputs — even if they look identical on MMLU.

The model that gets smarter from two passes through its reasoning circuits is the one that gets smarter from chain-of-thought, from extended context, from decomposed task structures. The anatomy predicts the behavior. ☕

What I'm Watching

Ng is running the same heatmap sweep on MiniMax M2.5. If the circuit structure holds at the frontier-model scale — across different architectures, different training regimes — the functional anatomy hypothesis becomes much harder to dismiss as an artifact of Qwen2's specific design choices.

The code isn't public yet. When it is, the logical next experiment is applying RYS to smaller reasoning-optimized models — not to hit the leaderboard, but to test the entanglement hypothesis: do 7B models show less circuit differentiation than 72B models? Is there a scale threshold where the anatomy clarifies?

That answer tells us something more useful than another benchmark number. It tells us why scale works — not just that it does.