Research · November 17, 2026

Near perfect in range, zero past it: the long-horizon collapse

Policies trained on synthetic problems needing 1 to 4 calculator calls solved 99-100% of them and 0-0.4% of problems needing 8. The stored state stayed exact and the needed facts decodable; the decision readout failed.

Mendrika Ramarlina · November 17, 2026 · 9 min read

Two small policies trained on synthetic word problems needing 1 to 4 calculator calls solved 99-100% of those problems and 0-0.4% of problems needing 8.

The same pattern showed up in every setup we tried over four days. As far as we can tell, the stored state is exact and the information a decision needs is still decodable; the function that reads that state and picks the next action is what fails.

In Cortex, a frozen 2B-parameter open language model encodes text, a small recurrent latent policy does the reasoning over a memory of facts and results, and exact tools (a calculator here) do exact computation. This is research at Saina, not a product. The experiments are small and run on synthetic and grade-school math problems.

Part 2 covered why exact values do not survive a purely latent path. This part starts where the calculator came in.

Codes that are not operations

Before the calculator, we let the model discover its own operations: a small discrete code per step, applied to two table quantities, with no language.

The codes were necessary (without them, one-step accuracy fell to 12.5%) but they were not operations. Code 0 added on 63% of operand pairs and subtracted on 32%, depending on digit features. A code taken from one state and applied in another computed the same operation only 49% of the time for add and subtract. Four ways of choosing codes ended the same way.

Our read is that nothing in the objective forces a code to mean the same thing in two different states. Telling the model, for some training examples, which transitions in different states were the same operation (without naming it) fixed it: transfer went from 49% to 99.9-100% (one seed). Too few such examples did nothing.

Exact computation was a separate bottleneck. Multiplication with two partial products was right 13% of the time, and 40% after five times more training. A latent rollout stayed within 2-5 points of one that wrote digits out and read them back, and both beat a direct answer (38% against 3% at 4 dependent steps), but single steps were not exact. So the next experiment handed arithmetic to a calculator.

Choosing calls toward a goal

There, the policy chooses the calls. Every quantity has a type, a hidden rule table says which typed pair and operation produce which type, and the goal is a type. The policy scores every call(op, i, j) over table slots plus answer(k).

The goal-blind reference follows the rules without looking at the goal. Deeper test problems carry the same distraction as training. On three seeds, trained on dependency depth 1 to 4:

Depth Policy Policy, own repeated calls masked Same-size text tool-caller Goal-blind reference
1-4 99.6% 97-98% 52%
8 85.2% 87.0% about 33% 25%
12 55.2% 72.7% about 1% 13%
20 6.1% 36.9% 0% 6%
Success by dependency depth, trained on depth 1 to 4 Line chart. The policy solves 99.6 percent at depth 1 to 4, 85.2 percent at depth 8, 55.2 percent at depth 12 and 6.1 percent at depth 20. With its own repeated calls masked it solves 87.0, 72.7 and 36.9 percent at depth 8, 12 and 20. 0% 50% 100% trainedhere 6.1%36.9% 99.6% 1 to 4 8 12 20 dependency depth of the problem policy, solvedown repeated calls masked
Trained on dependency depth 1 to 4, three seeds. The dashed line masks the policy's own repeated calls at inference.

At depth 20, 69% of wrong choices repeated a call whose result was already in the table; masking its own past calls at inference recovered a large part. Wider tables in training made depth worse, and eight layers did no better than four (one seed each). The driver was the number of computed results in the table, which grows with the horizon itself.

The text tool-caller, a same-size transformer trained from scratch that writes calls as tokens, was 92-99% right per step once the answer was within about four calls (its training range) and about 50% further out (one seed). It says nothing about pretrained language models.

Reading real word problems

Next the types came from language. The frozen model reads a grade-school math question once, each number mention gets a slot built from its states, and an 8.18M-parameter policy picks calculator calls. Arm A keeps results as fixed records with explicit operand links; arm B learns the result update (10.54M). Two question-blind arms saw only values, or values plus position.

Within 1 to 4 steps, A and B solved 82.7% and 83.7%, against 23.0% and 25.6% for the blind arms. Swapping the language part of two numbers roughly halved success, and the first call followed the language, not the value: grounding is causal.

Per-step accuracy on reference states stopped at 86.5-87.2%, short of the 95% we wanted. We first read the errors as reading errors; classifying every wrong call gave 46% grounding, 52% planning. Implicit constants (×100 for a percent, 60, 12, 365) were the weakest case, at 68.8%. Past 4 steps, 76-78% of wrong calls were planning: answering early, or taking a step whose inputs were not computed yet. On problems with 8 or more steps, success was 8.5% (A) and 5.4% (B).

The collapse follows computed results

Training-free diagnostics on A and B, two seeds and then a third that matched: holding the remaining depth fixed, accuracy on clean reference states fell 50-60 points from 0-2 computed results to 7 or more. A logistic fit gave −0.24 logit per computed result and +0.02 for remaining depth (conditional on total depth, itself −0.29 per level).

In failed episodes, the first decision that made success impossible was a premature answer 77-81% of the time. An oracle that blocked answering recovered 6-13 points; the policy then mostly ran out of calls. Structured and learned workspaces behaved the same.

Hiding irrelevant results with an oracle cut A's premature answers at 8 or more results from 69% to 13%, but its call accuracy fell to 10%. Interference explained part of the collapse, not most.

A text baseline with the same parameter budget

The fair comparison spends about 10M trainable parameters inside the frozen language model instead, writing calls as text with the same calculator and supervision.

Arm Trainable 1-4 steps 8+ steps
Cortex A 8.18M 82.7% 8.5%
Cortex B 10.54M 83.7% 5.4%
Text adapters 10.50M 91.5% 27.5%
Text LoRA 10.99M 91.3% 26.0%

Text is ahead, and the gap to arm A grows with depth: 1.7× at 5 steps, 3.2× at 8 or more. Against an earlier, larger text tool-calling adapter, Cortex was about 10× faster at batch size 1 (95 ms against 998 ms per problem) and 2-2.6× at batch size 64; we have no parameter-matched timing. Our hypothesis for the depth gap: the text arm re-reads its whole execution history through the language model at every step, while Cortex reads the question once. One seed per text arm.

Long chains in training move the threshold

Missing training states or large tables? In the original training data the two always went together, so we trained two variants of A. One inserted extra off-path calls into short problems, for more large-table states. The other included some problems with 5 to 7 steps.

The first saw more large tables and collapsed almost like A. The second kept 55-65% of decisions on a reference path at 7 to 10+ computed results, against 7-24% for A. On 8+ step problems, which neither trained on, success went from 6.2-8.5% to 19.8%. In range it cost 1.5-3.6 points.

On one seed, this points to missing long dependency chains, not table size, and 19.8% is still low.

Not the state, not error build-up

The fourth experiment separated three explanations: the policy's rule does not generalize, the state degrades, or small errors compound.

On clean reference paths, with no earlier mistakes, A's next-action accuracy fell from 74.3% with 0 results to 21.6% at 7-8 and 11.8% at 9-12. Premature answers rose to 48% and 72%. Rebuilding the table from scratch gave bit-identical action scores, so no hidden state drifts. One harmless forced error cost 5-7 points, small next to the clean-path collapse.

Then we trained two fresh policies on synthetic word problems needing 1 to 4 calls, one reading sentences and one reading the same program as an explicit dependency graph. Both solved 100% at 1 and 2 calls and 99.2% at 4, then 1.2-4.4% at 6 and 0-0.4% at 8. An explicit graph did not give length generalization.

The readout fails, not the memory

In that experiment, linear probes fit on short problems read "is this result still needed?" at chance past 9 results. On a fixed benchmark of long problems, probes fit on long problems did not lose it. At 9-12 computed results they decoded "needed" at 79-88% in every model, while correct next actions were 0-6% (63% for the long-chain variant). The policy's internal attention was equally diffuse at every length, in models that worked and models that did not.

Irrelevant results hurt in proportion to their number: 4 irrelevant calls cost 8.7-20.9 points on the target computation, while relevant history raised it. The sentence and graph policies collapsed at the same point, 4-5 completed calls. Far past the training range, no single one-step oracle recovered more than 5 points for the synthetic policies, because their call scores no longer favoured any valid call.

The stored state is exact and the needed information is there, at least for probes with one hidden layer. The function that turns it into a decision does not generalize over execution length.

Verifiers and value models do not escape it

Could a separate goal verifier, or a value model reranking calls, do better? Both were trained on short problems only. A 66k-parameter verifier head on the policy's state reached AUROC 0.9991 in range on synthetic problems and 0.574 at 6-8 computed results, missing 72.8% of finished states. A fresh 8.24M verifier failed the other way: on synthetic problems its false "done" rate went 5%, 56%, 85%, 96% as results accumulated. As a gate it added 0.8 points on synthetic problems (all of it at 6 calls) and cost 2.8 on the math problems.

The value model was worse. On the first state of in-range synthetic problems, the policy's top call was acceptable 100% of the time and the value model's top candidate 0.5%. Its labels were the policy's own success with two spare actions; with that slack 7.4 of 8 candidate calls succeed, so labels barely separate calls. On synthetic problems needing 12 or more calls its AUROC was 0.04-0.05. Reranking cost 33.5 points and beam search 25.6 against greedy on synthetic problems, at 15-46× the compute.

Horizon overfitting hits verifiers and value models too. On long synthetic problems the policy's top 8 calls rarely contain a correct one, so selection cannot help; on the math problems a value model with a better target might.

Part 4 picks up from here, with changes to what the policy sees.

Diagnosed, not fixed

The readout failure is diagnosed, not fixed. We could not tell whether the scorer fails because it only saw short-table encodings or because of its bilinear form. A verifier that sees each result only through its own operand tree, and a value target that actually ranks calls, were proposed and not run in this stretch.

Text is still ahead. Parameter-matched text arms beat Cortex at every depth tested, and its speed advantage was measured only against a larger text setup. Long multiplication in a learned executor was never made exact. The calculator sidesteps it.

The long-chain training result, the parameter-matched text arms and the equivalence-supervision fix for operation codes are one seed each.

Next in Cortex research notes
Removing depth and position as failure modes