Lab notes

Lab notes · 01 · July 2026

Sharper on one dataset, reversed on the next

A preregistered study with a negative result. A side network that reads four layers of a frozen model made it measurably more decisive, then failed the three tests that would make it worth scaling.

By Robert Balas, Redcom Software

Status
Stopped by a rule committed before the deciding runs
Backbone
Pythia-160M, frozen: 162,322,944 parameters, never updated
Added
325,250 trainable parameters, 0.2% of the backbone’s size
Method
Preregistered, hash-committed configs, byte-for-byte replays
Closed
Nine lines of inquiry, each closed with a recorded result
Code
Run summaries on request; a public repository is in preparation
Figure 1 · Sections 2.2 and 3.1On the development set the distinct reader sharpens and the control flattens. On a second corpus the order flips. Change in the gap between the model’s top two logits (the margin), one dot per cell: two build seeds by two training objectives. On the development set, at answer positions, the readers were routed through the sieve. On the second corpus, a sample of the model’s own pretraining text, the gate was held fully open, and the control raised the margin more than the distinct reader at every cell.
Figure 2 · Section 3.3Switched on everywhere, it drifts to nearly three times the limit. How far the side network pushes the frozen model’s output (KL divergence, in nats), against the limit set before the runs: when it writes at every position, when the 5% of positions it pushes hardest are switched off, and when a learned gate (the sieve, Section 1) decides where it writes. Routed through the sieve, drift is near zero only because the sieve fires on about 1% of ordinary-text positions. Bars show one cell (seed s179, family-record objective); across all four cells, always-on drift is 0.123–0.144.

Abstract

I asked whether a frozen pretrained language model can be extended by a tiny, reversible side network that reads several of the model’s internal layers and writes one bounded correction at the output. On the corpus I used to develop the method, the answer is yes, with a known mechanism. Reading four distinct depths makes the model measurably more decisive, and an equal-parameter control that reads one depth four times makes it measurably less. The effect replicated across independent random seeds, and it comes from one mechanism: the correction sharpens the output distribution.

It then failed the tests that would make it worth scaling. On the one corpus that played no part in developing the method, the effect reverses. It has no demonstrated effect on generated text. And it is safe to deploy only because a learned gate keeps it switched off on about 99% of ordinary text while opening on 87% to nearly 100% of answer positions. The gate itself mostly works as designed. On answer positions it opens where the write helps, and stays within 0.03 nats of an oracle with the same budget. It cannot be opened wider on ordinary text without breaking the model, for a structural reason I measured.

I stopped the work under a decision rule written and committed before the deciding runs. This note records the result, the method that produced it, and the three occasions on which a later experiment corrected an earlier claim of my own.


1. The question and the design

Large pretrained models are expensive to change and easy to damage. A recurring idea is to leave the pretrained weights entirely frozen and bolt on a small trainable module that reads internal activations and nudges the output. I built a deliberately conservative version of that idea: frozen weights, one bounded write at the output, and matched controls at every step.

The architecture, on a frozen Pythia-160M backbone (162,322,944 parameters, never updated):

Component Trainable parameters Role
Cross-layer reader 271,745 reads hidden states at layers 2, 4, 6, 12
Sieve (a learned router) 53,505 decides, per token position, whether to write at all
Total trainable 325,250 (0.2% of the frozen backbone)

The reader emits one bounded correction, added to the final hidden state just before the frozen language-model head. There are zero writes inside the backbone, and no recurrence: an earlier ambition to loop the network was tested and abandoned (Section 4), and the final configuration records recurrence: false and zero internal write locations.

The central comparison throughout is between two readers of the same size, trained and routed the same way:

  • the distinct reader, reading layers 2/4/6/12;
  • the repeated control, reading layer 12 four times.

Only which depths are read differs. That isolates the value of reading across depths from the value of simply adding parameters.


2. What the method does on its development corpus

All measurements in this section are on a controlled synthetic reasoning corpus of 462 evaluation records, across two freshly seeded model builds (s179, s191), two training objectives and the two readers, giving eight sealed experimental cells, four per reader.

2.1 It changes which token the model emits

Measured as strict-argmax agreement with the gold answer token, the distinct reader raises agreement over the frozen backbone at all four of its cells, by +0.00511 to +0.00718. The repeated control lowers it at all four of its cells, by -0.00400 to -0.00552.

This was the study’s first evidence that the mechanism changes behaviour as well as likelihood. The effect is small, and its sign is opposite for the two equal-parameter readers.

2.2 The mechanism is sharpening

The two readers differ in what they do to the shape of the output distribution. The distinct reader raises the gap between the top-1 and top-2 logits (the margin) and lowers the output entropy at all four of its cells:

distinct reader repeated control
Δ margin +0.01325 to +0.01766 (4/4 up) -0.03257 to -0.02208 (4/4 down)
Δ entropy -0.01162 to -0.01417 (4/4 down) +0.00463 to +0.00929 (4/4 up)

Reading four distinct depths makes a frozen model more decisive; reading one depth four times makes it less. This is the only mechanistic explanation the study produced, and it accounts for the token-level result in 2.1: sharpening the distribution around the correct token is what flips the argmax.

A recorded deviation. This run’s preregistered prediction had three parts, and two held. The third said the distinct reader would create no net new ties at the top of the prediction (two answers scoring exactly level); it created 10 at one cell and 1 at another, each out of 7,247 answer tokens. Read strictly, the preregistration would then retract the run’s finding about the repeated control. I keep that finding, because the failed part measures the distinct reader and does not involve the control. If you prefer the strict reading, treat the control result as withdrawn. The run record logs the deviation and the reason for it.

2.3 It replicated prospectively

The advantage of the distinct reader over the repeated one held in the same direction at three independent fresh build seeds (s127, s179, s191), and it survived a leave-one-record-out check: removing any single record did not change the sign at s179 or s191, the two seeds it was run on.


3. The three tests it failed

A result on one corpus is not yet a design pattern. Three tests decided whether it was worth scaling. It failed all three.

3.1 On a second corpus, the effect reverses

The only second corpus I could clear for use is a sample of the model’s own pretraining text (Pile-deduped). It played no part in choosing the architecture, the initialisation, the width or the objective.

A first attempt found the deployed mechanism inert there, but that comparison was confounded: the sieve is switched off on almost all pretraining text, so a null was partly guaranteed. I removed the confound by holding the gate fully open, which measures the reader itself instead of the gated system.

With the gate open, on pretraining text, the repeated control sharpens (Δ margin +0.00287 to +0.00670, up at all four cells), and it sharpens more than the distinct reader at every cell (Δ margin −0.00017 to +0.00346). On the development corpus that same control flattens. On this corpus the effect reverses sign. The advantage of reading across depths is specific to the corpus I used to choose the method.

3.2 No effect on generated text was shown

Whether sharpening changes what the model actually generates was never demonstrated. On the development corpus full-answer generation cannot register an effect: the frozen model assigns the whole gold answer a joint probability above 0.1 in 0 of 462 records, so greedy generation of a correct answer is essentially impossible regardless of the correction.

I also tried a measure on shorter outputs: does the model generate the correct first one, two or three tokens? An early version carried an instrument defect (a padding convention incompatible with the model’s positional encoding). I repaired it by removing padding entirely and re-ran: base and corrected prefix-decode accuracy are both 0.000000 at all eight cells and all three lengths. With the defect removed the result was the same, which agrees with the joint-probability fact above.

3.3 It cannot be applied more often

The correction is safe only because it is rarely applied to ordinary text. With the sieve disabled, the correction drives the output distribution 0.123 to 0.144 nats of KL away from the frozen model across the distinct reader’s four cells. The preregistered safety limit is 0.05. Routed through the sieve, that falls to near zero, but only because the sieve fires on about 1% of ordinary-text positions.

The cap is structural. The drift is diffuse: the worst 1% of positions carry only 8.2% to 10.0% of the total divergence, and the sieve already excludes 100% of them. Suppressing the worst 5% of positions still leaves KL at 0.092 to 0.110, about twice the limit. There is no concentration of harm for a better gate to remove.


4. What else was closed

The sharpening result was the surviving thread of a wider search. Nine lines of inquiry were closed, each with a recorded terminal result:

  1. Recurrence. Looping the reader was rejected four separate ways. In a fresh comparison, in a loop-depth diagnosis, after a state-update repair and in the full-recipe replication, it failed to beat matched controls (items 3, 4 and 6 below give the detail).
  2. Width. Reader widths from 64 to 256 left the task gain essentially flat; no width was selected.
  3. Loop depth. Depths 1, 2, 4 and 8 were tested on frozen builds; none beat both storage- and compute-matched controls at both seeds.
  4. State-update repair. Changing how the loop updates its state did not beat matched controls.
  5. Objective reshaping. An equal-record training loss did not stabilise the candidate and was not selected.
  6. Full-recipe seed stability. The complete train-and-select recipe was not stable enough across seeds to promote to scale.
  7. Gating to fix the drift. Section 3.3: no gate removes the cap.
  8. Reader generalisation. Section 3.1: the effect reverses on a second corpus.
  9. Sharpening reaching generated text. Section 3.2: accuracy stays at zero under a repaired instrument.

A later run asked whether the sieve writes in the right places, and it mostly does. At answer positions it opens where the write improves the answer (AUROC, how well it ranks helpful positions above the rest, where 0.5 is chance: 0.69 to 0.86 against the realised gain, above the 0.55 bar at all eight cells), and an oracle with the same write budget beats it by only 0.007 to 0.027 nats; the preregistered bar for headroom worth a better gate was 0.02, and only 3 of 8 cells reached it. Whether its open probability is calibrated could not be read, because it opens on 87% to nearly 100% of answer positions. Entropy is a weaker signal (AUROC 0.47 to 0.72), so the entropy-driven halting controller planned next was not trained.


5. The decision

After the first attempts at both measurements (the confounded second-corpus run in 3.1 and the defective decode in 3.2) and before the two deciding runs, I wrote and committed a decision table that set out what each outcome would mean:

Reader generalises (gate open)? Reaches generated text? Decision
yes any CONTINUE
no yes CONTINUE, narrowed scope
no no / at zero STOP
unmeasurable any NO DECISION

The measurements landed in the third row: the reader does not generalise, and decoding accuracy is zero under the repaired instrument. The recorded decision is STOP, and the criterion was not amended after the deciding run began.

STOP means the work does not warrant further investment. It does not retract the findings of Section 2, which stand on the corpus where they were measured and were verified by exact byte-level reproduction throughout.


6. Method, and three corrections

Every experiment was preregistered. Its configuration was frozen and its cryptographic hash committed before the run started. Each run used an immutable, hash-verified code release, and a result was accepted only after an independent byte-for-byte replay and a semantic re-derivation both matched it. Diagnostics were forbidden from selecting thresholds, arms or objectives. Sealed test and held-out data were never read.

The claims that survived are the ones a later run tested directly. Three times, a run overturned one of my conclusions:

  1. A scoring convention inflated an absolute level. An early behavioural result credited the gold token whenever nothing strictly beat it, counting ties as wins. Under 16-bit arithmetic, ties at the top of a 50,304-way distribution are common: they covered about a quarter of positions. A later run using strict argmax showed the model emits the gold token at 40.1% of positions, not the 64.7% first reported. The contrast between readers survived; the absolute level was corrected.

  2. A hypothesis about a disagreement was wrong. Two runs disagreed in sign on the control arm. I proposed that the cause was an aggregation choice. A reconciliation run recomputed both from one set of per-record vectors and refuted that: the sign flipped with the scoring convention, not the aggregation, and it revealed the arm asymmetry that became Section 2.

  3. A reliance ranking measured the wrong thing. An attempt to identify which of the four layers carries the effect, run with the gate in its normal mode, was found to be measuring the router’s surprise at a perturbed input rather than the reader’s dependence on a layer. Re-run with the gate forced open, the effect reversed sign between seeds, so I do not know which layer matters, and the record says so.

Each correction was recorded as a new event referencing the one it corrected; no prior record was rewritten.


7. Open questions

Two questions are unresolved. Neither points to restarting the work:

  • A reasoning corpus was never tried. The only second corpus available was pretraining text; the reasoning datasets I shortlisted were ruled out by their licences. Nothing in the present evidence suggests it would change the verdict. A positive result there would also have to show that the effect is not an artefact of the synthetic generator.
  • The layer that carries the effect is unknown, because the two ways of measuring it (gated and gate-open) disagree, and the gate-open measurement is seed-unstable. This does not reopen the decision.

Appendix: provenance of the main quantitative claims

Each single-cell value in this table was re-derived from a raw run summary before this document was written, by a standalone script independent of the analysis code that produced the summaries. Run identifiers are timestamped; each names a directory of machine-readable results.

Claim Value Source run Extraction
Frozen backbone size 162,322,944 D33a …T190145Z cell resolved config, model.loaded_parameter_count
Reader / sieve / total params 271,745 / 53,505 / 0.2004% of backbone frozen D33a config architecture block
No recurrence, no internal writes true / 0 frozen D33a config architecture
Token-level distinct arm +0.00511 to +0.00718 D38 …T174145Z per-cell grid.strict__token_weighted, multi_tap cells
Token-level repeated control -0.00400 to -0.00552 D38 …T174145Z same, final_tap_control cells
Δ margin, distinct / control +0.01325..+0.01766 / -0.03257..-0.02208 D39 …T182141Z per-cell mean_margin_change
Δ entropy, distinct / control 4/4 down / 4/4 up D39 …T182141Z per-cell mean_entropy_change
Forced-open control margin (native) +0.00287 to +0.00670 D42 …T203414Z second_corpus_forced_open, final_tap cells
Native base top-1 0.30928 D42 …T203414Z clause_verdicts.native_gate
Prefix decode base/routed 0.000000, all cells/lengths D42 …T203414Z prefix_decode
reader_generalises false D42 …T203414Z readings.reader_generalises
Joint answer prob > 0.1 0 of 462 D33a …T190145Z per-record evaluation, exp(−base_answer_nll)
Mean base answer NLL / routed gain / relative 56.822 / 0.10252 / 0.18% D33a …T190145Z per-record evaluation means
Always-on retention KL 0.143025 (s179 family-record cell); 0.123 to 0.144 over four cells D34 …T123741Z cells[...].retention.always_on_mean_kl_nats
Retention limit 0.05 frozen D33a config comparison.retention_maximum_mean_kl_nats
Top-1% share of KL / gated off 0.0817 to 0.0997 / 1.0 D34 …T123741Z retention.bands.top_0.01
KL after suppressing worst 5% 0.110290 (s179 family-record cell); 0.092 to 0.110 over four cells D34 …T123741Z retention.counterfactual_suppression
Decision criterion hash 409c17bf… frozen criterion file SHA-256, matches D42 binding

The script ran 36 checks on the single-cell values in this table. All 36 passed. The four-cell ranges and the 0.2004% share were read later from the same summaries.