Why I did this
A local model is cheap, private, and wrong in ways that look exactly like being right. The question is not whether the model is good; it is whether the system around it can tell.
The research began with a memory system: a 17M encoder curating a long conversation into a bounded context. It became a study of consequences - which decision, when wrong, actually costs something. That thread runs through every tab here: a false merge deletes a fact; a false accept ships a wrong answer.
The hypotheses that drove it
H1 · The harness is the lever
A small model with verification can beat a bigger model alone. The 27B reaches 26/28 against the 35B's 21/28; the 176B's own generation is net zero. Supported - for the harness, not for "bigger".
H2 · Consequence, not accuracy
Optimise what a wrong decision costs. A better dedup decision at
0.67 halves the facts (60.51% → 32.66%); the
judge's four misses at p ≥ 0.97 are plausible
answers. Supported.
H3 · Execution, not judgement
A reader cannot see a plausible wrong answer; a check can. The execution checker is measured: 6/7 errors, 4/4 at p ≥ 0.97, 0/21 false rejects, 0.23 ms median. Supported by the failures and the fix. The controller loop is measured; the API-driven controller is next. Grounding is not pursued.
H4 · One loop, two levels
The membrane and the controller make the same trade - what to keep, what to check, when to stop. The memory level and the scripted controller are measured; the API-driven controller is next.
What follows. The next tabs are the evidence: the memory's encoder at work, the routing, the controller's measured controller, the measured table, the measured checker, and the paper. The first one is interactive.
The memory: what the drone learned
The first harness curated within a conversation. A 17M encoder, “the drone”, scores every stored chunk; the membrane makes two decisions on every turn: dedup (is this chunk already said?) and drift (did the topic change?), and the assembler keeps the context under budget.
The stage above is the argument for the conservative dedup, on a prompt
that asks how order deletions are handled. At 0.92 the
two schema chunks are both selected - one of them carries the
deleted_at fact - and the prompt can answer. At 0.67 the
near-duplicate merges before selection, and the only evidence for deletes
goes with it. The drift chunk dims in both states: it is only penalised,
never deleted.
The slider is there to be explored: the pair's cosine is 0.89, so the boundary is discoverable, and either side has a visible cost. But the shipped 0.92 is not derived from this pair - it is set by the corpus replay in the cards below (60.51% of the facts at 0.92, 32.66% at 0.67). A threshold is a population decision; the lab is one case.
The shipped point
60.51% of the facts kept in 374 tokens. Precision 1.00: it never deletes a fact.
The tuned dedup at 0.67
32.66% of the facts in 139 tokens. The decision improved; the curation nearly halved.
The lesson
A true duplicate merged is a fact deleted. The threshold and the model are two dials on the same curve, and the constraint fixes where the system sits.
The drift threshold is the free dial: it only penalises, it never deletes. The dedup threshold is the constrained one, because a false merge deletes a fact - and that is what fixes where the system sits on the curve.
The routing: the same loop, one level up
The membrane answers "what should stay in the context?" The controller answers "what should handle this task, and when is it done?" They are the same question at different scope, so the machinery carries over:
| the memory | the controller |
|---|---|
| the router scores every chunk | ask(model, prompt) routes to the tier |
| dedup decides; drift only penalises | verify decides; score only advises |
| the judge is a readout, not the gate | score is consulted only when verify is unknown |
| the assembler keeps the facts under budget | the loop keeps the answer under a step budget |
| a false merge deletes a fact | a false accept ships a wrong answer |
The routing is not a different mechanism. It is the same trade - what to keep, what to check, when to stop - with the consequence moved from the context to the answer.
The controller: four tools, one table
The controller is a router and a verifier, not a knower. That is why it can be small: the tools do the work, and the table says what each tier is worth.
ask(model, prompt)
Route the task to a tier: the 35B (bulk), the 27B (repair), the API (residue).
verify(task, answer)
Execution
only. pass / fail / unknown; a check that cannot fail a
mutation is unknown. Grounding is not pursued.
score(task, answer)
The judge's
p(correct). Only when verify is unknown; it
never overrides a check.
ask_human(...)
The last local tier: solve, escalate or reject - with the evidence.
The stage above is the loop on one task. The judge read the answer and
wanted to keep it (p = 0.973); the checker recomputed it:
700×5 + 1,400×15 = 24,500¢ = $245. The
judge has a blind spot exactly where the answer is plausible;
the checker does not. This is one of the four errors at
p ≥ 0.97. On the 28 tasks the checker catches 6 of 7
errors, all 4 at that confidence, with 0 of 21 false
rejects, at 0.23 ms median. The scripted loop serves 28/28 with 7
escalations and 0/21 correct sent up.
The loop is bounded. After a configured
number of steps the harness removes the tools and requires a final answer:
accepted, escalated or rejected. The
invariants are enforced by the tool surface and the policy layer, not by
the prompt.
What is measured
The same 28 organic tasks throughout. Costs are wall-clock per item on this box.
| the arm | the result | the cost |
|---|---|---|
| the 35B alone | 21/28 (0.750) | the cheapest local |
| Bonsai 2 27B | 26/28 (0.929); +5 net (6 fixed, 1 broken) | 8.92 s/item |
| the 176B generating | net 0 (2 fixed, 2 broken) | 20.68 s/item |
| the 176B head judging the 35B | AUC 0.864; 3 of 7 errors, 0 of 21 correct sent up | 2.52 s/verdict |
| judge gate + 27B | 23/28 (0.821); 3 escalations, all errors | 3 escalations |
| oracle gate + 27B | 27/28 (0.964) | 7 escalations |
| the head + live 27B/API | 24/28 (0.857); 3 escalations; 0/21 correct sent up | 10.21 s mean; 17.81 s conditional |
| the API (DeepSeek V4.1 Flash Free) | 28/28 (1.000) | 2.6 s median; free tier |
| the scripted controller (checker + head + 27B + API) | 28/28 (1.000); 7 escalations (5 27B, 2 API); 0/21 correct sent up | 9.24 s mean; 17.22 s conditional |
| the API in the 27B slot (DeepSeek V4.1 Flash Free) | 28/28 (1.000); 7 API escalations; 0/21 correct sent up | 8.57 s mean; 14.54 s conditional |
| the API in the 27B slot (GLM 5.3 Flash Free) | 27/28 (0.964); 7 API escalations; 0/21 correct sent up; one unverifiable choice | 7.80 s mean; 11.45 s conditional |
| the checker (execution + arithmetic) | 6/7 errors; 4/4 at p ≥ 0.97; 0/21 false rejects; 17/17 mutations | 0.23 ms median; 30 ms p95 |
The 176B is the wrong shape for escalation: it is a judgement model, and its generation is no better than the 35B's. A 27B dense one is +5 net. The judge is the 176B's head; the execution checker and the scripted controller are measured. The API-driven controller is the missing part. The API-slot rows are one sample per task; the five-run repeat was blocked when the free tier returned zero-token failures for every call. The ten-sample choice probe is in the record.
The checker: execution, measured
Execution where a program can settle it (run the code,
recompute the arithmetic). Grounding is not pursued; a claim/citation
verifier would duplicate the memory bench's fidelity/recall and still
pass wrong retrieved facts. The execution checker sits behind one
verify; the log records the method.
The stage above is the falsifiability rule. A check that passes the
candidate and fails the mutation is evidence. A check that passes both is
print("PASS") - and it must be unknown, or a
small checker is worse than none.
The false-positive bar
The 176B head set it: 0 of 21 correct answers sent up. The checker is scored on the 21 correct first, then on the 7 errors.
Attribution, not a new tier
The memory bench already has context fidelity and recall. A cheap attribution check belongs there. A claim/citation verifier is not pursued.
The status
Execution: measured.
17 pass, 6 fail, 5 unknown on the 28 tasks; the four errors at
p ≥ 0.97 all fail; 0 of 21 correct
answers are rejected; every pass survives a mutation. Grounding is
not pursued; the controller loop is measured; the API-driven controller
is next.
The contract and the prior art are in
cascade/HARNESS-CONTROLLER.md and
cascade/CONTROLLER-PRIOR-ART.md.
The paper
The full paper lives in the repository, versioned with the code and the results it describes.
This page is the shop window; the paper is the record. It carries the method, the gold-free contract, the measured tier results and their corrections, the negative results I kept, and the controller table.
Read the paper on GitHub Raw markdown
The claim
On this class, a 35B with a check-and-repair loop serves 67/84 over three draws against 64/84 without it; a 27B escalation tier adds +5 net; the 12× larger 176B's generation adds nothing.
What it covers
The method and the gold-free contract; the taxonomy; the measured tier results; the learned gate's negative result; the harvest and the reporting; the routing table; and what is not claimed.
The record
Every number has a run bundle,
and every correction stays in
docs/AB-RESULTS.md.
The current state is in
docs/STATUS.md.