The gate: invariants, provenance and mutation
The gate of this repository is a file: tools/eval/invariants.mjs. It runs in pure node
and exits with code 1 if any critical invariant fails. It is what CI executes on every
PR (.github/workflows/ci.yml).
tools/eval/invariants.mjs: 2,275 lines, 65 declared invariant identifiers, with 28 declaredskip()paths.- The harness contains 201 scripts in
tools/eval/, plus 55 pipeline scripts intools/. - The number of critical checks in one run depends on the inputs present on that machine; dated results belong in
KNOWN-BUGS.md.
grep -o "put('[A-Z0-9_]*'" tools/eval/invariants.mjs | sort -u | wc -l
grep -o "skip('[A-Z0-9_]*'" tools/eval/invariants.mjs | sort -u | wc -l
Block generated by
node tools/gen-docs.mjs. Source:grep -o "put('[A-Z0-9_]*'" tools/eval/invariants.mjs | sort -u | wc -l
The third item above is the distinction that most confuses newcomers: declared
identifier ≠ evaluated invariant. Several become skip instead of put when their
input is missing (the viewmodel auditor's JSON, a GLB, a folder of anims). skip is a
green gate by absence of data, and that is why it always carries the reason. See
"Severity", below.
This page is the most useful one on the site. If you only read one, read this one.
Why it exists
From the header of the file itself, tools/eval/invariants.mjs:5-19:
The owner spent 3 days in a cycle where every round fixed one thing and broke another, and we only found out one round later. The cause wasn't lack of care: it was the lack of a RULER (quality gate). A critic (human or agent) judges screenshots; consistency and flow are properties of the game IN MOTION, and almost every defect he reported is not taste — it's a violated invariant.
And the translation, which is the most important thing in this entire repository:
| What the owner said | Which invariant it became |
|---|---|
| "the hands are floating in the air" | hand↔grip distance has a ceiling |
| "the weapon points downward" | the barrel has a maximum angle |
| "in ADS I can't see the weapon or the crosshair" | the weapon has a minimum and maximum area |
| "sniper with no zoom" | aiming FOV < hip FOV |
| "several weapons look the same" | silhouettes must differ |
| "the bot shoots out of nowhere" | damage requires prior LOS |
| "there are 2 of them eliminating me" | 1 killfeed per death |
tools/eval/invariants.mjs:20-21:
GOLDEN RULE: nothing gets committed with a RED invariant. And every new bug the owner reports becomes an invariant here — that's how it never comes back.
What an invariant is here
An invariant is a property of the game that can be measured without a human watching, with a ceiling or a range that has provenance. It is not a unit test: almost no invariant tests a function. They measure the state of the game actually running.
Three forms, all present in the file:
1. Read from the source code. Cheap, runs in milliseconds, catches whole classes of bug.
Real example, quoted verbatim from tools/eval/invariants.mjs:1439-1446 — the file's
comments are written in Portuguese:
// ARM1 — toda arma com luneta precisa de zoom de verdade. "Snipers sem zoom"
// é reclamação literal; a solução NÃO é tirar a luneta, é fazer a certa.
const bloco = gsrc.slice(0, gsrc.indexOf('};', gsrc.indexOf('const WEAPONS')) + 2);
const linhas = bloco.split('\n').filter((l) => /^\s*\w+:\s*\{/.test(l));
const semZoom = linhas.filter((l) => /scope:\s*true/.test(l) && !/spreadScope/.test(l))
.map((l) => l.trim().split(':')[0]);
put('ARM1', 'toda arma com scope:true declara spreadScope', semZoom.length === 0,
semZoom.length ? semZoom.join(', ') : `${linhas.length} armas conferidas`);
2. Measured in the real game running in node. tools/eval/harness.mjs boots the real
Game class, with the real maps, with DOM/canvas stubbed. It is the production code
that gets measured, not a reimplementation — tools/eval/botsim.mjs:8-9: "if the number
improves here, it improved in the game". This is where BOT1–BOT8, MAP1–MAP3, CTF1, MAT1/MAT2,
FOG1, TEX1, VM14, MOD1/MOD2 come from.
3. Measured on the geometry of the assets. vm-mint-audit.mjs opens every weapon GLB with
its own GLB parser and projects the viewmodel onto the screen. This is where VM1–VM19 come from.
What does not belong here: an invariant that requires browser pixels. Those are marked
browser and are skipped, with the reason stated — SwiftShader costs ~4 min per map load
on this machine (tools/eval/invariants.mjs:99).
Severity
put(id, desc, ok, evid, sev) accepts 'crit' (the default) or 'warn'
(tools/eval/invariants.mjs:81-82). A red critical fails the PR. Warn is measured noise that
someone needs to look at but doesn't block — it's where BOT1/BOT2/BOT3/BOT6/BOT7, ARM4 and
ARM5 live. skip() is the third state, and it is dangerous: a green gate by absence of
data. That is why every skip carries the reason.
The two house laws
Law 1 — Intention that doesn't become an invariant is optimized away
Source: tools/eval/invariants.mjs:452-461. The case, verbatim:
the previous round took the gate from 16/21 to 19/21 without loosening a single ceiling and was still REJECTED by the owner, because to close VM5/VM10 it ZEROED the
VM_OFFy and silently changed the look. No invariant encoded "where the muzzle sits", so the metric was optimized and the INTENT was destroyed. Goodhart's law, in full. INTENTION THAT DOESN'T BECOME AN INVARIANT IS OPTIMIZED AWAY.
Read again what happened, because it is counterintuitive: the agent did not cheat. It
didn't loosen any ceiling. It genuinely raised the score. And the result was worse, because
VM_OFF[1] is the term that dominates the weapon's position on screen — public/js/game.js:555
declares VM_OFF = [0.03, -0.1000, 0], and tools/eval/invariants.mjs:1163 measures the
sensitivity: "removing recuoZ moves the grip 3,5 cm; removing VM_OFF moves it 23 cm".
Zeroing that term closed two invariants and erased the aesthetic decision the owner had made — which wasn't written anywhere the ruler could read.
The fix was not punishing the agent. It was writing the intention as an invariant. Today
there is VM12 (tools/eval/invariants.mjs:497): "CS 1.6 look: muzzle RIGHT below
the crosshair (y between 0,50 and 0,62) in the 2 aspects". With it in place, the same
optimization goes red.
And the operational consequence, from the same comment:
Whoever wants to change the look has to change THIS ceiling explicitly, in a diff the owner sees, instead of touching
VM_OFFand reporting "+3 invariants".
If your change improves the gate's score, the first question is: what did I change that the gate doesn't look at? If the answer is "the look", "the feel" or "the vibe", write the invariant before sending the PR — or explain in the PR why it doesn't fit.
Law 2 — A ceiling without provenance is an opinion
Source: tools/eval/ref-measure.py:1-40. That docstring is the house doctrine. The case:
For three days the weapons gate was solved against asserted numbers:
- VM12 required "muzzle at y ≥ 0,66".
- The
vmattach.jsdoc said "ENTIRE stock in the corner".
Neither of the two was measured on any image. According to tools/eval/invariants.mjs:461-463,
the 0,66 floor came from a comment in public/js/vmattach.js — "the muzzle sits at ~0,66H" —
which in turn came from a video someone watched. (The gate's comment points to
vmattach.js:387-392; today the text is at vmattach.js:395, because the file moved. That is
exactly why ARCH.md is generated — see Architecture.)
The owner looked at the result and said, verbatim (ref-measure.py:14-17):
"it looks different from CS 1.6 and Quake and UT; in those 3 the weapon is always in the bottom-right corner and the stock is always OUTSIDE; after 3 days and an entire folder of reference neither you nor Kimi understood that."
Then the frames were measured. tools/eval/ref-measure.py does color segmentation on the
bottom-right quadrant, takes the largest connected component, and writes
tools/eval/ref_viewmodel.json. Result:
| Frame | Muzzle (x, y) | Screen area | Axis angle | Crosses the right edge? |
|---|---|---|---|---|
cs16_ak_dust.jpg | 0,564 ; 0,513 | 9,76% | 28,0° | yes |
cs16_m4_dust.jpg | 0,569 ; 0,598 | 9,78% | 34,8° | yes |
valorant_vandal.jpg | 0,648 ; 0,587 | 13,09% | 4,6° | yes |
Both asserted numbers were wrong:
- The CS 1.6 muzzle sits at 0,513–0,598 — right below the crosshair (0,5), 1 to 10
percentage points below center. Not at 0,66–0,93. The wrong floor was keeping our
weapon sunk at 0,667–0,816 (
tools/eval/invariants.mjs:472-475). - The stock EXITS through the corner in all 3 frames. Exiting is the standard, not the defect
(
ref_viewmodel.json→faixas.cruzaBordaDireita: truein all 3).
And the collateral damage: with the false ceiling, the previous round's solver "proved" that 3%
of area was infeasible. The proof was correct against that ceiling — and it was the ceiling that
was false (tools/eval/invariants.mjs:476-478).
The rule that remained, tools/eval/ref-measure.py:21-22:
AN INVARIANT CEILING ONLY GETS IN WITH PROVENANCE — reference file, measured pixel, and this script reproducing the number. A number without an image is an opinion.
Today the framing invariants carry the provenance in their own text: VM1 (range 0,50–0,60, ref 0,520–0,565), VM3 (22–42°, ref 28,0° and 34,8°), VM5 (6–16%, ref 9,76–13,09%), VM12 (0,50–0,62, ref 0,513–0,598), VM16 (slice at the right edge 0,02–0,20, ref 0,053–0,095).
tools/eval/invariants.mjs:599-602 refuses to create a ceiling for "how much of the weapon
sits outside the frame", because what is outside is invisible in the photo — there is no way to
know whether the AK's stock ends 5 cm or 50 cm past the edge. The numbers remain in the JSON as
evidence, without a gate. That is provenance taken seriously: the ruler says where it stops
knowing.
And the same rigor bites whoever wrote the ruler, in the most uncomfortable case possible: the
character reference photos arrived, were measured, and were REJECTED by the ruler
itself. tools/eval/char-probe.mjs:25-45 tells the whole episode — references/funkeiros/
has 23 files and references/palhacos/ has 21, all run through ref-body.py, with the
masks looked at (--masks). The verdict, said to our face by the comment itself: they are
selfies and close-ups; the heuristic segmentation returns the hand, a piece of jacket or the hair
of someone else in the background, and the shoulder/height ratio comes out between 0,42 and 3,78 when a human
measures 0,259. About ~1 usable full-body photo remains — that is not a sample.
ref-body.py requires 6 accepted photos for a ceiling to become measured, and it says why
it didn't. So the absolute ceiling of CHR1 remains a published fallback (Drillis &
Contini 1966, via Winter), declared as such in the procedencia field of the JSON and in the
report column.
Notice what that means: having the photo is not having the measurement. It was easier to accept that the data was bad than to promote a fragile measurement to a ceiling — and that is Law 2 applied against the interest of whoever wrote the ruler.
references/ does NOT come with the clone — and that is a decision, not carelessnessgit ls-files references returns zero. On 04/08/2026 the entire folder was
untracked by the owner's decision ("references can stay local because we're going to build
locally"): they are the UI target screens and the viewmodel reference frames, and they live only on
his machine.
What survives the clone are the NUMBERS measured from them: tools/eval/ref_ui.json and
tools/eval/ref_viewmodel.json are versioned. That is the contract — if a ruler of yours
needs to run in CI, it reads the JSON, never the PNG. A ruler that opens an image from
references/ goes red on every machine that isn't the owner's, and red-by-environment is
the worst kind: it teaches whoever works here to ignore red.
Mutation test of the ruler itself
This is the part almost no project has, and it is where this repository is genuinely different.
A gate that doesn't move when you break the code on purpose is blind.
The way to find that out is to mutate: take the fixed code, undo the fix on purpose, run the gate, and see whether it goes red. If it stays green, the gate is not measuring what you think it measures.
The case: 20/22 green with the fix removed
Source: tools/eval/invariants.mjs:910-920.
The context: public/js/game.js:577 declares
const vmOffY = (aspect) => VM_OFF[1] * ((16 / 9) / (aspect || 16 / 9));
It is the per-aspect vertical framing fix — the reason the weapon sits in the same
place in 16:9 and in 3:2 (the owner plays in 3:2). It is called in the Y argument of
this.vm.root.position.set(...), at public/js/game.js:4873.
The hole, measured in 08/2026:
the
vmOffstep only checked/this\.vm\.root\.position\.set\(\s*VM_OFF\[0\]/— the X term. The Y term was checked by no one, and the auditor (vm-mint-audit.mjs:196,loadOffYFn) reads the DECLARATIONconst vmOffY = (aspect) => ...by regex without ever asking whether anyone CALLS it.Result: swapping in
game.jsthe callvmOffY(...)forVM_OFF[1]in the Y argument — that is, removing the per-aspect vertical framing fix entirely — the whole gate stayed GREEN (20/22, with VM9, VM10, VM12 and VM15 all green). A gate that cannot tell the fixed build from the build without the fix is measuring nothing.
Notice the mechanism of the error, because it repeats in any language: the invariant was reading the declaration of a constant, and not the use. Declaring and not calling is the cheapest way for a fix to vanish with the gate green.
The repair was surgical and worth copying. AUD1 today separates the three arguments of
position.set(...) with a parenthesis scanner — not split(','), which would cut
inside the function call — and requires by name that the Y argument calls vmOffY(.
And it closes the other path along with it (tools/eval/invariants.mjs:1148-1151): the formula of
vmOffY is read from game.js and evaluated at 16/9, and it has to yield exactly VM_OFF[1].
The two checks together cover the two ways for the fix to vanish: deleting the CALL (mutation measured in 08/2026) or tampering with the FORMULA.
It was not an isolated case — there were three
The same hole showed up in two other places, and each one became a new AUD1 step:
| Mutation | Score with the fix undone | Cause of the false green | Where |
|---|---|---|---|
Swap vmOffY(...) for VM_OFF[1] in the Y argument | 20/22 green | the invariant read the declaration, not the use | invariants.mjs:910-920 |
Swap g.rotation.set(pit, yaw, t.roll) for g.rotation.set(0, 0, t.roll) | green | the VM_FRAME.cls table still holds the angles, and the three mirrors still match each other | invariants.mjs:932-944 |
Delete * (weaponCFG(id).vm ?? 1) from the mesh scale | 28/37 green, AUD1 included ("worst Δscale 0.0004") | both ends read vm from weapons.js; game.js is never asked — it was the auditor checking itself | invariants.mjs:971-975 |
Mutate this._adsPose['pistol'] | 20/22 green | ADS had no invariant at all | invariants.mjs:1185 |
The common pattern of the four is the same, and it is what you should look for in your invariant:
The ruler is checking a copy of the rule instead of the game. Whether because it reads the declaration and not the use, or because it compares two mirrors that read the same source, or because the parameter table stays correct while no one applies it. If the two ends of your comparison can stay consistent without going through the production code, your invariant is blind.
tools/eval/mat-check.mjs:18-27 solves this in the most direct way possible: the body of
fixVmMaterials is cut out of game.js and executed against a probe material. If the
code changes, the ruler changes with it. "a ruler that carries a COPY of the rule lies on the day
the rule changes."
Mutation as a first-class thing: ui-check.mjs
The UI harness has a versioned mutation table, and each one declares which gate
has to go red. tools/eval/ui-check.mjs:1046-1050:
Each mutation UNDOES one of this round's fixes (or punches through a gate on purpose) and says which gate MUST go red. A ruler that does not fail the previous version of its own file is not a ruler, it is decoration.
Running one:
MUT=ui1_ctf_scrim_fraco node tools/eval/ui-check.mjs # expects UI1 to go RED
MUT=ui3_prompt_na_mira node tools/eval/ui-check.mjs # expects UI3 to go RED
MUT=ui2_prompt_eterno node tools/eval/ui-check.mjs # expects UI2 to go RED
MUT=ui4_ctf_sem_relogio node tools/eval/ui-check.mjs # expects UI4 to go RED
The 7 mutations are in tools/eval/ui-check.mjs:1051-1134. Two mechanics: css rewrites
public/style.css read in memory (never on disk — other agents are editing the
file right now), and sim monkey-patches the already-booted Game object. If the css mutation
matches nothing, the script exits with code 2 saying "the CSS changed shape" — because a
mutation that doesn't apply is also a false green (ui-check.mjs:1164).
How to write an invariant
Checklist, in order:
- Start from the defect's sentence. Verbatim, with the words of whoever complained. Every
harness in this codebase starts that way, and it is not style: it is what keeps the invariant from
measuring something else. See the header of
tools/eval/map-check.mjs:5-12— five sentences from the owner, five invariants. - Translate it into a measurable quantity. "the players are SUBMERGED UNDER THE STATUE" → there is visible map geometry whose top rises more than 0,30 m above the floor local to that point (MAP1). Note that the operational definition includes why 0,30 m: it is the step the body climbs; above that it is not "stepping over", it is "being inside".
- Find the ceiling's provenance. Reference file + measured pixel + script that reproduces it. If it doesn't exist, say it is a fallback and cite the published source, as C1 does. Never invent the number.
- Measure the production code, not a copy of it. Import the real module, cut the function out of the file and execute it, or require the call by name in the source text.
- Mutate and confirm it goes red. Undo the fix you just made and
run the gate. If it stays green, your invariant is blind — go back to step 4. If it can
be automated, register the mutation in a table, as
ui-check.mjsdoes. - Write the evidence, not just the boolean. The fourth argument of
put()is what someone will read three months from now:"0,504 a 0,619 da altura em 52 medidas | 0 fora da faixa"is useful;"ok"is not. - Write the provenance comment above it. In Portuguese, saying what happened when the number was wrong. That comment is what keeps the next round from redoing the mistake.
Anti-patterns that have already cost dearly here
| Anti-pattern | What it produced |
|---|---|
| Reading a constant's declaration instead of its use | 20/22 green with the fix removed |
| Two mirrors that read the same source | 28/37 green, "worst Δscale 0.0004", with the knob turned off |
| Format adapter silently broken | VM1–VM6 have been SKIPPED since the auditor exists — 6 viewmodel invariants that never ran once (invariants.mjs:121-127) |
| Measuring the gap against the wrong local floor | a pickup inside the pool reported gap 0,0000 — GREEN (pickup-check.mjs:20-23) |
| "waypoint ≤ 3 m" as a reachability proxy | 74 false positives and green in a closed pocket (pickup-check.mjs:34-42) |
| Floor without ceiling | "muzzle ≥ 0,66" accepts the muzzle at 0,95 (weapon in the basement) — that is how we got to 0,816 (invariants.mjs:432-434) |
This page is the doctrine. The step-by-step is a skill
What to do, in order, when someone reports a defect — reproduce, measure before
fixing, refute the obvious guess, mutate the ruler, run the gate in the right order and report
what was not verified — is in .claude/skills/bug-hunt/SKILL.md, with the real case
that paid for each rule. It is written for agents and for people, and it points back to
this page instead of repeating it.
Running the gate
node tools/eval/invariants.mjs # everything that runs without a browser
node tools/eval/invariants.mjs --json # machine-readable output
npm run check # syntax + gate + vm + recoil + bots
Current production, data, and debt sources: Current state.