Glass Box dev log
What comes next
The jam build is frozen. This is what comes after it, and the design for a leaner engine built from everything above.
- Brown is barely shown in the Interrogation Room in play or video: give him camera time and blocking there (Cristian, 15 Nov).
- Subtitles: stage directions shown unlabelled (fix on integration 011cd72a, not in gold).
- Booth: remove the notice board and drawn paper; Mary-Grace's eyes must read through the glass.
- Jesse's jumpsuit sleeve moire on the forearm.
- 1st AD round-2 rc1 notes (set/art x4, characters x1), tasks 440db520 and b296bc77.
- Lane B Screening Room projector beam re-aim (M12.11.5.fix2, no passing capture).
- After submission: make the M16.2 gameplay video PUBLIC on Saluca Labs YouTube (Cristian, 15 Nov: "we'll make it public once we finish the submission"), then use it in an "updates on soulnode" post: "we started the game dev experience .. and now we're competing in Jams getting ready for our primary game release" (draft only; publishing stays his).
- NHE relay: live LLM replies never went live (R26 dropped M12.17.6 after npm test/tsc failures; R23 switch-on never happened). Game client and settings exist (M12.17.3-5, PIE-proven). Post-jam: finish persona-relay, run R23's smoke test, then switch on (Cristian asked 15 Nov).
- Settings: 'AI relay' page with modes Saluca relay (default) / Custom relay (URL + optional token; ~0.5 day) / Direct provider (provider, model, API key via DPAPI, safety checks ported from persona-relay; 1-2 days). Test-connection button; https only except loopback. Cristian asked 15 Nov; recommend Saluca + Custom first.
The game creation engine
What the engine is
One runner that takes a theme and a concept and drives AI agents to a shipped, submitted game, with a person deciding only. It keeps what worked in the jam:
- lanes that own maps;
- side slots for work that needs no editor;
- one asset per task;
- an independent reviewer (the 1st AD);
- rulings carried in every brief;
- recovery;
- logs for every act.
It removes the classes of failure below. It is built around four ideas:
- The plan is state, and the runner reloads it. A milestone, a ruling and a freeze all change behaviour at the next claim, never at the next restart.
- Gates run from hour one. Anything that can fail the submission is measured on every package, not discovered at the release candidate.
- Every task has an owner, a model, a budget and an id the runner allocated.
- Tokens are a measured resource, per task, per model and per kind of work.
Failure classes and the rule that prevents each
| # | What happened in the jam | Design rule |
|---|---|---|
| F1 | The 1 GB zip limit was found at rc1: 1,430 MiB, then three more packages to get under it | Size budget gate. Every package is zipped with the real tool and measured against the real limit (decimal), with a margin (950 MB target for a 1 GB limit). Lossless compression (Oodle Leviathan, deflate 9) is in the project template from day one. A landing that pushes the projected zip over budget is blocked. |
| F2 | Editor-only skeleton references broke two rc1 cooks (61 face animations) | Cook-safety gate. On every content landing, scan the changed assets' references for editor-only paths (/MetaHuman/IdentityTemplate/ and a maintained list), and cook every new asset type once on its first landing. |
| F3 | No Shipping package ran for about 9 hours while look work landed, so the white render was found late | Package smoke gate. A Shipping package with a boot, a luma check and frames looked at runs on every landing that touches maps, lighting or C++, on the workstation and the test device. |
| F4 | The store page claimed no AI voices or images after both had shipped | Page drift gate. Store text lives in the repo. The build's asset manifest (tools, sources, licences) is diffed against the page's AI-use and credits sections on every landing, and drift blocks the release task. |
| F5 | Video audio: the first attempt blocked on routing; the loudness gate refused the cut | Capture recipe in the template. Loopback audio capture and loudness mastering, proven with a 10 s clip in hour one. |
| F6 | Fix rounds and re-cuts kept coming after the video was cleared and the build was gold | Freeze is state. The plan records frozen artifacts by hash. Reviews of a frozen artifact become post-release notes. Recovery never reposts a task whose output is frozen, and fix tasks against frozen artifacts are refused at post time. |
| F7 | Haiku broke a checkout, stopped at a tagging allowlist, ran stale size work, and built a pack with the wrong video | Route by task type, for every attempt. Release, package, git surgery, submission and multi-asset merge tasks always run on the strong rung. A pin is a property of the task, not of its first attempt. |
| F8 | A superseded task's recovery ran in parallel with its replacement | One owner per goal. A task names its goal key. Posting a second task with the same goal closes the first and its recoveries. |
| F9 | Dependencies on stale mesh ids held finished work; a .size title was invisible; fix ids collided |
One id module. It allocates ids (the poster never picks the number), parses titles and refuses unparseable posts. Dependencies are task ids, resolved to the latest attempt. |
| F10 | A runner rename broke three modules; a dry run requeued live work; kills left index locks | Safe operations. Runners have stable ids with the model as a property. The dry run is read-only by construction (no writer is reachable from it). Stop and restart clean their own locks and rescue untracked files. |
| F11 | Restarts to load merged code killed in-flight work, about 11 times in two days | Hot reload at a save point. The runner re-imports its code and re-reads the plan between claims. A slot never has to die to pick up a fix. |
| F12 | Leftover assets dirtied checkouts; an orphan game instance sat at the menu | Process and asset ownership. A task owns every process it launches and every asset it loads. On exit both are closed, or the editor restarts before the next claim. Leftovers are rescued to a branch automatically. |
| F13 | One editor lane was a bottleneck; ownership deadlocks between lanes | Lanes own maps from hour one (R34, R39 made permanent). Shared characters and materials are read-only to the other lane. Records are merge=union. |
| F14 | Grouped generation: one reject resent 12 to 43 files | One generated asset per task, always (R32), fix tasks included. |
| F15 | Person cards for defects; silent no-ops on stale cards | Cards are decisions only. A defect becomes a fix task for its owner. Decisions batch at fixed times, and each card has one live button that closes when answered. |
| F16 | False DONEs; an invented human gate | Outcome by artifact, not by text. DONE needs its listed artifacts present and checked. "Failed", "not closed" or "blocked" anywhere in a DONE makes it blocked. Human gates are an explicit list in the plan, and nothing else is one. |
| F17 | The Fable usage limit at 23:06; Playwright's 50 MB upload cap; no butler |
Capacity probe in hour one. Model quotas and reset times, upload paths and sizes (butler installed, CDP local upload), store limits. The rung falls back automatically when a quota is hit. |
| F18 | No token data in the runner | Usage in every run record (section 3). |
| F19 | Mesh to MetaHuman ran 17 h wall time for a feature that did not ship | Long, risky features start on day one, split one per character, with a kill date written in the plan. |
| F20 | Clock drift in hand-written log stamps | Stamps come from the system clock, written by the logging helper, never typed. |
Token efficiency
Where the tokens went (measured)
Claude Code agents over the jam: 3.54 B cache-read tokens, 92.1 M fresh input and 14.2 M output, across 17,134 messages (tokens.txt). opencode and Codex are not included, because they recorded no usage.
| Where | Measure | What it tells us |
|---|---|---|
| The base context | median 81,770 tokens at the first turn of a runner session (p90 90,128); median 21 turns | About 80 k of fixed brief, rules and tool schemas is re-read every turn. As an estimate that is about 40 % of all cache-read tokens. |
| Review and repair | 1st AD reviews 502 M plus fix rounds 802 M = 37 % of cache-read | Checking and fixing cost as much as making (36 %). |
| The orchestrator | 947 M, 27 % of cache-read, in one long session | One growing history re-read every turn. |
| Opus on non-release work | Opus 2,160 M cache-read, 61 % of the total (947 M of it is the orchestrator) | Outside the orchestrator, the strong model also carried records, merges and verification that a cheaper model could do. |
| Retries | 145 blocked, 43 failed and 11 requeued records out of 648; 107 fix tasks out of 345 completed | Every failed attempt pays the full base context again. |
The mechanisms
- Measure first. The runner records
usage(input, cache creation, cache read, output, model) on every attempt from the agent's own output. With no measure there is no budget. This is the first thing to build. - Shrink the base brief. The target is under 25 k tokens before the first action (estimate of what is reachable). Ways to get there:
- the task brief carries only the rulings that apply to its lane and asset type;
- estate rules go in by pointer, with a short brief-sized digest;
- MCP tool schemas are loaded per task type (a sound task does not need the Unreal level toolset);
- long descriptions are cut to a one-screen template.
- Size tasks to the context. One asset or one change per task, finishing in about 20 turns. A task that passes 60 turns is split by the runner, not continued.
- Route by task type. A table in the plan maps a task verb to a runner:
- generation, records, docs and page text go to the cheap tier;
- making in the editor goes to the strong rung;
- release, package and git surgery go to the strong rung, pinned for every attempt;
- review goes to a strong model that did not make the work. A cheap model never takes a task whose failure costs a package.
- Review once, then decide. The 1st AD gets one round on a fix by default, and at most two. After the cap the item is accepted as a known issue or escalated to the owner task, never looped. Its brief carries only the files under review and the bible items they touch.
- Stop work after a freeze. No review, fix, repost or recovery runs against a frozen artifact (F6). On 15 Nov that would have saved the M16.2 re-cuts and the four round-2 rc1 fixes.
- Cache knowledge, not transcripts. Answers that were found once go into the project template and the frozen knowledge set, never rediscovered in a task. Examples:
- the editor-only skeleton;
- the white-field cause;
- the loopback capture;
- the CDP upload path;
- the lossless compression switch. The runner injects the matching entry by asset type.
- Budgets per task. Each task carries a token budget from its type, for example 1.5 M cache-read for an asset fix and 6 M for a feature (estimates, to be set from the first measured week). At 80 % of budget the agent is asked to commit and summarise. At 100 % the runner stops it and posts a smaller follow-up.
- Keep the orchestrator short. The orchestrator works from state files (live board, queue, log tail), not from its own history. A fresh orchestrator session per shift, seeded from a one-page handoff, replaces one session that grows for days.
- Retry with a reason, not a rerun. A retry carries the failure's first error and the changed condition. A second identical failure is closed and escalated, never tried a third time on the same model.
Ship gates, from hour one
Each gate runs automatically, logs its measurement and blocks the right thing.
| Gate | Runs on | Measures | Blocks |
|---|---|---|---|
| Size budget | every package | the real submission zip size against the store limit and margin | landings that exceed the projection; the release task |
| Cook safety | every content landing | references to editor-only paths; a cook of each new asset type | the landing |
| Package smoke | landings touching maps, lighting or C++, and nightly | boot, luma per map, frames looked at, FPS on the test device | the release task |
| Page drift | every landing that adds a tool, source or licence | the store text against the asset manifest | the release task |
| Capture | day one, then each video | loopback audio present; loudness in spec; resolution and length | the upload |
| Quotas and paths | hour one, then daily | model limits and resets, upload paths and sizes, store limits | nothing; it raises the fallback early |
Build plan, in order
All efforts are estimates for an agent-built implementation with review.
| Step | What | Effort |
|---|---|---|
| 1 | Usage capture in run records; a token report per task, model and kind | 0.5 day |
| 2 | The id module: allocate, parse, refuse; task-id dependencies; goal keys and one owner per goal | 1 day |
| 3 | Hot reload of code and plan at a save point; a read-only dry run; a stop and restart that clean locks and rescue files | 1.5 days |
| 4 | Freeze state: frozen hashes in the plan; reviews become notes; recovery and post-time refusal | 0.5 day |
| 5 | The routing table by task verb; pins that last for every attempt; a quota fallback | 1 day |
| 6 | Ship gates: size budget, cook safety (reference scan plus first-of-type cook), package smoke with luma, page drift | 2 to 3 days |
| 7 | The project template: lossless compression, capture recipe with loopback and mastering, .gitattributes union for records, lane map ownership, side slots |
1 day |
| 8 | Brief diet: a rulings filter, rules digest, per-type tool loading, a description template; measured before and after | 1 to 2 days |
| 9 | Budgets per task type, the 1st AD round cap with known-issue acceptance, retry with a reason | 1 day |
| 10 | Process and asset ownership: launched processes and loaded assets closed on exit; editor clean before claim | 1 day |
| 11 | The orchestrator shift model: state-file driven, a fresh session per shift with a one-page handoff | 0.5 day |
| 12 | A dry rehearsal: a 24-hour mini-jam on a small theme, measured against the jam's numbers | 1 day plus the run |
Total: about 12 to 14 working days before the rehearsal. Steps 1 to 4 alone remove most of the 15 Nov waste, and are about 3.5 days.
What success looks like
These are targets for the rehearsal and the next jam, not claims.
- No ship requirement is discovered after content lock. Every gate in section 4 has been green or red since hour one.
- Zero restarts that kill live work.
- Zero tasks run against frozen artifacts.
- Base context under 25 k tokens at the first turn, and cache-read tokens per completed task at least halved against the jam. The jam's figure is about 10 M per completed task over Claude Code work: 3.54 B over about 345 completed tasks, an estimate that includes the orchestrator.
- The person sees decisions only. No card carries a defect.