What did not work

This page is optional background for the curious: you do not need it to use Gitstalk. Most of Gitstalk's rules exist because something else failed first. These are the ideas we built or measured and then dropped or parked, with the numbers and what replaced them. Times are minutes; “simulated” means our engine simulator with replay agents, over 16 seeds. For the rules that stayed, read Architecture and Concepts.

Land first, fix forward

Expected: Land every change without waiting; a repair agent fixes any break afterwards.
Measured: Repairs cost about 7 times a task, not the 0.4 the simulator assumed. With 12 Sonnet agents only 2 of 40 tasks went green in 40 minutes; with Haiku, green froze.
Instead: A pre-land check on the exact merged tree, and a red goes back to its own author. The red trunk disappeared.

docs/claude-opus/08-validation-steps-1-3.md

Scheduling tasks by predicted footprint

Expected: Predict which modules a task touches from its title, and keep clashing tasks apart.
Measured: Module-level prediction cannot tell risky pairs from safe ones: even a perfect predictor flags 33–74% of safe pairs. Strict placement collapsed to about 2 busy agents; soft placement did nothing.
Instead: Land fast and check the exact tree. Scheduling survives only as dependency-aware starts for known chains and declared couplings.

research/REPORT.md

An early race harness that favoured Gitstalk

Expected: A fair race between Gitstalk and a merge queue.
Measured: Two handicaps. Queue agents had to wait for their pull request to land before taking new work, and CHANGELOG had no union merge driver. Separately, Gitstalk's pre-land check cost the suite's real 2 seconds while queue CI cost 60. With the free check Gitstalk finished in 3.3 minutes against the queue's 16.5, about 5×. With the check costing what CI costs, the lead was 1.5× and 2.5× on two seeds: most of the first gap was the harness.
Instead: Queue agents are released when their change is queued, both sides get the merge driver, checks cost what CI costs, and CI capacity is matched and stated with every run. The fastify races use GitHub's own merge queue.

08-validation-steps-1-3.md §5.2, research/REPORT.md

Two shared sandboxes for every pre-land check

Expected: Two 4-vCPU sandboxes per repository would carry its pre-land checks.
Measured: At 16 parallel workers, 8 suites ran at once in each sandbox and all crossed the 300-second limit, which counted queue time and read a timeout as a red. 101 timeouts became 105 false reds; authors re-pushed together and timed out together again. Gitstalk integrated 33 of 38 in 60 minutes; GitHub did 38 of 38 in 30.
Instead: One sandbox per bean in check, on demand from a shared pool; waiting never counts; a timeout is re-run, never red. Same load afterwards: 38 of 38 in about 10 minutes, with no timeouts, and CI minutes fell from 410 to 54.

18-git-native-flow.md §7.1, loadgen/README.md

All the validated rules at once, without a window

Expected: Rules that each helped in simulation (simulated: 39 of 40) would help together.
Measured: On Cloudflare with real agents: 24 green, 16 dropped. Releasing agents during their checks produced 19 landings in one minute, validation fell 27 commits behind, and innocent beans burned their rework rounds. Replay agents had not reproduced the burst.
Instead: The sprout window (at most W landings ahead of validation), which recovered 34 green, and the lesson that replay agents miss bursts, so rules are confirmed in real-agent races before they become defaults.

11-experiments-summary.md (CF v2.2, v2.3)

Live sync of landed beans into agents' work

Expected: Merging newly landed beans into agents' work in progress would prevent clashes.
Measured: Simulated. Syncing only overlapping beans fired almost never and changed nothing. Syncing every landed bean cost 16–36 extra agent turns per 40-task race and finished up to 2 minutes later. Even an optimistic bound saved under one red check per race.
Instead: Off by default. Informed repair after a red fixes the same clash for about one check's latency.

11-experiments-summary.md (2b, live sprout sync)

Promotion by evidence, with traced read maps

Expected: Skip CI for a sprout commit when no changed file is read by any test whose result is already known; this removed every window wait in simulation.
Measured: Safe: traced read sets caught every breaking mutation we tried, and audits caught every planted miss. But file-level read sets are too coarse on real suites: a test that boots the app reads nearly every module, so about 80% of the shop arena's tests and 94% of fastify's are affected by any change. Live races were no faster, and on fastify used 1.7–2.6× the CI minutes.
Instead: Parked, off by default, code kept. Check reuse covers the exact-commit case. Function-level coverage is the untested next lever.

11-experiments-summary.md (event-driven promotion), research/test-impact

Validation debounce

Expected: Batching validations for a few seconds would save CI without slowing anything.
Measured: Simulated. A quarter to half fewer validations and 21–30% fewer CI minutes, but beans waited in the window for the debounce: some races finished 1.2–2 minutes later.
Instead: Off. Fewer CI runs were not worth a later finish.

One ticket per red episode, repair landings, speculative validation, an adaptive window

Expected: Each would shorten the 30-agent stall.
Measured: Simulated. One ticket per episode alone made it worse: the unrevertable red stayed and the run finished at 41 minutes instead of 31. Repair landings past the window never fired. Speculative validation and a window that drops after repeated reds were identical to the reset alone. Keeping the green prefix in a partial reset gave nothing either.
Instead: The red-window reset alone: sprout green again about 1.3 minutes after a red; the others are off or were never merged.

11-experiments-summary.md (stall fixes)

Plugin guidance v1: push and move on

Expected: Agents told not to block on their own checks would integrate sooner.
Measured: The guidance took (51 plain pushes, none blocking), but workers replaced blocking pushes with sleep 45 to sleep 90 polls, and the lead sat in a blind sleep 420. The last task went green at 28 minutes against 22 with the blocking prompt; one run each, and checks were slower that hour.
Instead: Event-driven waits: refs/wait/any and refs/wait/all in git, and bean_wait over MCP, woken by the verdict; a lead that ends its turn and acts on worker notifications.

research/race/ORCHESTRATED.md

Containers sharing one ChatGPT login for Codex

Expected: Run cloud Codex agents on an existing ChatGPT subscription.
Measured: Ruled out before building, from OpenAI's own documentation: refresh tokens rotate on use, so copies of one login log each other out, and the docs say not to share one across concurrent jobs. The plan's allowance could not feed 60 agents anyway.
Instead: API keys for measured races, added outside the container so the agent never sees them; one seat leased to one container at a time for small runs.

17-cloud-agent-swarm.md §4

Every experiment, including the ones that worked, is in docs/claude-opus/exp.