Watch it catch a bad change
A real run from Evolve's ledger (its permanent record of everything it does), replayed. A coding agent, an AI tool that writes code, makes an online shop's checkout faster and says every coupon still works. The checks agree. Real traffic, the requests real people send, doesn't.
- note A goal arrives: make checkout fast, and keep every price right. Fast means p95 under 150 ms: the slowest 5% of requests answer within 150 milliseconds, measured against the current version, the baseline. The agent here is a fixture, a script standing in for a coding agent, so the demo needs no AI. The lines in typewriter type are what Evolve recorded; boxed notes like this one explain them.
- 19:52:04 run.created objective checkout-coupon-index · agent fixture
- 19:52:04 run.pinned base c0f48cc6e4c6 · policy 9eb59206ca42
- note The rules are locked now. Nothing the agent does can change the tests it will be judged by.
- 19:52:04 run.state_changed created → exploring
- 19:52:04 attempt.created attempt #1 memoize-and-index-coupons
- 19:52:05 attempt.result_recorded sha 8ab91e15c904 · 1 file, 23 lines
- claim · untrusted “all coupon behaviour is preserved”
- claim · untrusted “p95 checkout latency drops below 150 ms”
- note Those are the agent's words. Evolve records them as untrusted, and they play no part in the decision.
- 19:52:05 run.state_changed exploring → verifying
- note Now the checks: tests Evolve runs itself, which the agent can't touch. The last one times checkout.
- 19:52:14 evidence.recorded build: pass
- 19:52:14 evidence.recorded correctness: pass
- 19:52:14 evidence.recorded checkout_contract: pass
- 19:52:14 evidence.recorded checkout_latency: pass · p95 2.5 ms (baseline 172.1 ms)
- note Every check passes, and checkout drops from 172 ms to 2.5 ms. A busy person reading the change would accept it. Evolve picks it, and keeps the known-good version (the last one proven fine) ready to fall back to.
- 19:52:14 decision.recorded selected attempt #1 · 1 of 1 eligible
- 19:52:14 deployment.created desired 953d1c569e6a · known-good d5fd2d13d609
- 19:52:18 deployment.state_changed pending → canary
- 19:52:18 deployment.traffic_shifted stage 0 → 5%
- note Now real traffic. 5% of checkout requests go to the new version, called the canary; the rest stay on the one known to be good, the baseline. Evolve watches both, side by side.
- 19:53:04 observation.recorded 5%: fail · 31 canary / 573 baseline req · canary error rate 0.0968 exceeds 0.0010 (3 of 31 requests)
- note 3 of 31 real requests failed. They used a coupon that no test had tried.
- 19:53:04 deployment.state_changed canary → rolling_back
- 19:53:04 deployment.rolled_back removed 953d1c569e6a · restored d5fd2d13d609 · observed serving d5fd2d13d609, healthy · canary weight 0
- 19:53:04 run.state_changed observing → rolled_back
- note Back to the known-good version in the same second, checked against what is actually running. Nobody had to be woken up to fix it.
state —
checks —
traffic —
watching —
19:52:04
Tests passed. Traffic disagreed.
- The change passed four trusted checks, including the contract (real checkout requests with known right answers), and was still wrong. Only real traffic could show it.
- Only 5% of requests could reach it, for 46 seconds. Evolve compared them with everyone else in the same minute, then put the known-good version back on its own.
- Every event line above is from the run's permanent record. On your own machine, the evolve events and evolve why commands show the same.
Next: how rollbacks work, how agents earn the right to ship alone, or run the same demo on your machine.