Comparison

Kogen shaping and delivery-build speed

Measurements of Kogen's own shaping step (turning a request into an agreed specification) and of where time goes in its delivery builds. Each result tests Kogen against itself or reads recorded logs; no other agent is compared.

Comparison · site/content/finding-pages.json

Measurements of Kogen's own shaping step (turning a request into an agreed specification) and of where time goes in its delivery builds. Each result tests Kogen against itself or reads recorded logs; no other agent is compared.

Results, newest first

Result Verdict Finished
Six of nine Kogen test suites exceed their time budgets on a quiet host Unresolved 7 November 2026
On macOS, Go tests started detached are scanned by XProtect; attached to ssh they are not Yes 6 November 2026
The tests request's time budgets cannot yet be priced from recorded logs Unresolved 5 November 2026
Unconditional kogen check at landing fails its rule: it refuses good commits and reproduces 1 of 8 breakers No 5 November 2026
Reading a Build's final outcome fixes delivery's misreads Yes 5 November 2026
Batching Kogen's approval step-binding check cuts approval time by over 90% Yes 5 November 2026
Retained verification diagnostics and repair locations Unresolved 3 November 2026
Verification timeout diagnostics remain unconfirmed in task runs Unresolved 3 November 2026
Kogen's landed leftover-process sweep still misses a server started by a launcher that closes file descriptors No 3 November 2026
Verification caller cancellation follow-up Unresolved 3 November 2026
A rule asking Kogen's shaper for a protected property check saved a valid intent in 1 of 10 pairs against 4 for the adopted build, so we dropped it No 3 November 2026
Shaping repair completed paired screen Unresolved 3 November 2026
Shaping repair paired screen Unresolved 3 November 2026
Letting Kogen's shaper decide non-product questions itself looks better on paper, but the change did not cause the gains Unresolved 3 November 2026
With a stronger model, Kogen's data-stores tool and the adopted build both wrote a valid intent in all 13 pairs, and neither passed a hidden suite Unresolved 3 November 2026
Kogen's re-landed data-stores tool wrote fewer valid intents than the adopted build, by the narrowest margin our rule counts No 3 November 2026
Telling Kogen's shaper to decide reversible product questions itself did not get one stuck ticket to an Intent No 3 November 2026
Moving Kogen's project setup step into its own component changed none of its code or prompt text No 3 November 2026
Loop output line boundaries need a triggered confirmation Unresolved 3 November 2026
Three-task Loop evidence wave Unresolved 3 November 2026
Executor policy move cannot compile in the adopted assembly Unresolved 3 November 2026
Coordinator validation feedback screen Unresolved 3 November 2026
A readiness regression claim remains tentative after targeted repeats Unresolved 3 November 2026
A bounded follow-up leaves the component update unqualified No 3 November 2026
A coherent component update remains unqualified Unresolved 3 November 2026
A coherent component update regresses task readiness No 3 November 2026
A change to Kogen's review step that scopes its no-defect check made no difference in 2041 retained reviews No 3 November 2026
Build changes remain unverified after an incomplete screen Unresolved 3 November 2026
A bug in our Agents-on-Rails task converter would have hidden a correct solution's hidden checks on one new task; no published score is affected No 3 November 2026
The adopted Kogen release saved a valid intent in 4 of 12 control runs and passed every hidden check in none Unresolved 3 November 2026
Showing Kogen's builder a small repository's files in its first request cuts its reading steps to zero and build requests by 17%, but passed no more hidden tests (57 of 66 against 58) No 2 November 2026
Kogen's shaper on Claude Opus 5.5 through Claude Code could not be judged: the test stopped after 47 pairs ran with both arms, too few to tell Unresolved 2 November 2026
Kogen's data-stores change was refused at shaping again on a third machine: 0 of 8 valid intents pooled, against 8 of 8 for the build before it Unresolved 2 November 2026
Kogen's data-stores change was refused at shaping in 4 of 4 account-cleanup runs, so its coverage is unmeasured Unresolved 2 November 2026
Kogen split into component modules behaved like the build before it in three matched runs Unresolved 2 November 2026
Letting Kogen's loop wait 45 seconds for shell commands passes the same hidden tests (55 of 78 each) and saves no requests on these tasks No 2 November 2026
Frozen Kogen passed every hidden check in 0 of 18 runs on six Rails tasks, and shaped a valid intent in 6 Unresolved 2 November 2026
Four more runs per task on one machine: current Kogen shaped a valid intent in 3 of 14 counted runs and passed every hidden check in none Unresolved 2 November 2026
Current Kogen passed every hidden check in 1 of 18 runs on six Rails tasks, and shaped a valid intent in 8 Unresolved 2 November 2026
A second block of Kogen on Claude Opus 5.5 passed every hidden check in 4 of 12 runs on six Rails tasks, and 11 of 12 wrote a valid intent Unresolved 2 November 2026
With mise's trust setting applied, Kogen on Claude Opus 5.5 passed every hidden check in 1 of 6 runs on a third server, and 3 runs got no valid intent Unresolved 2 November 2026
With mise's trust setting applied, Kogen on Claude Opus 5.5 passed every hidden check in 2 of 12 runs on six Rails tasks, and 7 runs were still refused at shaping Unresolved 2 November 2026
Kogen on Claude Opus 5.5, third server: 3 of 6 runs landed a build and 1 passed every hidden check; pooled 2 of 18 Unresolved 2 November 2026
Kogen on Claude Opus 5.5 passed every hidden check in 1 of 12 runs on six Rails tasks, and every valid intent it shaped landed Unresolved 2 November 2026
A test-authoring paragraph in Kogen's builder prompt did not shorten builds on four Go tasks No 1 November 2026
Kogen's new shaping example check refuses good specifications and loses 7 hidden passes in 65 shapes No 1 November 2026
Turning on the request audit in Kogen shaping lost badly to unpatched Kogen over two Go repetitions and stays dropped No 1 November 2026
The stop-when-checks-pass sentence in Kogen's builder prompt lost to unpatched Kogen over two Go repetitions No 1 November 2026
The patched builder's losses were caused by defects written before it started, not by finishing early Unresolved 1 November 2026
A coverage ledger in Kogen shaping did not earn its keep over two Go repetitions and is dropped No 1 November 2026
A coverage-ledger patch for Kogen showed no gain on six hard Rails tasks, where no run passed No 1 November 2026
An unstated-requirement-pass patch for Kogen showed no gain on six hard Rails tasks, where no run passed No 1 November 2026
Updating Kogen's built-in loop to tonight's fixes removes refused shell calls but shows no gain in hidden passes (32 vs 31 of 52) Unresolved 1 November 2026
High reasoning effort for Kogen's shaper passes no more hidden tests than medium (57 of 78 each) No 1 November 2026
Kogen's current main passes more hidden tests than the build before its last nine commits (31 vs 23 of 39), mostly because the example check was removed Yes 1 November 2026
Kogen's cold prompt-cache requests are conversation starts, and putting the testing rules first does not warm them No 1 November 2026
Kogen's newer build matched the previous build on 13 Go tasks over three repetitions Unresolved 1 November 2026
Kogen's newer build with the delivered fixes did no worse than the previous build on 13 Go tasks Unresolved 1 November 2026
Which of six ways to reuse or skip checks in Kogen builds could save time Unresolved 31 October 2026
Three shaping changes screened: the second-host half was invalid, the valid half shows no gain Unresolved 31 October 2026
A clearer refusal message for shaping changed nothing: both versions accepted 25 of 26 No 31 October 2026
Allowing only one resubmission broke Kogen's shaping step; allowing five matched the default No 31 October 2026
A cheaper model at medium effort could not get Kogen's shaping step accepted (7 of 78) No 31 October 2026
Switching on Kogen's request audit lost four net tasks on 13 Go tasks, so the patch was dropped No 31 October 2026
A stop-when-checks-pass sentence in Kogen's builder prompt held the hidden result and cut effort on the first 13 Go tasks Yes 31 October 2026
A coverage ledger in Kogen shaping did not show a gain on the first 13 Go tasks Unresolved 31 October 2026
Adding review probes to Kogen lost two net tasks on 13 Go tasks, so the patch was dropped No 31 October 2026
An unstated-requirement pass in Kogen shaping did not show a gain on the first 13 Go tasks Unresolved 31 October 2026
An unstated-requirement pass in Kogen shaping lost to unpatched Kogen over two Go repetitions and is dropped No 31 October 2026
Where the time goes in Kogen builds and shaping sessions Unresolved 31 October 2026
A prototype that skips an already-passed starting check: safe, but small headroom Unresolved 31 October 2026