Published result
A bounded follow-up leaves the component update unqualified
A plain-language summary comes first, followed by the full text of the result's reproduction pack.
Result summary and pack text · reproduce/kogen-coherent-set-c92d897-confirmation/README.md
- Finding id:
kogen-coherent-set-c92d897-confirmation - Verdict: No
- Finished: 3 November 2026
- Comparison: Kogen shaping and delivery-build speed
- Reproduction pack: reproduce/kogen-coherent-set-c92d897-confirmation/ (data, analysis script and rerun steps)
- Category page: categories/harness-levers.md
In plain terms
Two follow-up attempts produced one valid Intent and one landed change, but neither passed every hidden check. A generated acceptance contract had undefined steps, triggering the registered stop before a third launch. Both historical-timeout readings retain a readiness harm. The original screen and every harm remain in the pack; no component is qualified and the confirmation is closed without adoption.
Verdict key: Yes, The result supports the change or claim that was tested. No, The result does not support it, so nothing changes. Unresolved, The evidence is not strong enough either way. Superseded, A later test replaced this result.
Full text of the reproduction pack
The text below is the pack's README as published in the repository (a system folder name may show without its leading slash, so that the site's privacy scan passes).
This pack retains the original screen and its bounded follow-up. The candidate is the exact committed assembly linked in source-commits.json. No component is qualified by an isolated mechanism claim. Whole-task hidden pass is primary; valid Intent, acceptance separation, landing and review are separate outcomes.
The original screen assigned nine full-pipeline tasks and three shape-only attempts. The follow-up permits at most three candidate attempts on existing compared tasks with historical Build/Review reach. Two start first; the third is conditional. Stop after two upstream/infrastructure stops or the first broken contract. Refusals and questions count as upstream stops. All earlier harms stay primary, including the deployment readiness harm. No useful gate is weakened.
All roles use gpt-6.1-terra medium through native OpenAI. Comparators are frozen stored outcomes for the adopted release or an identical source tree, with the same tasks, model and effort. They are historical comparisons; task selection, runtime differences and stochastic variation prevent a causal rescue claim. Timeout-sensitive readings are reported both ways. This is a small bounded confirmation, with no significance or equivalence claim.
Rerun
Run python3 report.py to replay the retained data without model calls. The
expected output is in expected-output.txt. Source-commits.json links the public
commits of the coherent candidate and every component. The treatment is that
committed assembly; there is no additional patch. Fresh model execution uses the existing lanes full-pipeline runner on the
qualified frozen task kits, with only the committed candidate and the stated
model lineup. A third attempt is allowed only after the stop-rule check. Exact
task identity hashes,
protocols, logs and evaluator artifacts remain in the evaluator's private
execution receipts. Reproducing model execution requires those qualified inputs
and an existing authorized login; this pack contains no source or task secrets.
Limits and accounting
Wall time includes failed attempts. Cached input is a subset of input; reasoning is a subset of output and is not added twice. Subscription use is unpriced, not zero. Orchestration model usage is unmetered. Earlier builds, setup, grading and failed attempts remain workflow overhead. Acceptance separation remains unknown unless the retained evidence demonstrates it. Hidden-check failures and upstream stops are retained, irrespective of attribution to a component.