> Data revision: baf7c2f4307d3a989a8fb9363df8000fda3d9310
> Source files:
- [reproduce/p4-ledger-go-screen/README.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/p4-ledger-go-screen/README.md)
- [categories/harness-levers.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/categories/harness-levers.md)
- [site/content/finding-pages.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/site/content/finding-pages.json)

# A coverage ledger in Kogen shaping did not show a gain on the first 13 Go tasks

- **Finding id:** `p4-ledger-go-screen`
- **Verdict:** Unresolved
- **Finished:** 31 October 2026
- **Comparison:** [Kogen shaping and delivery-build speed](/comparisons/kogen-shaping-and-builds/)
- **Reproduction pack:** [reproduce/p4-ledger-go-screen/](https://github.com/KogenAI/kogen-bench/tree/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/p4-ledger-go-screen) (data, analysis script and rerun steps)
- **Category page:** [categories/harness-levers.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/categories/harness-levers.md)

## In plain terms

A patch adding a model-free coverage ledger (a check of the request against the submitted Intent, shown to the reviewer) was compared with unpatched Kogen, same commit and model (gpt-6.1-terra, medium effort), one trial per task. On 13 Go tasks the hidden checks passed on 9 tasks both without and with the patch: one task was gained, one was lost (the lost one ended with no model response during the build), against a keep rule of at least three net wins. One repetition is still to finish and will be pooled with this run before the patch is kept or dropped.

Verdict key: **Yes**, The result supports the change or claim that was tested. **No**, The result does not support it, so nothing changes. **Unresolved**, The evidence is not strong enough either way. **Superseded**, A later test replaced this result.

## Full text of the reproduction pack

The text below is the pack's README as published in the repository (a system folder name may show without its leading slash, so that the site's privacy scan passes).

A Kogen patch that adds a model-free coverage ledger to shaping did not show a gain on its first Go run. **On 13 Go tasks unpatched Kogen passed the hidden checks on 9 and the patched build on 9**: one task gained, one lost, net 0, against a keep rule of at least +3. Verdict: **unresolved**: one repetition of the same patch and tasks is still to finish and will be pooled with this one; if the pooled net is below +3 the patch is dropped. Finished 31 October 2026.

## What was run

Same Kogen commit 3e759e88 (vendored Loop pin 8a2fcdae) in both arms, model gpt-6.1-terra at medium effort for shaping, building and review, request-only tasks, one trial per arm per task, cells interleaved, one Linux laptop-class host.

| Arm | What differs | Patch sha256 |
| --- | --- | --- |
| b0 | nothing (empty patch) | none |
| p4 | coverage ledger: a model-free check of the request against the submitted Intent, shown to the reviewer (`patch/p4-ne1-ledger-b0.diff`) | a700d61caac3e9bc8b7d349f4f45ca9016f09da02bbe5bbb97bb0a43ae9e645b |

## Result

| Arm | Hidden pass | Wall s | Tokens | Requests |
| --- | --- | --- | --- | --- |
| b0 | 9/13 | 3025 | 2976453 | 192 |
| p4 | 9/13 | 3761 | 3201401 | 205 |

Wall, tokens and requests are summed per cell and **include cells that ended refused, build_blocked or unavailable**. Two pairs differ: G10 fails unpatched (build_blocked) and passes with the patch (landed); G12 passes unpatched and fails with the patch because the build stage got no model response (outcome `unavailable`, a service failure rather than a wrong implementation). G03, G08 and G13 fail in both arms. Outcomes: unpatched landed 10, build_blocked 2, refused 1; patched landed 11, refused 1, unavailable 1.

## Limits

- One trial per arm per task; a one-task difference is within run-to-run variance.
- Intent validity and acceptance separation are not scored here; the numbers are hidden-check results only. The model has no listed price, so cost is token counts.
- Cold-cache laptop-class host, so wall times do not compare with other hosts.

## Reproduce the numbers

```sh
python3 -I reproduce/p4-ledger-go-screen/report.py reproduce/p4-ledger-go-screen/data p4-ledger-go p4 | diff - reproduce/p4-ledger-go-screen/expected-output.txt
```

`report.py` uses only the standard library. `data/p4-ledger-go_cells.tsv` has one row per cell (G01 to G13 are the tasks in alphabetical order; names and hidden suites are not included).

## Rerun

The table reruns from `data/`. To run the cells again you need Kogen at 3e759e88 with `patch/p4-ne1-ledger-b0.diff` applied, the same 13 Go tasks and a ChatGPT login.
