> Data revision: baf7c2f4307d3a989a8fb9363df8000fda3d9310
> Source files:
- [rounds/l0-reconcile/README.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l0-reconcile/README.md)
- [rounds/l0-reconcile/RESULTS.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l0-reconcile/RESULTS.md)
- [results/run-records/index.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/run-records/index.json)
- [rounds/index.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/index.json)
- [results/source-crosswalk/index.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/source-crosswalk/index.json)
- [rounds/STATUS.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/STATUS.md)
- [reproduce/round-inventory.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/round-inventory.json)
- [reproduce/reconcile_l0.py](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/reconcile_l0.py)

# L0 historical claim reconciliation

**Round ID:** l0-reconcile  
**Status:** VALID

> Reviewed status from rounds/STATUS.md: VALID.

> Round README and results text are source material. Numeric values not tied to a claim ledger record remain source-reported.

Round date: HISTORICAL (before 2026-10-09)
Publication badge: VALID; KEPT FOR AUDIT
Recomputation status: NOT RECOMPUTABLE


## Status

**VALID** — reconciliation only

### Limits
- Restored to VALID 2026-10-08: the round executed its registered rule as registered and its numbers recompute; the points below are limits on public evidence, not contradictions in the round's own files.
- **PRE_REGISTRATION_NOT_APPLICABLE:** This retrospective reconciliation has no pre-registered result rule.
- **EVIDENCE_INCOMPLETE:** The full R74 scored cohort is not reconstructable from retained public records.


## Lifecycle counts for this lane

This reconciliation added no benchmark cells; the historical cohort counts above belong to their source rounds.

| Lifecycle field | n | Basis |
|---|---:|---|
| Planned | 0 | No new benchmark cells were planned in this reconciliation lane. |
| Started | 0 | No new benchmark cells were launched. |
| Finished | 0 | No new benchmark cells were run. |
| Officially graded | 0 | No new benchmark cells were graded. |
| ITT denominator | 0 | This lane generated no new benchmark outcomes. |


## Required reproduction metadata

- Kogen commit: [260a73bc06be16e563cb4e56a168ffd078cec677](https://github.com/KogenAI/kogen-ex/commit/260a73bc06be16e563cb4e56a168ffd078cec677) (the exact pin recorded in the cited round source, resolved against local Kogen history).
- Harness commit: not recorded in the cited round source as a Git commit; any adapter or runtime fingerprints are not Git commits.
- Historical execution command: not recorded in the cited round source.
- Raw records: the cited sources do not provide a complete public round-specific request and grade bundle; public delivery and outcome exports are linked above where applicable.


STATUS: **VALID** (reconciliation only)

Pre-registered: no new benchmark cells; this lane reconciles frozen historical receipts.

Label: DESCRIPTIVE

Question: Do the R72, R74, R69, and R57 historical claims match their recoverable cell level receipts?

n: R72 P1 168 cells and partial P3 24; R74 167 scored rows plus 9 smoke attempts; R69 400 assigned rows; R57 pooled 311 scored cells plus one unrecovered planned slot.

Headline: R72 P1 and R69 counts reproduce exactly; R57 counts and published interim bound reproduce; R74 has 167 internal scored grades but no public scored export, and its full retained cohort is not reconstructable.

Configuration: Historical configurations are retained per row in the cohort CSVs. R72 direct Codex CLI 0.160.0; R74 direct Codex CLI 0.160.0. R69 and R57 used `kh-gpt`; no separate harness version or Codex CLI version was recorded. Effective model and effort are listed per cell; NOT-RECORDED and NOT-RUN are explicit.

Limit: This reconciliation does not establish new performance estimates. R74 has 231 retained slots without recovered cell IDs or terminal statuses. R57 has one missing shell-only slot, excluded from the scored denominator. R57’s source declaration specified stratification but not an estimator; this page recomputes the analyst’s published estimator. The R74 cohort is withdrawn and mixed across time, venue, and configuration.

Sources: `ROUNDS.md`; `LATEST.md`; R72 round README and decision rule; R74 `results/run-records/index.json`, `CODEX-POOL.md`, amendment, status, and exclusion receipts; R69 `itt.jsonl`, decision rule, and verdict; `R57-POOLED.md`; official `grades.final.jsonl` and `cells.jsonl`; the probe grade ledger; and `R2-EXPERIMENTS-LEVERS.md`, `GAPS.md`, and `MASTER-MAP.md`. See [reconstructed cohort data](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l0-reconcile/data/) and the [calculation script](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/reconcile_l0.py).

## Reproduce

Run from the `kogen-bench-sot` repository root:

```sh
python3 reproduce/reconcile_l0.py
```

The standard-library-only script reads only the sanitized files below. Cell IDs, arms, tasks, reps, outcomes, stop causes, effective model/effort, and public-grade matches are in these files:

- `rounds/l0-reconcile/data/r72-p1.csv`
- `rounds/l0-reconcile/data/r72-p3-partial.csv`
- `rounds/l0-reconcile/data/r74-graded-subset.csv`
- `rounds/l0-reconcile/data/r74-smoke-attempts.csv`
- `rounds/l0-reconcile/data/r74-coverage.csv`
- `rounds/l0-reconcile/data/r69-itt.csv`
- `rounds/l0-reconcile/data/r57-pooled-itt.csv`

Recorded Kogen commits:

- R72: [260a73bc06be16e563cb4e56a168ffd078cec677](https://github.com/KogenAI/kogen-ex/commit/260a73bc06be16e563cb4e56a168ffd078cec677)
- R74: [cde7a380455e9793cb1fc8315791bc9beffb863b](https://github.com/KogenAI/kogen-ex/commit/cde7a380455e9793cb1fc8315791bc9beffb863b)
- R57: [6747368333e55a694338d32312eb1f49ec2b4813](https://github.com/KogenAI/kogen-ex/commit/6747368333e55a694338d32312eb1f49ec2b4813)
- R69: no Kogen source commit recorded.

Harness/CLI receipts: R72 and R74 direct Codex CLI `0.160.0`; Kogen versions are pinned by the commits above. R69 and R57 `kh-gpt`; a separate version string was not recorded. The per-cell CSVs contain the observed effective model and effort, including role maps for Kogen cells.

Task IDs represented in the recovered data:

- R72 P1 (12): `elx-02-ingest-supervision`, `elx-04-queue-backpressure`, `elx-07-cli-stats`, `elx-12-retry-api-deprecation`, `elx-port-board-publish-unpublish-public-boundary`, `elx-port-erase-account`, `syn-01-live-ticket-filters`, `syn-06-migration-ticket-numbers`, `syn-15-validation-ticket-form`, `syn-20-email-invite-flow`, `syn-24-csv-import`, `syn-31-inbound-email-webhook`.
- R72 partial P3 (9): `elx-02-ingest-supervision`, `elx-04-queue-backpressure`, `elx-12-retry-api-deprecation`, `syn-01-live-ticket-filters`, `syn-06-migration-ticket-numbers`, `syn-15-validation-ticket-form`, `syn-20-email-invite-flow`, `syn-24-csv-import`, `syn-31-inbound-email-webhook`.
- R74 scored subset (12): `elx-02-ingest-supervision`, `elx-04-queue-backpressure`, `elx-07-cli-stats`, `elx-12-retry-api-deprecation`, `elx-port-board-publish-unpublish-public-boundary`, `elx-port-erase-account`, `syn-01-live-ticket-filters`, `syn-06-migration-ticket-numbers`, `syn-15-validation-ticket-form`, `syn-20-email-invite-flow`, `syn-24-csv-import`, `syn-31-inbound-email-webhook`. This is the recovered scored subset; task IDs for the 231 ungraded retained slots are not recovered.
- R69 (25): `elx-02-ingest-supervision`, `elx-04-queue-backpressure`, `rails-ac-throttle-search`, `rails-aj-enqueue-after-commit`, `rails-ar-archive-book-access`, `rails-ar-bulk-access-grants`, `rails-as-variant-processed-once`, `rails-ft-active-storage-tracking`, `rails-ft-activity-feed-api`, `rails-ft-board-publish-unpublish-public-boundary`, `rails-ft-cancelled-account-cleanup`, `rails-ft-entropy-sweep`, `rails-ft-i18n-support`, `rails-ft-mysql-fulltext-search-foundation`, `rails-ft-notification-bundle-window-overlap`, `rails-ft-saas-billing`, `rails-hw-scoped-broadcast`, `rails-sec-audit-sweep`, `rails-sup-legacy-conversions`, `syn-01-live-ticket-filters`, `syn-06-migration-ticket-numbers`, `syn-15-validation-ticket-form`, `syn-20-email-invite-flow`, `syn-24-csv-import`, `syn-31-inbound-email-webhook`.
- R57 pooled (11): `elx-05-cache-single-flight`, `elx-07-cli-stats`, `elx-12-retry-api-deprecation`, `rails-ac-throttle-search`, `rails-aj-enqueue-after-commit`, `rails-ar-bulk-access-grants`, `rails-as-variant-processed-once`, `rails-ft-mysql-fulltext-search-foundation`, `rails-hw-scoped-broadcast`, `syn-13-bug-empty-filter-crash`, `syn-14-bug-sla-business-hours`.


## Results source

# Results

Exact cell IDs, task IDs, reps, arm assignments, outcomes, stop causes, effective model/effort, and public grade matches are recorded in the linked sanitized CSVs.

## R72: 64/84 versus 60/84; stopped early

| Recovered cohort | Cell outcomes | Public export | Reconciliation |
|---|---|---|---|
| P1 Kogen Luna ladder, n=84 | 64 PASS, 19 FAIL, 1 INVALID | 0 matching public scored rows | **Reproduced exactly**: 64/84. Invalid remains in denominator and is not a pass. |
| P1 Codex Luna max, n=84 | 60 PASS, 24 FAIL | 0 matching public scored rows | **Reproduced exactly**: 60/84. |
| P3 partial, n=24 | Kogen ladder 12/13; Codex Sol high 10/11 | 0 matching public scored rows | Partial follow-on only; not included in the P1 claim. |

The stopped-early description is consistent with the recovered record. One invalid Kogen P1 row has no effective model/effort receipt; its row is marked `NOT-RECORDED` in [P1 data](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l0-reconcile/data/r72-p1.csv). The cell-level P1 and partial P3 rows are [here](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l0-reconcile/data/r72-p1.csv) and [here](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l0-reconcile/data/r72-p3-partial.csv). Source cohort label: INTERIM.

## R74: stopped; no public scored export

| Recovered cohort | Count | Reconciliation |
|---|---:|---|
| Amendment C retained specs | 398 | Receipt count reproduced. |
| Internal scored grades | 167 | Recovered in the internal grade ledger; 37/57 Codex Luna max, 11/11 Codex Sol high, 51/60 Kogen Luna ladder, 33/36 Kogen Sol high ladder, 3/3 Kogen Sol medium ladder. |
| Smoke attempt grade rows | 9 | 8 selected smoke rows and 1 withdrawn environment fault; excluded from the 167-cell scored subset. |
| Public scored export | 0 | Claim reproduced as a statement about public publication. |
| Retained slots without scored grade | 231 | Exact cell IDs and individual terminal statuses are not recovered. |

The claim is **true for the public export**, while internal scored grades exist. Full retained-cohort ITT is **not reconstructable** from the captured receipts. This source round was withdrawn. See [graded subset](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l0-reconcile/data/r74-graded-subset.csv), [smoke attempts](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l0-reconcile/data/r74-smoke-attempts.csv), and [coverage record](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l0-reconcile/data/r74-coverage.csv). Source cohort label: WITHDRAWN.

## R69: SPEC / APPROACH / ACCEPT / NONE

Source-reported, not reproducible from the public export: PASS / retained ITT was SPEC 63/99, APPROACH 66/99, ACCEPT 66/98, and NONE 59/100. The listed intervals are source-reported in the linked cohort reconciliation.

| Pairwise comparison | Difference | 95% Newcombe interval | One-sided task-stratified exact p |
|---|---:|---:|---:|
| APPROACH − SPEC | +3.03 pp | −10.11 to +16.02 pp | 0.252423469388 (1979/7840) |
| ACCEPT − SPEC | +3.71 pp | −9.45 to +16.68 pp | 0.197455412923 (5309813/26891200) |

The retained ITT contains 327 official grades and 69 ungraded ITT failures; four documented infrastructure exclusions were not replaced. Rejoining the refreshed 2026-10-07 official export by exact `cell_id` finds 150/327 grades (113 pass, 37 fail); [the pinned export](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/inputs/grades.final.jsonl) has SHA-256 `4d639a3848b3661dcb5646630b6a4914e5053ff4b94ab029424d47d040e3c977`. The prior 95/327 count applies only to the export snapshot at benchmark commit `58c9ca5ee40fb958c61c61e381191a922886f88f` (2026-10-07T16:18:30+03:00; export SHA-256 `cbf85effb6334c650fa0204da6e2777ed8a5a699f14ab854529d998b5177703a`) and is superseded. The four-arm counts reproduce from the linked cohort CSV, but the row outcomes remain source-reported. The attempted configuration was `kh-gpt`, shell-only, Luna requested max clamped to effective xhigh, 64 turns/3600 seconds; per-cell model and effort are marked `NOT-RUN` where there was no effective run receipt. See [R69 ITT data](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l0-reconcile/data/r69-itt.csv). No separate Kogen source commit is recorded.

## R57 pooled shell-only versus default tools

Source-reported, not reproducible from the public export: PASS / scored valid cells was shell-only 114/155 and default tools 111/156; public grade matches were 155/155 and 156/156, respectively.

| Method | Difference | One-sided 95% lower bound | −10 pp rule |
|---|---:|---:|---|
| Published host × task weights; stratified Newcombe/MOVER | +1.92 pp | −4.70 pp | Met |
| Equal-stratum sensitivity | +0.00 pp | −7.70 pp | — |

Source-reported, not reproducible from the public export: the counts and published interim bound reproduce in the linked cohort CSV. One planned Studio shell-only slot is missing; it has no recovered cell ID, grade, or terminal cause and is not counted as a failure. The source declaration specified the strata but not the estimator; these interval values reproduce the analyst-selected method documented in the pooled snapshot. Source cohort label: INTERIM. See [R57 pooled data](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l0-reconcile/data/r57-pooled-itt.csv). Effective configuration for scored cells: `kh-gpt`, GPT-6 Luna, xhigh; shell-only exposes bash, default exposes read/write/edit/bash.


## Captured run records

No records for this round are listed in the [run-record index](/downloads/results/run-records/index.json).


## Reproduction inventory

- **Type:** analysis-only
- **Script:** `python3 reproduce/reconcile_l0.py`
- **Inputs:** `rounds/l0-reconcile/data/r72-p1.csv`, `rounds/l0-reconcile/data/r72-p3-partial.csv`, `rounds/l0-reconcile/data/r74-graded-subset.csv`, `rounds/l0-reconcile/data/r74-smoke-attempts.csv`, `rounds/l0-reconcile/data/r74-coverage.csv`, `rounds/l0-reconcile/data/r69-itt.csv`, `rounds/l0-reconcile/data/r57-pooled-itt.csv`
- **Full execution replay:** unavailable
- **Scope:** Recomputes selected reconciliation CSV summaries only; it does not replay historical model executions or complete the canonical official outcome join.

See the [complete per-round reproduction inventory](/reproduce/) and the [repository reproduction guide](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/README.md).

## Related records

### Related hypotheses

- [H113](/hypotheses/h113/)
- [H115](/hypotheses/h115/)
- [H136](/hypotheses/h136/)
- [H138](/hypotheses/h138/)

### Related experiments

- [l0-reconcile](/rounds/l0-reconcile/)

### Related claims

- [CL-WA-r57-source-reported-denominator-variants](/claims/cl-wa-r57-source-reported-denominator-variants/)
- [CL-WA-r69-source-reported-denominator-variants](/claims/cl-wa-r69-source-reported-denominator-variants/)
- [CL-WA-r72-lifecycle-and-outcome](/claims/cl-wa-r72-lifecycle-and-outcome/)
- [CL-WA-r74-source-reported-denominator-variants](/claims/cl-wa-r74-source-reported-denominator-variants/)
