> Data revision: baf7c2f4307d3a989a8fb9363df8000fda3d9310
> Source files:
- [rounds/l3-repair/README.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/README.md)
- [rounds/l3-repair/RESULTS.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/RESULTS.md)
- [results/run-records/index.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/run-records/index.json)
- [rounds/index.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/index.json)
- [results/source-crosswalk/index.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/source-crosswalk/index.json)
- [rounds/STATUS.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/STATUS.md)
- [reproduce/round-inventory.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/round-inventory.json)
- [rounds/l3-repair/reproduce/l3_repair.py](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/reproduce/l3_repair.py)

# L3 actionable self-verification repair

**Round ID:** l3-repair  
**Status:** DESCRIPTIVE

> Reviewed status from rounds/STATUS.md: DESCRIPTIVE.

> Round README and results text are source material. Numeric values not tied to a claim ledger record remain source-reported.

Round date: HISTORICAL (before 2026-10-09)
Publication badge: DESCRIPTIVE; KEPT FOR AUDIT
Why not VALID: The historical inventory retains DESCRIPTIVE because of REGISTERED_THRESHOLD_NOT_MET. See the round evidence and limitations below.
Recomputation status: FULLY RECOMPUTABLE


## Required reproduction metadata

- Kogen commit: not applicable; the scored arm is direct Codex. The public task bases are Kogen-ex commits listed below.
- Harness commit: no harness Git commit is recorded. The runner tree fingerprint on both workers is `7e7a8d406e99436bce20de4f1f1347e9dfa91e73b9b89759e8887cd073faf6d4`; contestant wrapper SHA-256 is `08019d8aab7c15481f6d14126645eebd7b2c2c98025f94118baeee2167e4a300`.
- Model and effort: direct Codex `gpt-6-luna` at `max`; Codex CLI `0.160.0`.
- Runner: Python `3.14.7`; 3,600-second cap; zero retries.
- Task IDs: `r70-2-elixir-l3r31`, `r70-2-elixir-l3r33`, `r70-2-go-l3r32`, `r70-5-go-l3r1`, `r70-5-go-l3r32`, and `r70-7-rust-l3r31`.
- Host assignment: EU for Elixir and Rust; US for Go.
- Variant base provenance: the operator created and signed each variant commit from its registered public base plus the delivered failed patch. Exact parent and variant base SHAs are linked below.

STATUS: **DESCRIPTIVE** — six graded repairs; the registered decision is not fully supported.

Pre-registered: **operator-reported** before the first scored cell; the public bundle does not independently establish the exact commitment time.
Label: DESCRIPTIVE — raw captures not retained; results verified against official grades
Question: When direct Codex `gpt-6-luna` at max resumes a prior failed implementation with a fixed self-verification instruction, how often does it repair an official hidden-suite failure without losing checks passed by the prior implementation?
n: **6 matched failures**.
Headline: **2/6 repairs rescued; the ≥3/6 threshold was not met, so repair-with-verification remains budget-limited.**
Configuration: same direct Codex harness, runner, and sandbox; official grade route `r70-macbook-window-v1` (the MacBook-driven grade window that drains the host and runs the pinned grade worker). The fixed instruction asks Codex to check every prompt requirement against the code, write and run its own tests for each, and fix any gap. No hidden-test names or grader output were given to the model.
Limit: six selected failures across tasks 2, 5, and 7; one model; descriptive rescue rule. The comparator is the official pass rate among each original cell's other reps. Aggregate test counts do not establish per-test pass-set preservation; that registered condition is unverifiable from the public data.

## Why not VALID

- The registered decision required at least three rescues; only 2/6 occurred ([decision rule](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/DECISION-RULE.md#decision), [results](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/RESULTS.md#decision)).
- The registered per-test preservation condition is unmeasured: aggregate counts cannot show whether every previously passing test still passed.
- No round `ENVIRONMENT.md` records start and end host conditions. The six Standard records omit effective model and effort, grader, host class, sandbox, price, and cost fields; public reproducibility is incomplete.
- The rule first appears in public Git after the run. The operator-reported design time is a filesystem time, not immutable evidence that the rule preceded the first result.
Raw records: raw run captures are not retained. The public bundle includes [aggregate official-grade rows](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/data/cells.csv) and the [sanitized lane release-state summary](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/data/lane-release-state.json); the per-cell official grade receipt index and per-test result file are also absent, so do not infer either from aggregate counts.
Sources: [decision rule](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/DECISION-RULE.md); [results](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/RESULTS.md); [pre-registration inputs](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/INPUTS.json); [lane release state](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/data/lane-release-state.json); [aggregate cell data](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/data/cells.csv); [reproducer](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/reproduce/l3_repair.py); [H12 / F01 evidence summary](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/hypotheses/f01-shape-intent-plan.md); [status-only claim](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl).

Evidence links: [H12 — checks, review, and actionable retries](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/hypotheses/f01-shape-intent-plan.md); [F01 findings summary](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/FINDINGS.md#f01-shape-intent-and-plan); claim `CL-L3-REPAIR-RESCUE-RATE` (**STATUS_ONLY**).

## Reproduce

### Public task base commits

- Task 2 Elixir: [470ba65c1d3c19191a2190a032cd9ebf24e71975](https://github.com/KogenAI/kogen-ex/commit/470ba65c1d3c19191a2190a032cd9ebf24e71975)
- Task 2 Go: [13a08875ea5e1fd0a3debc6f0b8c6b98a4b37a82](https://github.com/KogenAI/kogen-ex/commit/13a08875ea5e1fd0a3debc6f0b8c6b98a4b37a82)
- Task 5 Go: [e0b4a2478a17a924fcecb965b2941ca3396f49d6](https://github.com/KogenAI/kogen-ex/commit/e0b4a2478a17a924fcecb965b2941ca3396f49d6)
- Task 7 Rust: [111a0c776ae6d93f6e7fccd3fc694d5f4fa26838](https://github.com/KogenAI/kogen-ex/commit/111a0c776ae6d93f6e7fccd3fc694d5f4fa26838)

### Operator-created variant base commits

- `r70-2-elixir-l3r31`: [5c5727999486d1c783a704121a182165422d443e](https://github.com/KogenAI/kogen-ex/commit/5c5727999486d1c783a704121a182165422d443e)
- `r70-2-elixir-l3r33`: [bcf8ab895355cfe822b08d187373dbc6faccb36c](https://github.com/KogenAI/kogen-ex/commit/bcf8ab895355cfe822b08d187373dbc6faccb36c)
- `r70-2-go-l3r32`: [a25ca44fcd42ac0f98b47de2361517171775480a](https://github.com/KogenAI/kogen-ex/commit/a25ca44fcd42ac0f98b47de2361517171775480a)
- `r70-5-go-l3r1`: [aade0efcbc2a6c592bf0c22ea84f82405c369d5b](https://github.com/KogenAI/kogen-ex/commit/aade0efcbc2a6c592bf0c22ea84f82405c369d5b)
- `r70-5-go-l3r32`: [54091c79f09540e6d773be15dcc54df6232a48d8](https://github.com/KogenAI/kogen-ex/commit/54091c79f09540e6d773be15dcc54df6232a48d8)
- `r70-7-rust-l3r31`: [ec773802393fe9d7cd19e493a5e749809470cb03](https://github.com/KogenAI/kogen-ex/commit/ec773802393fe9d7cd19e493a5e749809470cb03)

### Commands and data files

The worker dispatcher sequence was `python3 dispatch.py repair --batch first`, then `python3 dispatch.py repair --batch next-two`, then `python3 dispatch.py repair --batch final-three`, with the preregistered release gates between batches. Variant admission controls used the documented commands:

```sh
python3 grade_window_l3.py kogen-bench-eu --admission-controls --controls-only --fixed-variants r70-2-elixir-l3r31,r70-2-elixir-l3r33,r70-7-rust-l3r31
python3 grade_window_l3.py kogen-bench-us --admission-controls --controls-only --fixed-variants r70-2-go-l3r32,r70-5-go-l3r1,r70-5-go-l3r32
```

From the repository root, reproduce the public results and validate the page with:

```sh
python3 rounds/l3-repair/reproduce/l3_repair.py
python3 reproduce/validate_repo.py
```

The reproducer uses only the standard library and reads [rounds/l3-repair/data/cells.csv](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/data/cells.csv). The CSV records cell IDs, aggregate test counts, wall seconds, uncached/cached/output token counts, independent base pass rates, and operator variant SHAs. No hidden-test names or reference content are included. The frozen [pre-registration bundle](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/INPUTS.json) preserves the design-time inputs; the final grades and deployment SHAs are recorded in this page and the CSV.

## Recomputed from raw records

[`recomputed.json`](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/recomputed.json), [mined cell records](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/data/mined/l3-repair.jsonl.gz), and the [token audit](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/data/mined/TOKEN-AUDIT.json) back observed official outcomes, per-arm pass counts/rates, manifest token counters, and wall medians. The source contains 3 rows (fail 2, pass 1); overall pass rate is 33.3% (1/3) among pass/fail grades. Per-arm detail and coverage counts are in the JSON.

No shared cell IDs are present in the committed public grade export; the source-only cohort is checked against the page's reported figures below.
No shared public-export cell IDs are available for a paired outcome comparison.
No complete published metadata/manifest token pairs are available for this round (0 missing manifest usage; 3 missing published metadata). Missing counters remain unknown, not zero.
This round also has 3 raw-only or non-public rows; they are kept separate from the published-export denominator.


## Results source

# L3 results

STATUS: **DESCRIPTIVE** — the registered three-rescue threshold failed and preservation is unmeasured.

Release label: DESCRIPTIVE — raw captures not retained; results verified against official grades

All reported outcomes and aggregates below are recomputed from [data/cells.csv](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/data/cells.csv) by [reproduce/l3_repair.py](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/reproduce/l3_repair.py).

<!-- BEGIN L3 GENERATED RESULTS -->

## Matched pair results

| Task ID | Exact arm | Original cell ID | Original tests | Repair cell ID | Outcome | Repair tests | wall_s | Uncached | Cached | Output | Total tokens |
| --- | --- | --- | ---: | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |
| `r70-2-elixir-l3r31` | Arm A → Arm B | `codex__gpt-6-luna__max__default__r70-2-elixir__r31` | 24/25 | `codex__gpt-6-luna__max__default__r70-2-elixir-l3r31__r1` | PASS | 25/25 | 1435.708 | 145,573 | 4,334,848 | 71,901 | 4,552,322 |
| `r70-2-elixir-l3r33` | Arm A → Arm B | `codex__gpt-6-luna__max__default__r70-2-elixir__r33` | 21/25 | `codex__gpt-6-luna__max__default__r70-2-elixir-l3r33__r1` | FAIL | 21/25 | 2753.349 | 384,209 | 9,389,568 | 113,206 | 9,886,983 |
| `r70-2-go-l3r32` | Arm A → Arm B | `codex__gpt-6-luna__max__default__r70-2-go__r32` | 21/25 | `codex__gpt-6-luna__max__default__r70-2-go-l3r32__r1` | FAIL | 21/25 | 1020.239 | 121,775 | 2,307,072 | 50,768 | 2,479,615 |
| `r70-5-go-l3r1` | Arm A → Arm B | `codex__gpt-6-luna__max__default__r70-5-go__r1` | 23/24 | `codex__gpt-6-luna__max__default__r70-5-go-l3r1__r1` | FAIL | 23/24 | 489.475 | 76,408 | 503,808 | 24,511 | 604,727 |
| `r70-5-go-l3r32` | Arm A → Arm B | `codex__gpt-6-luna__max__default__r70-5-go__r32` | 23/24 | `codex__gpt-6-luna__max__default__r70-5-go-l3r32__r1` | PASS | 24/24 | 232.210 | 72,800 | 931,584 | 23,072 | 1,027,456 |
| `r70-7-rust-l3r31` | Arm A → Arm B | `codex__gpt-6-luna__max__default__r70-7-rust__r31` | 23/24 | `codex__gpt-6-luna__max__default__r70-7-rust-l3r31__r1` | FAIL | 23/24 | 471.463 | 80,040 | 409,600 | 24,696 | 514,336 |
| **Total** | — | — | — | — | **2/6 rescues** | — | **6402.444** | **880,805** | **17,876,480** | **308,154** | **19,065,439** |

## Independent base pass rates

Each rate uses the same task and stack's other official reps, excluding that pair's original cell; invalidated rows and reps at or above 700 are excluded.

- `r70-2-elixir-l3r31` / original `codex__gpt-6-luna__max__default__r70-2-elixir__r31`: **1/2** across 2 other official reps.
- `r70-2-elixir-l3r33` / original `codex__gpt-6-luna__max__default__r70-2-elixir__r33`: **1/2** across 2 other official reps.
- `r70-2-go-l3r32` / original `codex__gpt-6-luna__max__default__r70-2-go__r32`: **2/2** across 2 other official reps.
- `r70-5-go-l3r1` / original `codex__gpt-6-luna__max__default__r70-5-go__r1`: **5/6** across 6 other official reps.
- `r70-5-go-l3r32` / original `codex__gpt-6-luna__max__default__r70-5-go__r32`: **5/6** across 6 other official reps.
- `r70-7-rust-l3r31` / original `codex__gpt-6-luna__max__default__r70-7-rust__r31`: **5/5** across 5 other official reps.

## Decision

Rescues: **2/6**. The preregistered threshold is **at least 3/6**; it was not met, so repair-with-verification remains budget-limited.
Per-test no-regression requirement: **UNVERIFIABLE**; per-test grade results are not present in this public bundle. Aggregate pass counts are a separate diagnostic and cannot establish that every previously passed check was preserved.
Aggregate count diagnostic: **PASS**; these counts do not determine the no-regression condition.
Token definition: uncached input + cached input + output; total is their sum.

## Process log

- **Admission controls, initial attempt:** Invalid: reference controls passed, while no-op controls were infrastructure-invalid because deployed variant skeletons lacked repository history.
- **Control deployment correction:** The lane-local deployer was corrected to preserve repository history; stray prepared skeletons were archived, hosts cleaned and redeployed, and variant base archives verified.
- **Admission controls, retry:** Valid: every reference passed and each no-op reproduced its paired original pass count.
- **Initial scored-cell launch attempt:** Voided before any model call after a runner workdir error.
- **Task-record deployment correction:** Task records were corrected to point to each variant skeleton; the host variants were redeployed.
- **Official release and grading sequence:** Official grades were returned through the MacBook-driven official grade window (route r70-macbook-window-v1); the first repair received its clean grade before the next release gate.

<!-- END L3 GENERATED RESULTS -->


## Captured run records

No records for this round are listed in the [run-record index](/downloads/results/run-records/index.json).


## Reproduction inventory

- **Type:** analysis-only
- **Script:** `python3 rounds/l3-repair/reproduce/l3_repair.py`
- **Inputs:** `rounds/l3-repair/data/cells.csv`
- **Full execution replay:** unavailable
- **Scope:** Recomputes aggregate repair counts from six published repair rows; per-test pass sets, grade receipts, and historical model executions are unavailable.

See the [complete per-round reproduction inventory](/reproduce/) and the [repository reproduction guide](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/README.md).

## Related records

### Related hypotheses

- [H12](/hypotheses/h12/)

### Related experiments

- [l3-repair](/rounds/l3-repair/)

### Related claims

- [CL-L3-REPAIR-RESCUE-RATE](/claims/cl-l3-repair-rescue-rate/)
