> Data revision: baf7c2f4307d3a989a8fb9363df8000fda3d9310
> Source files:
- [rounds/l4b-confirm/README.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/README.md)
- [rounds/l4b-confirm/RESULTS.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/RESULTS.md)
- [results/run-records/index.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/run-records/index.json)
- [rounds/index.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/index.json)
- [results/source-crosswalk/index.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/source-crosswalk/index.json)
- [rounds/STATUS.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/STATUS.md)
- [reproduce/round-inventory.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/round-inventory.json)
- [rounds/l4b-confirm/reproduce/reproduce_us.py](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/reproduce/reproduce_us.py)

# L4b independent context-packet replication

**Round ID:** l4b-confirm  
**Status:** VALID

> Reviewed status from rounds/STATUS.md: VALID.

> Round README and results text are source material. Numeric values not tied to a claim ledger record remain source-reported.

Round date: HISTORICAL (before 2026-10-09)
Publication badge: VALID; KEPT FOR AUDIT
Recomputation status: FULLY RECOMPUTABLE


## Status

**VALID**

### Limits
- Restored to VALID 2026-10-08: the round executed its registered rule as registered and its numbers recompute; the points below are limits on public evidence, not contradictions in the round's own files.
- **RELEASE_EVIDENCE_MISSING:** Standard run records and strict-validation receipts are unavailable, so execution checklist compliance cannot be established.


STATUS: **VALID** — US part complete

Pre-registered: yes; design frozen 7 October 2026 before any scored cell. The published design is a sanitized copy; original-source SHA-256: `94d0158342fd83188e5a49d3aed538bda401cd96f5335b2992a24e5342634ebc`; published-byte SHA-256: `7b6bd259e4dcf0229f1d4840daaed75fbee27722f916cec3fcc5ac9743f8dc7f`.
Label: DESCRIPTIVE — the 18 US outcomes are reported from official grades; Standard run records and smoke/strict-validation receipts are unavailable in the public snapshot, so strict release compliance is not established.
Question: On three L1-admitted variants unused by L4, does a frozen public-only context packet improve official full-suite results versus a contemporaneous no-packet control?
n: 18 scored US cells (3 variants × 2 arms × 3 paired seeds); all 18 officially graded.
Headline: NOT CONFIRMED: packet 6/9 vs control 6/9; Σ(P−C)=0 (pre-registered threshold: +2); 2 paired rescues and 2 paired losses.
Configuration: direct Codex, `gpt-6-luna` at `max`; worker Codex CLI `0.160.0`; packet-author Codex CLI `0.161.0`; Python `3.14.7`; 3,600-second timeout; zero retries; sequential on `kogen-bench-us`; official route `r70-macbook-window-v1`. Launcher code SHA-256: `d4d181af6ddb8cfa2bb6efe6f73b2ed8de533d1a21340270c864ae7483af25cb`. Token total is uncached input + cached input + output.
Limit: n=3 per arm per variant, one model and one host. A dated pre-outcome preparation receipt recorded ceiling risk from high Luna-max baselines; contemporaneous controls passed 6/9. L4's KEEP came from low-baseline variants. This result is no-confirmation, not evidence against the packet; it does not establish an effect outside these variants and seeds.

## Release evidence disclosure

The public snapshot has no Standard 1.2 run record for any of the 18 cell IDs, no strict-validation output for the seven-cell smoke prefix before the remaining eleven were released, and no strict-validation output for all 18 cells before analysis. The published outcome table and operator-reported grade chronology do not replace these records. The round is published descriptively; strict checklist completion is not established. See [MEASURED.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/MEASURED.md) for the exact evidence gap and [READY.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/READY.md) for the execution record.

### US cohort accounting

| Population | Count |
| --- | ---: |
| Planned scored cells | 18 |
| Started scored cells | 18 |
| Finished cells | 18 |
| Officially graded cells | 18 |
| ITT denominator | 18 |

Sources: [frozen design](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/DESIGN.md), [pre-registered decision rule](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/DECISION-RULE.md), [US results](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/RESULTS.md), [release and execution record](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/READY.md), [sanitized cell data](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/data/cells.csv), [frozen order](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/reproduce/ORDER.json), and [reproducer](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/reproduce/reproduce_us.py).

## Reproduce

### Source and packet commits

| Source task | Source base commit | Packet task | Packet child commit |
| --- | --- | --- | --- |
| `r70-4-go-fe2` | [21d21a2f420a387c921f998e5b4d16efc8f80904](https://github.com/KogenAI/kogen-ex/commit/21d21a2f420a387c921f998e5b4d16efc8f80904) | `r70-4-go-fe2-l4pk` | [7401a896ef458cd71830dc4d2677ad69e7f4561b](https://github.com/KogenAI/kogen-ex/commit/7401a896ef458cd71830dc4d2677ad69e7f4561b) |
| `r70-4-ts-bun-fe2` | [cc3a831a167020538c4ca2ad90185606a2e4ad6a](https://github.com/KogenAI/kogen-ex/commit/cc3a831a167020538c4ca2ad90185606a2e4ad6a) | `r70-4-ts-bun-fe2-l4pk` | [30c1860e763742c85a20af189bf397a1b726f8b6](https://github.com/KogenAI/kogen-ex/commit/30c1860e763742c85a20af189bf397a1b726f8b6) |
| `r70-5-go` | [e0b4a2478a17a924fcecb965b2941ca3396f49d6](https://github.com/KogenAI/kogen-ex/commit/e0b4a2478a17a924fcecb965b2941ca3396f49d6) | `r70-5-go-l4pk` | [a763145f30890271c8cba0bd5bc048b330cb280b](https://github.com/KogenAI/kogen-ex/commit/a763145f30890271c8cba0bd5bc048b330cb280b) |

The operator created the three signed empty child commits; each packet task used the matching source tree and appended its frozen packet. Packet SHA-256 values and public-source hashes are in [PACKETS.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/reproduce/PACKETS.json).

### Harness, tasks, commands, and data

Tasks: `r70-4-go-fe2`, `r70-4-go-fe2-l4pk`, `r70-4-ts-bun-fe2`, `r70-4-ts-bun-fe2-l4pk`, `r70-5-go`, and `r70-5-go-l4pk`. Seeds 1–3 map to reps 61–63. The exact randomized order and cell IDs are preserved in `reproduce/ORDER.json` and reproduced with the outcomes in [RESULTS.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/RESULTS.md).

The worker used Codex CLI `0.160.0`; packet authorship used Codex CLI `0.161.0`; Python was `3.14.7`. The lane dispatcher code is identified by the SHA-256 above. Official grades came through `r70-macbook-window-v1`. The public cell table contains only cell IDs, outcomes, test counts, arm metadata, and token counters.

Kogen commit: the source and packet child commits are linked above; `21d21a2f420a387c921f998e5b4d16efc8f80904` and `7401a896ef458cd71830dc4d2677ad69e7f4561b` are one linked pair.
Harness commit: not recorded; the lane dispatcher code SHA-256 is recorded above.
Model and effort: `gpt-6-luna` at `max`.
Task IDs: `r70-4-go-fe2`, `r70-4-go-fe2-l4pk`, `r70-4-ts-bun-fe2`, `r70-4-ts-bun-fe2-l4pk`, `r70-5-go`, `r70-5-go-l4pk`.
Historical execution command: the operator ran the two exact batch commands below on `kogen-bench-us`.
Raw records: sanitized official cell data in `data/cells.csv`; frozen IDs and execution order in `reproduce/ORDER.json`.

Operator batch entrypoints on `kogen-bench-us`, run in the frozen order:

```sh
python3 levers/lanes-2026-10-07/l4b/dispatch.py run --batch first
python3 levers/lanes-2026-10-07/l4b/dispatch.py run --batch rest
```

Recompute the public results and validate the SOT repository from its root:

```sh
python3 rounds/l4b-confirm/reproduce/reproduce_us.py
python3 reproduce/validate_repo.py
```

Inputs: `rounds/l4b-confirm/data/cells.csv`, `rounds/l4b-confirm/reproduce/ORDER.json`, `rounds/l4b-confirm/DESIGN.md`, and `rounds/l4b-confirm/DECISION-RULE.md`. The reproducer uses only the Python standard library, checks the frozen inputs and every cell's accounting, then verifies the generated tables and headline.

### Process notes and interpretation

- The task metadata repository path was corrected before deployment.
- The seven-cell smoke prefix stalled for about 25 minutes at the US disk floor. This delayed the run; the prefix was officially graded before the remaining eleven cells were released. Public Standard run records and strict-validation receipts are unavailable, so checklist completion is not verifiable from this snapshot.
- L4's KEEP came from low-baseline variants. Here the controls passed 6/9, consistent with the ceiling-risk concern recorded before scored cells in the [dated preparation receipt](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/reproduce/CEILING-RISK-RECEIPT.md). The 0 net pass difference does not confirm a packet advantage and is not evidence against the packet.
- Per-variant mean `tests_passed` was equal between arms. The small cohort cannot support a general claim beyond these three variants, one model, one host, and three paired seeds per variant.
- The EU extension, `l4b-confirm-eu`, is **NOT-RUN**. Only task 7 was admitted, its baseline was 5/6, and no cells were spent. The completed 18-cell US cohort is separate; there is no EU scored result.

## Recomputed from raw records

[`recomputed.json`](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/recomputed.json), [mined cell records](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/data/mined/l4b-confirm.jsonl.gz), and the [token audit](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/data/mined/TOKEN-AUDIT.json) back observed official outcomes, per-arm pass counts/rates, manifest token counters, and wall medians. The source contains 18 rows (fail 6, pass 12); overall pass rate is 66.7% (12/18) among pass/fail grades. Per-arm detail and coverage counts are in the JSON.

No shared cell IDs are present in the committed public grade export; the source-only cohort is checked against the page's reported figures below.
No shared public-export cell IDs are available for a paired outcome comparison.
No complete published metadata/manifest token pairs are available for this round (0 missing manifest usage; 18 missing published metadata). Missing counters remain unknown, not zero.
This round also has 18 raw-only or non-public rows; they are kept separate from the published-export denominator.


## Results source

# L4b US results

STATUS: **VALID** — the US part of the round is complete. All 18 scored cells were officially graded through `r70-macbook-window-v1`; all remain in the intention-to-treat denominator, with no exclusions.

**Decision: NOT CONFIRMED** under the pre-registered rule. The packet and control arms each passed 6/9 cells, so Σ(P−C)=0, below the required +2. The observed paired outcomes were two rescues and two losses. Per-variant mean `tests_passed` was equal between packet and control, so there was no test-count regression in this cohort.

The ceiling-risk concern from high Luna-max baselines was recorded before scored cells in a dated [preparation receipt](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/reproduce/CEILING-RISK-RECEIPT.md) on 7 October 2026. Contemporaneous controls passed 6/9. L4's KEEP came from low-baseline variants; this confirmation cohort tested a different baseline regime. The result is no-confirmation, not evidence against the packet. With n=3 per arm per variant, one model, and one host, it supports no general claim beyond these cells.

The smoke prefix stalled for about 25 minutes at the US disk floor. It received official grades before the remaining cells were released, according to the published operator chronology. The public snapshot has no Standard 1.2 run records for the 18 IDs, no strict-validation receipt for the seven-cell smoke prefix before the remaining cells were released, and no strict-validation receipt for all 18 cells before analysis. Strict checklist completion is not established; see [MEASURED.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/MEASURED.md). The EU extension `l4b-confirm-eu` is **NOT-RUN**: only task 7 was admitted, its baseline was 5/6, and no cells were spent. There is no EU scored result.

The marked block below is generated and checked by the standard-library [reproducer](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/reproduce/reproduce_us.py) from [cells.csv](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/data/cells.csv) and the frozen [ORDER.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4b-confirm/reproduce/ORDER.json).

<!-- R70-L4B-US:BEGIN -->

## Per-variant outcomes

| Variant | Packet full passes | Control full passes | Packet mean tests_passed | Control mean tests_passed | P−C |
| --- | ---: | ---: | ---: | ---: | ---: |
| `r70-4-go-fe2` | 3/3 | 3/3 | 18.00 | 18.00 | 0 |
| `r70-4-ts-bun-fe2` | 1/3 | 1/3 | 17.33 | 17.33 | 0 |
| `r70-5-go` | 2/3 | 2/3 | 23.67 | 23.67 | 0 |
| **Total** | **6/9** | **6/9** | — | — | **0** |

**Decision: NOT CONFIRMED** under the pre-registered rule (required Σ(P−C) ≥ +2; observed Σ(P−C) = 0).
Paired outcomes: **2 rescues** and **2 losses**.

## Paired outcomes by variant and seed

| Variant | Seed (rep) | Packet outcome (`tests_passed`) | Control outcome (`tests_passed`) | Pair result |
| --- | ---: | --- | --- | --- |
| `r70-4-go-fe2` | 1 (r61) | PASS (18/18) | PASS (18/18) | same |
| `r70-4-go-fe2` | 2 (r62) | PASS (18/18) | PASS (18/18) | same |
| `r70-4-go-fe2` | 3 (r63) | PASS (18/18) | PASS (18/18) | same |
| `r70-4-ts-bun-fe2` | 1 (r61) | FAIL (17/18) | FAIL (17/18) | same |
| `r70-4-ts-bun-fe2` | 2 (r62) | PASS (18/18) | FAIL (17/18) | rescue |
| `r70-4-ts-bun-fe2` | 3 (r63) | FAIL (17/18) | PASS (18/18) | loss |
| `r70-5-go` | 1 (r61) | PASS (24/24) | PASS (24/24) | same |
| `r70-5-go` | 2 (r62) | PASS (24/24) | FAIL (23/24) | rescue |
| `r70-5-go` | 3 (r63) | FAIL (23/24) | PASS (24/24) | loss |

## Token use per official full pass

Numerator is total usage across all nine cells in that arm, including failed cells; denominator is official full passes. One token definition applies: uncached input + cached input + output = total.

| Arm | Full passes | Uncached input / pass | Cached input / pass | Output / pass | Total / pass |
| --- | ---: | ---: | ---: | ---: | ---: |
| packet | 6/9 | 70,861.5 | 625,066.7 | 31,642.3 | 727,570.5 |
| control | 6/9 | 63,870.2 | 661,077.3 | 30,814.0 | 755,761.5 |

All-cell token sums (packet): uncached 425,169; cached 3,750,400; output 189,854; total 4,365,423.

All-cell token sums (control): uncached 383,221; cached 3,966,464; output 184,884; total 4,534,569.

## Frozen seeded interleaved order and official outcomes

| Order | Cell ID | Variant | Arm | Seed | Rep | Outcome | tests_passed |
| ---: | --- | --- | --- | ---: | ---: | --- | ---: |
| 1 | `codex__gpt-6-luna__max__default__r70-4-go-fe2__r63` | `r70-4-go-fe2` | control | 3 | 63 | PASS | 18/18 |
| 2 | `codex__gpt-6-luna__max__default__r70-5-go-l4pk__r62` | `r70-5-go` | packet | 2 | 62 | PASS | 24/24 |
| 3 | `codex__gpt-6-luna__max__default__r70-4-ts-bun-fe2__r63` | `r70-4-ts-bun-fe2` | control | 3 | 63 | PASS | 18/18 |
| 4 | `codex__gpt-6-luna__max__default__r70-4-ts-bun-fe2-l4pk__r62` | `r70-4-ts-bun-fe2` | packet | 2 | 62 | PASS | 18/18 |
| 5 | `codex__gpt-6-luna__max__default__r70-4-ts-bun-fe2-l4pk__r63` | `r70-4-ts-bun-fe2` | packet | 3 | 63 | FAIL | 17/18 |
| 6 | `codex__gpt-6-luna__max__default__r70-5-go__r63` | `r70-5-go` | control | 3 | 63 | PASS | 24/24 |
| 7 | `codex__gpt-6-luna__max__default__r70-4-go-fe2-l4pk__r61` | `r70-4-go-fe2` | packet | 1 | 61 | PASS | 18/18 |
| 8 | `codex__gpt-6-luna__max__default__r70-4-ts-bun-fe2-l4pk__r61` | `r70-4-ts-bun-fe2` | packet | 1 | 61 | FAIL | 17/18 |
| 9 | `codex__gpt-6-luna__max__default__r70-4-ts-bun-fe2__r62` | `r70-4-ts-bun-fe2` | control | 2 | 62 | FAIL | 17/18 |
| 10 | `codex__gpt-6-luna__max__default__r70-5-go-l4pk__r61` | `r70-5-go` | packet | 1 | 61 | PASS | 24/24 |
| 11 | `codex__gpt-6-luna__max__default__r70-5-go__r61` | `r70-5-go` | control | 1 | 61 | PASS | 24/24 |
| 12 | `codex__gpt-6-luna__max__default__r70-5-go__r62` | `r70-5-go` | control | 2 | 62 | FAIL | 23/24 |
| 13 | `codex__gpt-6-luna__max__default__r70-4-go-fe2__r62` | `r70-4-go-fe2` | control | 2 | 62 | PASS | 18/18 |
| 14 | `codex__gpt-6-luna__max__default__r70-4-go-fe2-l4pk__r63` | `r70-4-go-fe2` | packet | 3 | 63 | PASS | 18/18 |
| 15 | `codex__gpt-6-luna__max__default__r70-4-go-fe2__r61` | `r70-4-go-fe2` | control | 1 | 61 | PASS | 18/18 |
| 16 | `codex__gpt-6-luna__max__default__r70-5-go-l4pk__r63` | `r70-5-go` | packet | 3 | 63 | FAIL | 23/24 |
| 17 | `codex__gpt-6-luna__max__default__r70-4-go-fe2-l4pk__r62` | `r70-4-go-fe2` | packet | 2 | 62 | PASS | 18/18 |
| 18 | `codex__gpt-6-luna__max__default__r70-4-ts-bun-fe2__r61` | `r70-4-ts-bun-fe2` | control | 1 | 61 | FAIL | 17/18 |

<!-- R70-L4B-US:END -->


## Captured run records

No records for this round are listed in the [run-record index](/downloads/results/run-records/index.json).


## Reproduction inventory

- **Type:** analysis-only
- **Script:** `python3 rounds/l4b-confirm/reproduce/reproduce_us.py`
- **Inputs:** `rounds/l4b-confirm/README.md`, `rounds/l4b-confirm/RESULTS.md`, `rounds/l4b-confirm/data/cells.csv`, `rounds/l4b-confirm/reproduce/ORDER.json`, `rounds/l4b-confirm/DESIGN.md`, `rounds/l4b-confirm/DECISION-RULE.md`
- **Full execution replay:** unavailable
- **Scope:** Recomputes official outcomes, paired seed results, test counts, token totals, and published tables from sanitized cell metrics. Model execution and official grading are not replayed.

See the [complete per-round reproduction inventory](/reproduce/) and the [repository reproduction guide](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/README.md).

## Related records

### Related experiments

- [l4b-confirm](/rounds/l4b-confirm/)
