> Data revision: baf7c2f4307d3a989a8fb9363df8000fda3d9310
> Source files:
- [rounds/l4-context-packet/README.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4-context-packet/README.md)
- [rounds/l4-context-packet/RESULTS.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4-context-packet/RESULTS.md)
- [results/run-records/index.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/run-records/index.json)
- [rounds/index.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/index.json)
- [results/source-crosswalk/index.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/source-crosswalk/index.json)
- [rounds/STATUS.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/STATUS.md)
- [reproduce/round-inventory.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/round-inventory.json)
- [rounds/l4-context-packet/reproduce/l4-context-packet.py](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4-context-packet/reproduce/l4-context-packet.py)

# L4 deterministic context packet

**Round ID:** l4-context-packet  
**Status:** DESCRIPTIVE

> Reviewed status from rounds/STATUS.md: DESCRIPTIVE.

> Round README and results text are source material. Numeric values not tied to a claim ledger record remain source-reported.

Round date: HISTORICAL (before 2026-10-09)
Publication badge: DESCRIPTIVE; KEPT FOR AUDIT
Why not VALID: The historical inventory retains DESCRIPTIVE because of CONDITIONS_MISMATCH, POST_HOC_AMENDMENT. See the round evidence and limitations below.
Recomputation status: FULLY RECOMPUTABLE


## Status

**DESCRIPTIVE**

### Why not VALID
- **POST_HOC_AMENDMENT:** Amendment 1 was written after 6 of 12 packet outcomes were known.
- **CONDITIONS_MISMATCH:** The US controls were not interleaved with packet cells.
- **relabelled 2026-10-08 after scrutiny:** Previously labelled VALID; the README documents a post-result amendment and non-interleaved controls.


STATUS: **DESCRIPTIVE**

<!-- L4-COMPUTED:BEGIN -->
Pre-registered: yes; original design frozen 7 October 2026 before the first scored packet cell. Amendment 1 was written after 6 of 12 packet outcomes were known; its disclosure is retained in the decision rule.
Label: descriptive pilot; n=3 per arm per variant. The thresholds are coarse and support no general claim.
Question: Do frozen public context packets increase official full-suite passes over same-seed contemporaneous no-packet controls on four L1-admitted variants?
n: 24 official scored cells (12 packet, 12 control; 4 variants × 3 seeds per arm); 20 historical cells are context only.
Headline (descriptive; not a validated registered finding): observed **KEEP** for packets in specification sections 3.2 and 4.9: packet-minus-control = +6 passes, 6 paired rescues, 0 losses, and no regression. The US controls were not interleaved.
Configuration: direct Codex, `gpt-6-luna` at `max`, 3,600-second cap, zero retries; Elixir and Rust on EU, Go on US; official `r70-macbook-window-v1` grades. Tokens are uncached input + cached input + output; total is their sum.
Official runner wrapper: Codex 0.159.3, SHA-256 `08019d8aab7c15481f6d14126645eebd7b2c2c98025f94118baeee2167e4a300`; this same fingerprint appears on all scored cells.
Limit: outcome-selected tasks, n=3 per arm per variant, coarse thresholds, and mixed scheduling; the result supports no general claim.

| Lifecycle count | Count |
| --- | ---: |
| Planned scored cells | 24 |
| Started scored cells | 24 |
| Finished scored cells | 24 |
| Officially graded scored cells | 24 |
| ITT denominator for scored cells | 24 |
<!-- L4-COMPUTED:END -->

Sources: [decision rule and Amendment 1](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4-context-packet/DECISION-RULE.md), [results](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4-context-packet/RESULTS.md), [paired-cell data](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4-context-packet/data/paired-cells.csv), [historical baseline data](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4-context-packet/data/historical-baselines.csv), and [reproducer](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4-context-packet/reproduce/l4-context-packet.py).

## Reproduce

### Public base revisions

| Packet task | Source public base | Packet public base |
| --- | --- | --- |
| `r70-2-elixir-l4pk` | [470ba65c1d3c19191a2190a032cd9ebf24e71975](https://github.com/KogenAI/kogen-ex/commit/470ba65c1d3c19191a2190a032cd9ebf24e71975) | [08a1ca88b8dfd67760586f433fc773a6b23842bb](https://github.com/KogenAI/kogen-ex/commit/08a1ca88b8dfd67760586f433fc773a6b23842bb) |
| `r70-2-go-l4pk` | [13a08875ea5e1fd0a3debc6f0b8c6b98a4b37a82](https://github.com/KogenAI/kogen-ex/commit/13a08875ea5e1fd0a3debc6f0b8c6b98a4b37a82) | [8b8c92e8d0f09d406f328e5426c839b8a47fe2a2](https://github.com/KogenAI/kogen-ex/commit/8b8c92e8d0f09d406f328e5426c839b8a47fe2a2) |
| `r70-4-elixir-fe2-l4pk` | [5ccd0bf1e816e2b2f5b2694861f8596402409f52](https://github.com/KogenAI/kogen-ex/commit/5ccd0bf1e816e2b2f5b2694861f8596402409f52) | [8f85ca4d634373c2417a1f16e7649e031e8f7027](https://github.com/KogenAI/kogen-ex/commit/8f85ca4d634373c2417a1f16e7649e031e8f7027) |
| `r70-7-rust-l4pk` | [111a0c776ae6d93f6e7fccd3fc694d5f4fa26838](https://github.com/KogenAI/kogen-ex/commit/111a0c776ae6d93f6e7fccd3fc694d5f4fa26838) | [6e5af6a2e14c8f64eb88ced5d04a223bcd2c4b94](https://github.com/KogenAI/kogen-ex/commit/6e5af6a2e14c8f64eb88ced5d04a223bcd2c4b94) |

Kogen commit: source `470ba65c1d3c19191a2190a032cd9ebf24e71975` and packet `08a1ca88b8dfd67760586f433fc773a6b23842bb` are exact linked examples; all per-task source and packet SHAs are listed above.

Harness commit: not recorded as a Git commit in the official aggregate grade rows. The lane dispatcher SHA-256 is `f23824e32fb74f42e95e3a0c106df4070fac5425b9c8981dfbd293859acfdfee`; the common wrapper fingerprint recorded on all scored rows is SHA-256 `08019d8aab7c15481f6d14126645eebd7b2c2c98025f94118baeee2167e4a300`.

Model and effort: direct Codex, `gpt-6-luna` at `max`; official grade route `r70-macbook-window-v1`; 3,600-second cap and zero retries. The lane handoff lists worker Codex CLI pin `0.160.0` and Python `3.14.7`. Packet authoring used `codex-cli 0.160.1` in an ephemeral public-only workspace. The reported runner wrapper version and fingerprint appear in the CSV-backed summary above.

Task IDs: packet tasks `r70-2-elixir-l4pk`, `r70-2-go-l4pk`, `r70-4-elixir-fe2-l4pk`, and `r70-7-rust-l4pk`; control tasks `r70-2-elixir`, `r70-2-go`, `r70-4-elixir-fe2`, and `r70-7-rust`. Exact cell IDs, reps, and seeds are in the paired-cell CSV.

Historical execution command: not recorded as a shell transcript. Reproduction command, run from the SOT repository root:

```sh
python3 rounds/l4-context-packet/reproduce/l4-context-packet.py
```

The script uses only the standard library and checks every reported outcome number against `rounds/l4-context-packet/data/paired-cells.csv` and `rounds/l4-context-packet/data/historical-baselines.csv`.

Raw records: these two CSV files contain the latest official aggregate row for each exact scored cell ID and the registered historical baseline cell IDs. The token total per cell is uncached input + cached input + output; arm totals sum all cell totals.

## Recomputed from raw records

[`recomputed.json`](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l4-context-packet/recomputed.json), [mined cell records](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/data/mined/l4-context-packet.jsonl.gz), and the [token audit](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/data/mined/TOKEN-AUDIT.json) back observed official outcomes, per-arm pass counts/rates, manifest token counters, and wall medians. The source contains 6 rows (fail 1, pass 5); overall pass rate is 83.3% (5/6) among pass/fail grades. Per-arm detail and coverage counts are in the JSON.

No shared cell IDs are present in the committed public grade export; the source-only cohort is checked against the page's reported figures below.
No shared public-export cell IDs are available for a paired outcome comparison.
No complete published metadata/manifest token pairs are available for this round (0 missing manifest usage; 6 missing published metadata). Missing counters remain unknown, not zero.
This round also has 6 raw-only or non-public rows; they are kept separate from the published-export denominator.


## Results source

# L4 deterministic context packet results

STATUS: **DESCRIPTIVE** (see the round README: post-result amendment, non-interleaved controls)

The primary comparison follows Amendment 1: packet and contemporaneous no-packet controls are paired by variant and seed. Each cell is the latest official aggregate grade row for its exact cell ID, from the MacBook grading route.

## Paired outcomes

| Source variant | Seed | Packet cell | No-packet control | Pair |
| --- | ---: | --- | --- | --- |
| `r70-2-go` | 1 | PASS 25/25 | PASS 25/25 | both pass |
| `r70-2-go` | 2 | PASS 25/25 | FAIL 21/25 | rescue |
| `r70-2-go` | 3 | PASS 25/25 | PASS 25/25 | both pass |
| `r70-2-elixir` | 1 | PASS 25/25 | FAIL 21/25 | rescue |
| `r70-2-elixir` | 2 | PASS 25/25 | FAIL 21/25 | rescue |
| `r70-2-elixir` | 3 | PASS 25/25 | PASS 25/25 | both pass |
| `r70-4-elixir-fe2` | 1 | FAIL 17/18 | FAIL 17/18 | both fail |
| `r70-4-elixir-fe2` | 2 | FAIL 17/18 | FAIL 17/18 | both fail |
| `r70-4-elixir-fe2` | 3 | FAIL 17/18 | FAIL 17/18 | both fail |
| `r70-7-rust` | 1 | PASS 24/24 | FAIL 23/24 | rescue |
| `r70-7-rust` | 2 | PASS 24/24 | FAIL 23/24 | rescue |
| `r70-7-rust` | 3 | PASS 24/24 | FAIL 23/24 | rescue |

## Pass counts, test counts, and tokens

A full pass is an official `pass` outcome. Mean `tests_passed` is the arithmetic mean across the three cells in that variant and arm. One cell's total tokens are uncached input + cached input + output; total is the sum. Tokens per pass is all tokens consumed by every cell in that arm divided by its full-pass count, so failed-cell tokens remain in the numerator.

| Source variant | Arm | Successful cells | Mean tests_passed | Total tokens | Tokens per successful cell |
| --- | --- | ---: | ---: | ---: | ---: |
| `r70-2-go` | packet | 3/3 | 25.00 | 1,922,619 | 640,873 |
| `r70-2-go` | control | 2/3 | 23.67 | 1,320,610 | 660,305 |
| `r70-2-elixir` | packet | 3/3 | 25.00 | 24,525,211 | 8,175,070.3 |
| `r70-2-elixir` | control | 1/3 | 22.33 | 27,080,262 | 27,080,262 |
| `r70-4-elixir-fe2` | packet | 0/3 | 17.00 | 3,043,793 | — (0 passes) |
| `r70-4-elixir-fe2` | control | 0/3 | 17.00 | 3,256,526 | — (0 passes) |
| `r70-7-rust` | packet | 3/3 | 24.00 | 1,053,801 | 351,267 |
| `r70-7-rust` | control | 0/3 | 23.00 | 928,201 | — (0 passes) |
| All variants | packet | 9/12 | 22.75 | 30,545,424 | 3,393,936 |
| All variants | control | 3/12 | 21.50 | 32,585,599 | 10,861,866.3 |

## Historical baselines (context only)

These are the registered historical Luna-max cells. Amendment 1 makes the contemporaneous paired controls the decision basis; the historical baseline is shown only as context.

| Source variant | Historical successes | Eligible cells |
| --- | ---: | ---: |
| `r70-2-go` | 3/4 | 4 |
| `r70-2-elixir` | 2/4 | 4 |
| `r70-4-elixir-fe2` | 0/5 | 5 |
| `r70-7-rust` | 6/7 | 7 |

## Decision and protocol notes

The net packet-minus-control change is **+6 full passes**. There are **6 paired rescues** and **0 paired losses**. Packet pass counts are no lower than controls on **4/4 variants**. The regression rule did not fire; the worst mean `tests_passed` comparison is 17.00 packet versus 17.00 control on `r70-4-elixir-fe2` (difference 0.00).

**Verdict: KEEP** packets in specification sections 3.2 and 4.9, as a descriptive pilot. The result is based on four outcome-selected variants and three cells per arm per variant; the rule thresholds are coarse and support no general claim.

The EU controls followed the seeded order recorded for the lane. One EU control was launched while a grade-window drain was in progress; this delayed the window only and affected no result. The three US controls ran after their packet cells had finished and were **not interleaved**.

A separate confirmation round, `l4b-confirm`, is running on three new variants; it is not included here.


## Captured run records

No records for this round are listed in the [run-record index](/downloads/results/run-records/index.json).


## Reproduction inventory

- **Type:** analysis-only
- **Script:** `python3 rounds/l4-context-packet/reproduce/l4-context-packet.py`
- **Inputs:** `rounds/l4-context-packet/data/paired-cells.csv`, `rounds/l4-context-packet/data/historical-baselines.csv`
- **Full execution replay:** unavailable
- **Scope:** The public aggregate CSVs support analysis replay only; benchmark execution and official grading are not replayed.

See the [complete per-round reproduction inventory](/reproduce/) and the [repository reproduction guide](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/README.md).

## Related records

### Related experiments

- [l4-context-packet](/rounds/l4-context-packet/)
