Round r74
r74 Kogen builder configurations versus Codex
Source-reported narrativeRound README and results text are source material. Numeric values that are not tied to a claim ledger record remain source-reported.
Round source README · rounds/r74/README.md
Round date: HISTORICAL (before 2026-10-09) Publication badge: INCOMPLETE; KEPT FOR AUDIT Why not VALID: The historical inventory retains INCOMPLETE because of INCOMPLETE_EVIDENCE, NO_PUBLIC_CELLS, WITHDRAWN. See the round evidence and limitations below. Recomputation status: OBSERVED SOURCE ONLY
Status
INCOMPLETE
- WITHDRAWN — The registered comparison was withdrawn before completion.
- NO_PUBLIC_CELLS — No r74-tagged rows appear in the public result exports.
- INCOMPLETE_EVIDENCE — Retained IDs or terminal statuses are missing for 231 planned slots; recovered internal rows are not joined to official outcomes.
Required reproduction metadata
- Kogen commit: cde7a380455e9793cb1fc8315791bc9beffb863b (the exact pin recorded in the cited round source, resolved against local Kogen history).
- Harness commit: not recorded in the cited round source as a Git commit; any adapter or runtime fingerprints are not Git commits.
- Model and effort: a complete per-cell mapping is not recorded in the cited round source; values present in the source summary remain as stated above.
- Raw records: the cited sources do not provide a complete public round-specific request and grade bundle; public delivery and outcome exports are linked above where applicable.
Terminology: public round glossary.
STATUS: INCOMPLETE
Data completeness: N/A (0 deliveries; NO DELIVERED DATA). Measurement contract; declared gaps.
Pre-registered: yes; registration and amendments are dated 5–6 October 2026.
Label: CONFIRMATORY design, withdrawn before completion.
Question: How do four Kogen builder configurations compare with direct Codex at the same builder setting and with direct Sol high on a fixed 12-task panel?
Status: Withdrawn on 6 October 2026, when the harness version it measured was retired; no further cells will run and no conclusions are drawn. Cells ran before the withdrawal. The current public results/ exports contain no r74-tagged rows. L0 publishes a scoped reconciliation of 167 recovered internal scored rows, 9 smoke attempts, and coverage counts, but these rows are not independently joined to the canonical official outcome export; 231 retained slots have no recovered IDs or terminal statuses.
Planned n: 480 confirmatory cells (8 arms × 12 tasks × 5 reps). This is the plan size, not a recovered executed count.
Headline: No result comparison is supportable from the public export at this time.
Sources: operative decision rule and dated history; measurement coverage; missing-record declaration.
Reproduction record and source reconciliation
This section documents exact replay limits and keeps the page lifecycle and earlier statuses unchanged.
- Kogen build pin: source-reported as
cde7a380; the prefix resolves to full public commit `cde7a380455e9793cb1fc8315791bc9beffb863b`. No per-cell r74 row is public, so use of this pin in each delivered cell is unverified. - Harness source commit/fingerprint: no per-cell public record for this tag.
- Registered models and effort: Kogen uses Sol high for shaping, planning, auditing and edge-test writing; builders are Luna max, Sol low, Sol medium or Sol high. Direct Codex arms use the corresponding single-model setting. No per-cell model or effort records are public for this withdrawn tag; see DECISION-RULE.md.
- Registered task IDs:
elx-02-ingest-supervision,elx-04-queue-backpressure,syn-01-live-ticket-filters,syn-06-migration-ticket-numbers,syn-15-validation-ticket-form,syn-20-email-invite-flow,syn-24-csv-import,syn-31-inbound-email-webhook,elx-07-cli-stats,elx-12-retry-api-deprecation,elx-port-erase-account,elx-port-board-publish-unpublish-public-boundary(from DECISION-RULE.md). - Original execution command: not recorded in the public snapshot; the benchmark cells cannot be replayed from this repository.
- Rebuild the public metadata from the repository root with
python3 reproduce/export_results.pyandpython3 reproduce/build_records.py; check this declaration withpython3 reproduce/validate_round.py --round r74. These commands regenerate or validate the archived public records. An original cell-launch command is not available. - Raw public records: run-record index filtered by
audit_round/round_id, and cells.jsonl filtered byround.
Public snapshot rows for this tag: 0 captured deliveries (0 pass, 0 fail, 0 ungraded/unknown in results/run-records/index.json); official outcome export has 0 rows: 0 pass, 0 fail in cells.jsonl. These are separate source totals. The public files do not provide a complete joined intention-to-treat result.
Round-specific reconciliation: No r74-tagged rows occur in the public delivery or official outcome exports. The L0 reconciliation CSVs reproduce the selected internal snapshot counts, but do not complete the retained-cohort reconstruction or independently join those scores to the canonical official outcome export. Preserve WITHDRAWN; later execution/stop details must not replace the page lifecycle with an efficacy claim.
Recomputed from raw records
`recomputed.json`, mined cell records, and the token audit back observed official outcomes, per-arm pass counts/rates, manifest token counters, and wall medians. The source contains 177 rows (fail 32, pass 144, ungraded 1); overall pass rate is 81.8% (144/176) among pass/fail grades. Per-arm detail and coverage counts are in the JSON.
No shared cell IDs are present in the committed public grade export; the source-only cohort is checked against the page's reported figures below. No shared public-export cell IDs are available for a paired outcome comparison. No complete published metadata/manifest token pairs are available for this round (0 missing manifest usage; 177 missing published metadata). Missing counters remain unknown, not zero. This round also has 177 raw-only or non-public rows; they are kept separate from the published-export denominator.
Captured run records
Standard-schema records assigned to this round in the indexed snapshot.
No per-round file is listed. Check the run-record index for the complete partition list.
Reproduction inventory
Published analysis replay and full experiment replay are reported separately.
- Replay type
analysis-only- Script
- reproduce/reconcile_l0.py
python3 reproduce/reconcile_l0.py - Inputs
- Full execution replay
- Unavailable
- Scope
- Recomputes selected reconciliation CSV summaries only; it does not replay historical model executions or complete the canonical official outcome join.
View the complete per-round reproduction inventory · Repository reproduction guide