Round r70-task8
Round 70 task 8 extension
Source-reported narrativeRound README and results text are source material. Numeric values that are not tied to a claim ledger record remain source-reported.
Round source README · rounds/r70-task8/README.md
Round date: HISTORICAL (before 2026-10-09) Publication badge: DESCRIPTIVE / INCOMPLETE (PILOT); KEPT FOR AUDIT Why not VALID: The historical inventory retains PILOT because of NO_OUTCOME, NO_SCORED_RESULTS, SMOKE_ONLY. See the round evidence and limitations below. Recomputation status: OBSERVED SOURCE ONLY
Recomputation limit: Published summary figures: planned/observed counts, outcome totals/rates, and numeric estimates as present in README.md and RESULTS.md. These figures cannot be recomputed from repository inputs because no public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.
Status
PILOT
- NO_SCORED_RESULTS — The README reports scored n = 0.
- SMOKE_ONLY — Four smoke grades are explicitly not scored; reference controls failed.
- NO_OUTCOME — No task-8 scored comparison exists.
Required reproduction metadata
- Kogen commit: not recorded in the cited round source.
- Harness commit: not recorded in the cited round source as a Git commit; any adapter or runtime fingerprints are not Git commits.
- Model and effort: a complete per-cell mapping is not recorded in the cited round source; values present in the source summary remain as stated above.
- Task IDs: r70-8-elixir, r70-8-go, r70-8-rust, r70-8-ts-bun; the complete scored-cell mapping is not recorded where noted above.
- Historical execution command: not recorded in the cited round source.
- Raw records: the cited sources do not provide a complete public round-specific request and grade bundle; public delivery and outcome exports are linked above where applicable.
Terminology: public round glossary.
STATUS: PILOT
Data completeness (standard run-record capture): N/A (0 deliveries; NO DELIVERED DATA). Supplemental evidence: 4 unscored smokes; 26 control receipts. Measurement contract; declared gaps.
Pre-registered: yes Label: DESCRIPTIVE Lifecycle: PRE-REGISTERED; the scored phase remains NOT-RUN because the Rust and Go v1/v2 smokes failed Rule L, which requires at least 13/25 when the reference passes 25/25. Question: What hidden-suite pass counts do Rust, Elixir, Go and TypeScript/Bun record in the separate Round 70 task-8 cohort? n: Planned 8 scored cells; scored n = 0. The four Rust/Go v1 and v2 smoke grades are not scored and are reported in results. Headline: Each Rust and Go v1/v2 smoke returned 1/25. Elixir and TypeScript/Bun reference controls failed; those stacks had no task-8 smoke or scored cells. Task 8 is excluded from the language comparison.
Limit: Official smoke and control receipts are public, but versioned task-8 prompts and grader version/hash are not in this snapshot; the prompt-based diagnosis cannot be checked.
Sources: decision rule; results; test-count ledger; controls ledger; frozen picks; cell plan.
Public delivery and outcome reconciliation
No scored task-8 cells are recorded in the public cell export. The four smoke outcomes are retained as not-scored rows in the round-specific test-count ledger; admission and X-controls are in controls.jsonl.
Round results · rounds/r70-task8/RESULTS.md
STATUS: NOT-RUN for the scored cohort; no scored cells were released.
Setup and status
Status: planned n=8; scored n=0. The registered v1 design planned two reps for each of four stacks; the four rows below are unscored smoke checks. Cells used kogen-bench-us, direct Codex with gpt-6-luna at max, a 3600-second cap, and zero retries. Rule L required at least 13/25 when the reference passed 25/25; both Rust and Go v1/v2 smokes returned 1/25.
No task-8 smoke was launched for Elixir or TypeScript/Bun: their v1 reference controls failed at 24/25, so no smoke was admitted, and the later v2 scope was limited to Rust and Go. This page makes no task-8 language comparison. Per-smoke usage and cost are unverified in the permitted public receipts; they are not zero. The separate original comparison is documented in the 40-cell as-graded RvE table.
The test-count ledger contains the four officially graded smoke rows. All four Rust/Go v1 and v2 smokes returned 1/25 and failed Rule L. These smoke rows are marked not scored.
| Version | Language | Cell ID | Outcome | Hidden tests passed/total | Scored |
|---|---|---|---|---|---|
| v1 | Rust | codex__gpt-6-luna__max__default__r70-8-rust__r90 | fail | 1/25 | no |
| v1 | Go | codex__gpt-6-luna__max__default__r70-8-go__r90 | fail | 1/25 | no |
| v2 | Rust | codex__gpt-6-luna__max__default__r70-8-rust-v2__r1 | fail | 1/25 | no |
| v2 | Go | codex__gpt-6-luna__max__default__r70-8-go-v2__r1 | fail | 1/25 | no |
Admission and X-control outcomes
Exact control cell IDs, source labels, timestamps, test counts, and recorded patch hashes are in controls.jsonl. None of these rows is scored.
| Version and variant | Reference | No-op | Reference with skeleton front end | Reference through contestant path |
|---|---|---|---|---|
| v1 Rust | 2 pass, each 25/25 | 2 fail, each 0/25 | 1 pass, 25/25 | 1 pass, 25/25 |
| v1 Elixir | 2 fail, each 24/25 | 2 fail, each 0/25 | 1 fail, 24/25 | not run |
| v1 Go | 2 pass, each 25/25 | 2 fail, each 0/25 | 1 pass, 25/25 | 1 pass, 25/25 |
| v1 TypeScript/Bun | 2 fail, each 24/25 | 2 fail, each 0/25 | 1 fail, 24/25 | not run |
| v2 Rust | 1 pass, 25/25 | 1 fail, 0/25 | not run | not run |
| v2 Go | 1 pass, 25/25 | 1 fail, 0/25 | not run | not run |
The Elixir and TypeScript/Bun reference controls each failed all three recorded checks at 24/25. Their task-8 smokes were not run. Rust and Go v1 and v2 smoke counts were below the Rule L threshold: a smoke must pass at least half of the reference pass fraction, which is 13/25 when the reference passes 25/25.
The public receipts establish outcomes and test counts. Versioned task-8 prompt text and grader version/hash are not in this public snapshot, so the prompt-based diagnosis cannot be checked here.
Captured run records
Standard-schema records assigned to this round in the indexed snapshot.
No per-round file is listed. Check the run-record index for the complete partition list.
Reproduction inventory
Published analysis replay and full experiment replay are reported separately.
- Replay type
unavailable- Script
- No per-round analysis script is published.
- Inputs
- No identified public input bundle.
- Full execution replay
- Unavailable
- Scope
- No public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.
View the complete per-round reproduction inventory · Repository reproduction guide