Round r70-task8

Round 70 task 8 extension

Status · PILOTReviewed status is transcribed from rounds/STATUS.md. The original narrative and its qualifications remain below.

Source-reported narrativeRound README and results text are source material. Numeric values that are not tied to a claim ledger record remain source-reported.

Round source README · rounds/r70-task8/README.md

Round date: HISTORICAL (before 2026-10-09) Publication badge: DESCRIPTIVE / INCOMPLETE (PILOT); KEPT FOR AUDIT Why not VALID: The historical inventory retains PILOT because of NO_OUTCOME, NO_SCORED_RESULTS, SMOKE_ONLY. See the round evidence and limitations below. Recomputation status: OBSERVED SOURCE ONLY

Recomputation limit: Published summary figures: planned/observed counts, outcome totals/rates, and numeric estimates as present in README.md and RESULTS.md. These figures cannot be recomputed from repository inputs because no public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.

Status

PILOT

  • NO_SCORED_RESULTS — The README reports scored n = 0.
  • SMOKE_ONLY — Four smoke grades are explicitly not scored; reference controls failed.
  • NO_OUTCOME — No task-8 scored comparison exists.

Required reproduction metadata

  • Kogen commit: not recorded in the cited round source.
  • Harness commit: not recorded in the cited round source as a Git commit; any adapter or runtime fingerprints are not Git commits.
  • Model and effort: a complete per-cell mapping is not recorded in the cited round source; values present in the source summary remain as stated above.
  • Task IDs: r70-8-elixir, r70-8-go, r70-8-rust, r70-8-ts-bun; the complete scored-cell mapping is not recorded where noted above.
  • Historical execution command: not recorded in the cited round source.
  • Raw records: the cited sources do not provide a complete public round-specific request and grade bundle; public delivery and outcome exports are linked above where applicable.

Terminology: public round glossary.

STATUS: PILOT

Data completeness (standard run-record capture): N/A (0 deliveries; NO DELIVERED DATA). Supplemental evidence: 4 unscored smokes; 26 control receipts. Measurement contract; declared gaps.

Pre-registered: yes Label: DESCRIPTIVE Lifecycle: PRE-REGISTERED; the scored phase remains NOT-RUN because the Rust and Go v1/v2 smokes failed Rule L, which requires at least 13/25 when the reference passes 25/25. Question: What hidden-suite pass counts do Rust, Elixir, Go and TypeScript/Bun record in the separate Round 70 task-8 cohort? n: Planned 8 scored cells; scored n = 0. The four Rust/Go v1 and v2 smoke grades are not scored and are reported in results. Headline: Each Rust and Go v1/v2 smoke returned 1/25. Elixir and TypeScript/Bun reference controls failed; those stacks had no task-8 smoke or scored cells. Task 8 is excluded from the language comparison.

Limit: Official smoke and control receipts are public, but versioned task-8 prompts and grader version/hash are not in this snapshot; the prompt-based diagnosis cannot be checked.

Sources: decision rule; results; test-count ledger; controls ledger; frozen picks; cell plan.

Public delivery and outcome reconciliation

No scored task-8 cells are recorded in the public cell export. The four smoke outcomes are retained as not-scored rows in the round-specific test-count ledger; admission and X-controls are in controls.jsonl.

Round results · rounds/r70-task8/RESULTS.md

STATUS: NOT-RUN for the scored cohort; no scored cells were released.

Setup and status

Status: planned n=8; scored n=0. The registered v1 design planned two reps for each of four stacks; the four rows below are unscored smoke checks. Cells used kogen-bench-us, direct Codex with gpt-6-luna at max, a 3600-second cap, and zero retries. Rule L required at least 13/25 when the reference passed 25/25; both Rust and Go v1/v2 smokes returned 1/25.

No task-8 smoke was launched for Elixir or TypeScript/Bun: their v1 reference controls failed at 24/25, so no smoke was admitted, and the later v2 scope was limited to Rust and Go. This page makes no task-8 language comparison. Per-smoke usage and cost are unverified in the permitted public receipts; they are not zero. The separate original comparison is documented in the 40-cell as-graded RvE table.

The test-count ledger contains the four officially graded smoke rows. All four Rust/Go v1 and v2 smokes returned 1/25 and failed Rule L. These smoke rows are marked not scored.

Version Language Cell ID Outcome Hidden tests passed/total Scored
v1 Rust codex__gpt-6-luna__max__default__r70-8-rust__r90 fail 1/25 no
v1 Go codex__gpt-6-luna__max__default__r70-8-go__r90 fail 1/25 no
v2 Rust codex__gpt-6-luna__max__default__r70-8-rust-v2__r1 fail 1/25 no
v2 Go codex__gpt-6-luna__max__default__r70-8-go-v2__r1 fail 1/25 no

Admission and X-control outcomes

Exact control cell IDs, source labels, timestamps, test counts, and recorded patch hashes are in controls.jsonl. None of these rows is scored.

Version and variant Reference No-op Reference with skeleton front end Reference through contestant path
v1 Rust 2 pass, each 25/25 2 fail, each 0/25 1 pass, 25/25 1 pass, 25/25
v1 Elixir 2 fail, each 24/25 2 fail, each 0/25 1 fail, 24/25 not run
v1 Go 2 pass, each 25/25 2 fail, each 0/25 1 pass, 25/25 1 pass, 25/25
v1 TypeScript/Bun 2 fail, each 24/25 2 fail, each 0/25 1 fail, 24/25 not run
v2 Rust 1 pass, 25/25 1 fail, 0/25 not run not run
v2 Go 1 pass, 25/25 1 fail, 0/25 not run not run

The Elixir and TypeScript/Bun reference controls each failed all three recorded checks at 24/25. Their task-8 smokes were not run. Rust and Go v1 and v2 smoke counts were below the Rule L threshold: a smoke must pass at least half of the reference pass fraction, which is 13/25 when the reference passes 25/25.

The public receipts establish outcomes and test counts. Versioned task-8 prompt text and grader version/hash are not in this public snapshot, so the prompt-based diagnosis cannot be checked here.

Captured run records

Standard-schema records assigned to this round in the indexed snapshot.

No per-round file is listed. Check the run-record index for the complete partition list.

Reproduction inventory

Published analysis replay and full experiment replay are reported separately.

Replay type
unavailable
Script
No per-round analysis script is published.
Inputs
No identified public input bundle.
Full execution replay
Unavailable
Scope
No public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.

View the complete per-round reproduction inventory · Repository reproduction guide