Round r70-rve-ext

r70-RvE language extension

Status · VALIDReviewed status is transcribed from rounds/STATUS.md. The original narrative and its qualifications remain below.

Source-reported narrativeRound README and results text are source material. Numeric values that are not tied to a claim ledger record remain source-reported.

Round source README · rounds/r70-rve-ext/README.md

Round date: HISTORICAL (before 2026-10-09) Publication badge: VALID; KEPT FOR AUDIT Recomputation status: OBSERVED SOURCE ONLY

Recomputation update: The mined raw records now provide per-cell outcome, manifest token-counter, and wall-time recomputation for the observed cohort. This does not reconstruct missing planned cells, a full intention-to-treat denominator, or model execution.

Status

VALID — registered reps 31–32 only; rep 33 is post-hoc and reported separately

Limits

  • Restored to VALID 2026-10-08: the round executed its registered rule as registered and its numbers recompute; the points below are limits on public evidence, not contradictions in the round's own files.
  • PARTIAL_PREREG — Only reps 31–32 are stated as pre-registered; rep 33 is explicitly post-hoc.
  • NO_PREREG_EVIDENCE — The files do not establish that the limited registration predates the first result.
  • SCOPE — The reported cohort mixes registered and post-hoc cells.

Required reproduction metadata

  • Kogen commit: not recorded in the cited round source.
  • Harness commit: not recorded in the cited round source as a Git commit; any adapter or runtime fingerprints are not Git commits.
  • Model and effort: a complete per-cell mapping is not recorded in the cited round source; values present in the source summary remain as stated above.
  • Task IDs: r70-2-elixir, r70-2-go, r70-2-rust, r70-2-ts-bun, r70-5-elixir, r70-5-go, r70-5-rust, r70-5-ts-bun, r70-7-elixir, r70-7-go, r70-7-rust, r70-7-ts-bun; the complete scored-cell mapping is not recorded where noted above.
  • Historical execution command: not recorded in the cited round source.
  • Raw records: the cited sources do not provide a complete public round-specific request and grade bundle; public delivery and outcome exports are linked above where applicable.

Terminology: public round glossary.

STATUS: VALID — registered decision covers the 24 pre-registered rep 31–32 cells; the 5 post-hoc rep-33 cells are reported separately and are outside the registered rule.

Data completeness (standard run-record capture): N/A (0 deliveries; NO DELIVERED DATA). Supplemental evidence: 29 official grades; 60 control receipts. Measurement contract; declared gaps.

Pre-registered: yes for reps 31–32; design date 6 October 2026. Label: DESCRIPTIVE Question: What hidden-suite full-pass outcomes did Rust, Elixir, Go, and TypeScript/Bun record on tasks 2, 5, and 7, alongside the prior Round 70 language observations? n: 24 pre-registered cells at reps 31–32; 5 post-hoc rep-33 cells; 29 official grades total. Headline: Reps 31–32 per stack: Rust 5/6, Elixir 4/6, Go 4/6, and TypeScript/Bun 6/6. Evidence note: The public test-count and controls ledgers identify the rep-31 smokes, admission controls, and contestant-path X-controls. All smokes met Rule L; admission controls and X-controls had their expected outcomes. Configuration: direct Codex, gpt-6-luna at max; n=2 per task and stack for pre-registered reps 31–32, with selected post-hoc rep-33 additions. Rust and Elixir ran on kogen-bench-eu; Go and TypeScript/Bun ran on kogen-bench-us. Wall comparisons are within-host only. Per-cell token and wall medians are reported separately for pre-registered reps 31–32 and post-hoc rep 33 in the results; the cost/time ledger provides per-cell token and wall fields. Original cohort context: 40-cell as-graded RvE comparison. Limit: Descriptive, small per-task samples, one model, and author-selected task templates. The extension observations do not establish differences among stacks.

Sources: decision rule; results; test-count ledger; cost/time ledger; controls ledger; sanitized official grade rows; public cell export.

Recomputed from raw records

`recomputed.json`, mined cell records, and the token audit back observed official outcomes, per-arm pass counts/rates, manifest token counters, and wall medians. The source contains 41 rows (fail 6, pass 35); overall pass rate is 85.4% (35/41) among pass/fail grades. Per-arm detail and coverage counts are in the JSON.

Published-export outcome cross-check: 29/29 shared cell IDs match; 0/29 public-export rows have no mined source record. Exact IDs and results for unpaired rows are in recomputed.json. Arm-label mismatches between public and raw records: 29; exact labels and cell IDs are in recomputed.json. No per-cell outcome mismatches were found in the shared IDs. Token reconciliation under METHOD.md: 29/29 published metadata totals equal the manifest input + cached input + output sum; 0 differ (no mismatches). Reasoning remains a separate reported component and is not added to output. The recompute keeps both numbers visible. 0 unexplained single-response numeric errors were identified; multi-response rows retain the published full-cell total and the manifest counter separately because per-response usage records are excluded. This round also has 12 raw-only or non-public rows; they are kept separate from the published-export denominator.

Round results · rounds/r70-rve-ext/RESULTS.md

STATUS: VALID — DESCRIPTIVE

The primary outcome is official hidden-suite full pass. A timeout counts as a failure. The public cell-level test-count ledger identifies the 40 original RvE cells, 15 fixed-front-end rerun cells, and 29 extension cells. Earlier invalidated grade rows are excluded; the latest official row per exact cell ID is retained.

Configuration and sample: direct Codex, gpt-6-luna at max. The pre-registered extension used n=2 per task and stack at reps 31–32; rep 33 is a selected post-hoc addition. Rust and Elixir ran on kogen-bench-eu; Go and TypeScript/Bun ran on kogen-bench-us. Wall time is comparable only within a host. Effective model/effort and harness-version receipts are unavailable for these supplemental rows. Original cohort context: 40-cell as-graded RvE comparison.

Extension outcomes

Pre-registered reps 31–32 per stack: Rust 5/6, Elixir 4/6, Go 4/6, and TypeScript/Bun 6/6. Rep 33 is a separate post-hoc addition.

Per-task counts include the official extension grades at reps 31–33 where present. Test fractions and exact cell IDs are in test-counts.jsonl.

Task ID Exact arm label in export Passes (all included reps)
r70-2-rust rust 2/2
r70-2-elixir elixir 1/3
r70-2-go go 2/3
r70-2-ts-bun ts-bun 2/2
r70-5-rust rust 2/2
r70-5-elixir elixir 2/2
r70-5-go go 2/3
r70-5-ts-bun ts-bun 2/2
r70-7-rust rust 2/3
r70-7-elixir elixir 2/3
r70-7-go go 2/2
r70-7-ts-bun ts-bun 2/2
Language Reps 31–32 Reps 31–33 (rep 33 post-hoc)
Rust 5/6 6/7
Elixir 4/6 5/8
Go 4/6 6/8
TypeScript/Bun 6/6 6/6

These counts describe the observed cells. They do not establish differences among stacks. The timeout cell remains a failure even though its delivered state received a full test count.

Controls

The controls ledger contains the admission and X-control cell IDs, outcomes, test counts, timestamps, and available patch hashes. The 48 reference/no-op admission controls had their expected outcomes: every reference passed and every no-op failed. All 12 contestant-path X-controls passed. The admission controls and X-controls ran on kogen-bench-eu, while the Go and TypeScript/Bun cells ran on kogen-bench-us; the controls validate the task and the grader, not the host. The 12 rep-31 smoke cells met Rule L. Controls are not scored cells; the rep-31 smokes are scored and counted in the 24 pre-registered cells.

Seven-task descriptive counts

Task-4 admission controls failed for Rust, Elixir, and TypeScript/Bun. The pre-registered fixed-front-end rerun cells are shown in a separate column; original as-graded outcomes are kept. These totals do not establish a corrected stack comparison.

Language Original tasks 1/3/4/6 (as graded) FE2 rerun cohort (15 cells) Original + extension reps 31–32 Original + extension reps 31–33
Rust 8/10 3/3 13/16 14/17
TypeScript/Bun 7/10 2/3 13/16 13/16
Go 7/10 3/3 11/16 13/18
Elixir 2/10 3/6 6/16 7/18

Original as-graded outcomes plus extension reps 31–32, excluding the separate FE2 rerun cells: Rust 13/16, TypeScript/Bun 13/16, Go 11/16, and Elixir 6/16.

Illustrative mixed cohort with FE2 substitution

This is an illustrative mixed cohort, not a corrected estimate. It substitutes the separate FE2 task-1 Elixir and task-4 results into the original tasks 1/3/4/6 cohort, then adds the extension. Task-4 admission controls failed or were invalid for three stacks, the cohorts differ, and rep 33 is post-hoc. Do not treat this table as corrected or pooled with the as-graded observations above.

Stack Mixed cohort + extension reps 31–32 Same mix + reps 31–33 (rep 33 post-hoc)
Rust 15/16 16/17
TypeScript/Bun 15/16 15/16
Go 13/16 15/18
Elixir 9/16 10/18

The public receipts report outcomes, test counts, cell IDs, timestamps, and available patch hashes. Per-cell token and wall observations are in the cost/time ledger.

Stack Host Reps 31–32 median total tokens (headline) Reps 31–32 median wall s (headline) Reps 31–33 median total tokens (post-hoc) Reps 31–33 median wall s (post-hoc)
Rust kogen-bench-eu 384,939 331 397,284 283
Elixir kogen-bench-eu 991,440 891 991,440 891
Go kogen-bench-us 415,341 326 417,204 326
TypeScript/Bun kogen-bench-us 624,256 463 624,256 463

Wall time is comparable only within a host. Task 8 remains a separate not-run scored cohort; its smoke and control receipts are in task 8 results.

Publishability

  • The extension is descriptive, with two or three cells per task and stack, one model, and author-selected task templates.
  • Rep 33 was added under a rule recorded before those cells ran and remains labelled post-hoc.
  • The combined counts retain the original, rerun, and extension cohorts as separate, traceable rows in the public ledger.

Captured run records

Standard-schema records assigned to this round in the indexed snapshot.

No per-round file is listed. Check the run-record index for the complete partition list.

Reproduction inventory

Published analysis replay and full experiment replay are reported separately.

Replay type
unavailable
Script
No per-round analysis script is published.
Inputs
No identified public input bundle.
Full execution replay
Unavailable
Scope
No public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.

View the complete per-round reproduction inventory · Repository reproduction guide