Round sol-medium

Sol medium

Status · DESCRIPTIVEReviewed status is transcribed from rounds/STATUS.md. The original narrative and its qualifications remain below.

Source-reported narrativeRound README and results text are source material. Numeric values that are not tied to a claim ledger record remain source-reported.

Round source README · rounds/sol-medium/README.md

This round compared Rust, Go, and TypeScript/Bun; descriptive full passes by language (Go, Rust, TS/Bun) were go 5/8, rust 6/8, ts-bun 6/8. The results are descriptive and establish no language decision.

Round date: 2026-10-09 Publication badge: DESCRIPTIVE; KEPT FOR AUDIT Why not VALID: Required execution metadata was not captured, so these results are descriptive and support no language ranking. Registered reasons: DESCRIPTIVE_ONLY, RAW_EVIDENCE_PARTIAL. Recomputation status: OBSERVED SOURCE ONLY Label: DESCRIPTIVE STATUS: DESCRIPTIVE

How and where it ran

Runs used Hetzner CCX13 Linux hosts, one in Europe and one in the US. At 21:27 UTC on 9 October, grading stopped copying generated build caches; regrades are explained below. Tasks were frozen at registration. The Sol medium arm was separately registered at one rep per task-language, paired by task. It was a tiebreak arm in the registered rule, after Luna max and Ksub. It ran on the Hetzner hosts and was interleaved with max cells. The primary metric was full pass and hidden-test fraction secondary; no arm pooling is used. Task authors used gpt-6.1-sol high, overlapping with this model family.

All results in this package are DESCRIPTIVE; required execution metadata was not captured.

Results

Per-cell results. P/F is the selected full-pass result. Gate-fail indicates the original official grade did not run tests; timeouts are listed separately. Regrade column shows the original official boolean and selected result where a registered regrade exists.

Task Language Exact arm Rep Recorded outcome Hidden tests passed/total Timeout Gate fail Regrade
c4b-1 go gpt-6.1-sol / medium r1 F 28/29 no no —
c4b-1 rust gpt-6.1-sol / medium r1 P 29/29 no no —
c4b-1 ts-bun gpt-6.1-sol / medium r1 F 27/29 no no —
c4b-2 go gpt-6.1-sol / medium r1 P 29/29 no no —
c4b-2 rust gpt-6.1-sol / medium r1 P 0/0 no yes F 0/0 → P 29/29 (clean copy)
c4b-2 ts-bun gpt-6.1-sol / medium r1 P 29/29 no no —
c4b-3 go gpt-6.1-sol / medium r1 P 29/29 no no —
c4b-3 rust gpt-6.1-sol / medium r1 P 0/0 no yes F 0/0 → P 29/29 (clean copy)
c4b-3 ts-bun gpt-6.1-sol / medium r1 P 29/29 no no —
c4b-4 go gpt-6.1-sol / medium r1 P 28/28 no no P 28/28 → P 28/28 (fixed suite)
c4b-4 rust gpt-6.1-sol / medium r1 P 0/0 no yes F 0/0 → P 28/28 (fixed suite)
c4b-4 ts-bun gpt-6.1-sol / medium r1 P 28/28 no no P 28/28 → P 28/28 (fixed suite)
c4b-5 go gpt-6.1-sol / medium r1 P 29/29 no no —
c4b-5 rust gpt-6.1-sol / medium r1 P 0/0 no yes F 0/0 → P 29/29 (clean copy)
c4b-5 ts-bun gpt-6.1-sol / medium r1 P 29/29 no no —
c4b-6 go gpt-6.1-sol / medium r1 F 27/28 no no —
c4b-6 rust gpt-6.1-sol / medium r1 P 0/0 no yes F 0/0 → P 28/28 (clean copy)
c4b-6 ts-bun gpt-6.1-sol / medium r1 P 28/28 no no —
ksub-1 go gpt-6.1-sol / medium r1 P 30/30 no no —
ksub-1 rust gpt-6.1-sol / medium r1 F 29/30 no no —
ksub-1 ts-bun gpt-6.1-sol / medium r1 F 29/30 no no —
ksub-2 go gpt-6.1-sol / medium r1 F 26/28 no no —
ksub-2 rust gpt-6.1-sol / medium r1 F 26/28 no no —
ksub-2 ts-bun gpt-6.1-sol / medium r1 P 28/28 no no —

This is a separate gpt-6.1-sol medium rep-1 arm and is never pooled with Luna max. Rust build-gate failures use the registered clean-copy regrade; c4b-4 also reports its fixed-suite grade.

Cell accounting

The outcome table below covers selected runs only; it does not count every started attempt.

Measure Count
Planned cells in published cohort 24
Started cells in published cohort 24
Finished cells in published cohort 24
Officially graded cells 24
ITT denominator 24

What's missing and why

  • circumstances.cap_end (24 missing): The per-cell value was not captured in the retained source records. Reason: Boundary telemetry not retained (m08).
  • circumstances.cap_start (24 missing): The per-cell value was not captured in the retained source records. Reason: Boundary telemetry not retained (m08).
  • circumstances.concurrent_cells_end (24 missing): The per-cell value was not captured in the retained source records. Reason: Host-wide concurrent cell count was not captured (m66).
  • circumstances.concurrent_cells_start (24 missing): The per-cell value was not captured in the retained source records. Reason: Host-wide concurrent cell count was not captured (m66).
  • circumstances.dispatcher_id (24 missing): The per-cell value was not captured in the retained source records. Reason: Dispatcher receipt unavailable (m12).
  • circumstances.incidents (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell incident receipt was not captured (m78).
  • circumstances.load1_end (24 missing): The per-cell value was not captured in the retained source records. Reason: Boundary telemetry not retained (m08).
  • circumstances.load1_start (24 missing): The per-cell value was not captured in the retained source records. Reason: Boundary telemetry not retained (m08).
  • circumstances.load_samples (24 missing): The per-cell value was not captured in the retained source records. Reason: No matching controller samples retained (m29).
  • circumstances.queue (24 missing): The per-cell value was not captured in the retained source records. Reason: Queue receipt unavailable (m49).
  • cost.calculator_version (24 missing): The per-cell value was not captured in the retained source records. Reason: Cost calculator version not recorded (m61).
  • cost.price_table_version (24 missing): The per-cell value was not captured in the retained source records. Reason: Price table version not recorded (m89).
  • cost.usd (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell cost and pricing inputs are not present in the public round data (m70).
  • effort.effective (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell effective-effort receipt not published (m73).
  • environment.account_class (24 missing): The per-cell value was not captured in the retained source records. Reason: No owner-approved class-only account receipt is available for the cell (m100).
  • environment.cores (24 missing): The per-cell value was not captured in the retained source records. Reason: No per-cell CPU allocation receipt (m33).
  • environment.cpu_model (24 missing): The per-cell value was not captured in the retained source records. Reason: No per-cell CPU receipt (m34).
  • environment.kernel (24 missing): The per-cell value was not captured in the retained source records. Reason: No per-cell kernel receipt (m36).
  • environment.network.allowlist_hosts (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell network allowlist not present in the public round data (m79).
  • environment.network.profile (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell network profile not present in the public round data (m80).
  • environment.os (24 missing): The per-cell value was not captured in the retained source records. Reason: Toolchain version/inventory not recorded (m54).
  • environment.ram_gib (24 missing): The per-cell value was not captured in the retained source records. Reason: No per-cell RAM receipt (m35).
  • environment.toolchains.elixir (24 missing): The per-cell value was not captured in the retained source records. Reason: Toolchain version/inventory not recorded (m54).
  • environment.toolchains.erlang (24 missing): The per-cell value was not captured in the retained source records. Reason: Toolchain version/inventory not recorded (m54).
  • environment.toolchains.node (24 missing): The per-cell value was not captured in the retained source records. Reason: Toolchain version/inventory not recorded (m54).
  • environment.toolchains.other_inventory (24 missing): The per-cell value was not captured in the retained source records. Reason: Toolchain version/inventory not recorded (m54).
  • environment.toolchains.python (24 missing): The per-cell value was not captured in the retained source records. Reason: Toolchain version/inventory not recorded (m54).
  • environment.toolchains.ruby (24 missing): The per-cell value was not captured in the retained source records. Reason: Toolchain version/inventory not recorded (m54).
  • environment.toolchains.rust (24 missing): The per-cell value was not captured in the retained source records. Reason: Toolchain version/inventory not recorded (m54).
  • grade.grader (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell grader software version is not in the public data (m77).
  • grade.timestamp (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell official repair-grade timestamp was not retained in the public record (m101).
  • host.cpu (24 missing): The per-cell value was not captured in the retained source records. Reason: No per-cell CPU receipt (m34).
  • host.kernel (24 missing): The per-cell value was not captured in the retained source records. Reason: No per-cell kernel receipt (m36).
  • host.os (24 missing): The per-cell value was not captured in the retained source records. Reason: Toolchain version/inventory not recorded (m54).
  • host.ram_gib (24 missing): The per-cell value was not captured in the retained source records. Reason: No per-cell RAM receipt (m35).
  • host.spec_ref (24 missing): The per-cell value was not captured in the retained source records. Reason: No measured host-spec reference (m30).
  • host.vcpu (24 missing): The per-cell value was not captured in the retained source records. Reason: No per-cell CPU allocation receipt (m33).
  • model.effective (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell effective-model receipt not published (m74).
  • sandbox.egress_allow (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell egress allowlist is not present in the public round data (m75).
  • sandbox.egress_profile (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell egress profile is not present in the public round data (m76).
  • sandbox.profile (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell sandbox profile is not present in the public round data (m87).
  • sandbox.profile_sha256 (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell sandbox profile fingerprint is not present in the public round data (m86).
  • setup.adapter_harness_sha (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell Codex adapter fingerprint not recorded in the public round data (m67).
  • setup.deps_source (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell dependency source is not recorded (m71).
  • setup.sandbox_mode (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell sandbox mode is not present in the public round data (m85).
  • setup.sandbox_profile_sha256 (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell sandbox fingerprint is not present in the public round data (m84).
  • stop_reason (24 missing): The per-cell value was not captured in the retained source records. Reason: Runner status does not establish normalized stop cause (m50).
  • timestamps.attempts (24 missing): The per-cell value was not captured in the retained source records. Reason: Attempt boundary receipts unavailable (m06).
  • timestamps.cell.end_utc (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell contestant end timestamp not recorded in the public round data (m68).
  • timestamps.cell.start_utc (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell contestant start timestamp not recorded in the public round data (m69).
  • timestamps.phases.develop.end_utc (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase timestamp not present in the public per-cell data (m88).
  • timestamps.phases.develop.start_utc (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase timestamp not present in the public per-cell data (m88).
  • timestamps.phases.gate.end_utc (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase timestamp not present in the public per-cell data (m88).
  • timestamps.phases.gate.start_utc (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase timestamp not present in the public per-cell data (m88).
  • timestamps.phases.grade.end_utc (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase timestamp not present in the public per-cell data (m88).
  • timestamps.phases.grade.start_utc (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase timestamp not present in the public per-cell data (m88).
  • timestamps.phases.plan.end_utc (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase timestamp not present in the public per-cell data (m88).
  • timestamps.phases.plan.start_utc (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase timestamp not present in the public per-cell data (m88).
  • timestamps.phases.review.end_utc (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase timestamp not present in the public per-cell data (m88).
  • timestamps.phases.review.start_utc (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase timestamp not present in the public per-cell data (m88).
  • timestamps.phases.setup.end_utc (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase timestamp not present in the public per-cell data (m88).
  • timestamps.phases.setup.start_utc (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase timestamp not present in the public per-cell data (m88).
  • timestamps.phases.shape.end_utc (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase timestamp not present in the public per-cell data (m88).
  • timestamps.phases.shape.start_utc (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase timestamp not present in the public per-cell data (m88).
  • timing.phases_s.develop (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase wall not emitted or not separable (m45).
  • timing.phases_s.gate (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase wall not emitted or not separable (m45).
  • timing.phases_s.grade (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase wall not emitted or not separable (m45).
  • timing.phases_s.plan (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase wall not emitted or not separable (m45).
  • timing.phases_s.review (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase wall not emitted or not separable (m45).
  • timing.phases_s.setup (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase wall not emitted or not separable (m45).
  • timing.phases_s.shape (24 missing): The per-cell value was not captured in the retained source records. Reason: Phase wall not emitted or not separable (m45).
  • timing.total_wall_s (24 missing): The per-cell value was not captured in the retained source records. Reason: Wall counter unavailable (m58).
  • tokens.phases.develop.cached_input (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.develop.input (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.develop.output (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.develop.reasoning (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.gate.cached_input (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.gate.input (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.gate.output (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.gate.reasoning (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.grade.cached_input (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.grade.input (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.grade.output (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.grade.reasoning (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.plan.cached_input (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.plan.input (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.plan.output (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.plan.reasoning (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.review.cached_input (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.review.input (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.review.output (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.review.reasoning (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.setup.cached_input (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.setup.input (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.setup.output (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.setup.reasoning (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.shape.cached_input (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.shape.input (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.shape.output (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.phases.shape.reasoning (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-phase token counter not emitted (m44).
  • tokens.total.cached_input (24 missing): The per-cell value was not captured in the retained source records. Reason: Usage counter unavailable (m56).
  • tokens.total.input (24 missing): The per-cell value was not captured in the retained source records. Reason: Usage counter unavailable (m56).
  • tokens.total.output (24 missing): The per-cell value was not captured in the retained source records. Reason: Usage counter unavailable (m56).
  • tokens.total.reasoning (24 missing): The per-cell value was not captured in the retained source records. Reason: Usage counter unavailable (m56).
  • tools.codex_cli (24 missing): The per-cell value was not captured in the retained source records. Reason: Historical tool version not pinned in cell artifacts (m18).
  • tools.grader (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell grader software version is not in the public data (m77).
  • tools.harness (24 missing): The per-cell value was not captured in the retained source records. Reason: Historical tool version not pinned in cell artifacts (m18).
  • tools.runner (24 missing): The per-cell value was not captured in the retained source records. Reason: Per-cell runner version not recorded in the public round data (m83).
  • tools.toolchains.elixir (24 missing): The per-cell value was not captured in the retained source records. Reason: Toolchain version/inventory not recorded (m54).
  • tools.toolchains.erlang (24 missing): The per-cell value was not captured in the retained source records. Reason: Toolchain version/inventory not recorded (m54).
  • tools.toolchains.node (24 missing): The per-cell value was not captured in the retained source records. Reason: Toolchain version/inventory not recorded (m54).
  • tools.toolchains.other_inventory (24 missing): The per-cell value was not captured in the retained source records. Reason: Toolchain version/inventory not recorded (m54).
  • tools.toolchains.python (24 missing): The per-cell value was not captured in the retained source records. Reason: Toolchain version/inventory not recorded (m54).
  • tools.toolchains.ruby (24 missing): The per-cell value was not captured in the retained source records. Reason: Toolchain version/inventory not recorded (m54).
  • tools.toolchains.rust (24 missing): The per-cell value was not captured in the retained source records. Reason: Toolchain version/inventory not recorded (m54).

Reproduction metadata

  • Token and wall-time values are included where retained in the public source table; other values are declared missing.

  • Pre-registration: yes. Status remains descriptive; required execution metadata was not captured.

  • Kogen commit: not applicable to direct Codex cells; no per-cell Kogen commit was recorded.

  • Harness commit: not recorded in the public cell evidence.

  • Model and effort: shown for every cell in the results table.

  • Task IDs: c4b-1, c4b-2, c4b-3, c4b-4, c4b-5, c4b-6, ksub-1, ksub-2.

  • Historical execution command: not retained per cell.

  • Raw records: data/cells.csv and records.jsonl where exact source IDs are available.

Captured run records

Standard-schema records assigned to this round in the indexed snapshot.

No per-round file is listed. Check the run-record index for the complete partition list.

Reproduction inventory

Published analysis replay and full experiment replay are reported separately.

Replay type
analysis-only
Script
No per-round analysis script is published.
Inputs
Full execution replay
Unavailable
Scope
The published source table and Standard records preserve available cell outcomes; raw execution receipts are incomplete and full model execution cannot be replayed.

View the complete per-round reproduction inventory · Repository reproduction guide