Reproduction guide

Reproduce

Check published records, inspect the available task bases and graders, and see what is still missing for a full rerun.

Verify published numbers · Rerun and regrade tasks

Analysis replay and experiment execution are differentFollow the repository's stated availability for each round. A public record transformation does not promise that an experiment can be rerun or officially graded.

Reproduction source guide · reproduce/README.md

Reproduce the public records

Requires Python 3 (standard library only). Run these commands from the repository root. For the claim-by-claim verification map, see VERIFY.md; for rerunning and regrading tasks, see RERUN.md.

python3 reproduce/export_results.py
python3 reproduce/build_records.py
python3 reproduce/build_grade_join.py
python3 reproduce/build_l3b_records.py
python3 reproduce/audit_rounds.py
python3 reproduce/r70_completeness.py --write
python3 reproduce/validate_repo.py

export_results.py reads the committed outcome-only grade snapshot and numeric cell metadata, then adds the 15 scored FE2 rerun grades from results/test-counts.jsonl under their separate round tag. It rebuilds results/cells.csv, results/cells.jsonl, results/unmapped.json, and results/export-report.json; it does not rebuild the indexed run-record files or the validation summary. build_records.py reads the indexed run, context, and ungraded evidence under reproduce/inputs/, then rebuilds results/run-records/<round_id>.jsonl, unassigned.jsonl, and their index. Run build_grade_join.py afterward to refresh captured-delivery references and rebuild the per-round files under results/source-crosswalk/. Each family index records row counts, byte lengths, and checksums. audit_rounds.py separately rewrites round audit documents and the validation summary; review its scope before running it.

build_grade_join.py rebuilds the official-export cross-check CSV and the corresponding captured-delivery links in the indexed results/source-crosswalk/ partitions. It verifies exact equality and uniqueness for all 5,020 captured IDs against the public export; no capture-reported-only mappings remain.

build_l3b_records.py converts the three sanitized L3b pilot source records in reproduce/inputs/l3b-pilot-records.jsonl to the published compact schema 1.2 round records. The other 21 L3b scored cells have official grade rows but no retained Standard run records.

The exporter keeps the latest official result per cell ID, reports duplicate reconciliation counts, and preserves null usage and cost as missing values. Invalid control_apply results stay marked invalid and are not counted as model failures. It does not read transcripts, private references, grader code, or sealed inputs, and does not grade, authenticate, connect to a host, or launch a cell. Venue identifiers are retained; hardware and operating-system details are generally withheld except for the scoped r70 compile diagnostic described in STANDARD.md.

r70_completeness.py --write rebuilds the 64-row Round 70 cost/time ledger and its documented summaries from public test-count, own-check, and sanitized resource/timing inputs in this repository. It also regenerates the original RvE, rerun, mixed-cohort, compile diagnostic, and transcript wall tables. The 44 rerun/extension rows preserve nulls for unverified cost, reasoning, effective model/effort, and harness version; no cost is inferred from token counts.

The per-round reproduction inventory lists one row per registered round with its script, identified inputs, replay type, and full-execution replay availability. analysis-only means a published-data transformation or scoring replay; it does not repeat historical model calls. No registered round currently has a full execution replay in this snapshot.

mine_probe.py and recompute_mined.py provide a separate observed-source audit for 105 mined round tags. The miner reads the recovered source manifests and eligible patch files read-only; the recomputer writes safe mined records, per-round recomputed.json files, and token comparisons. This supports only observed rows and does not reproduce the full published round headlines, a planned denominator, or historical execution. See TOKEN-AUDIT.md for the token definition and mismatch decisions.

This is an official-grade snapshot, not every planned delivery. It cannot establish a full intention-to-treat denominator on its own: historical round records account for ungraded timeouts, cancellations, and corrections. External ledgers require matching approved numeric metadata. Cost estimates in the committed records are source-reported API equivalents; the calculator and analysis inputs are absent, so the calculations are not re-derivable from the public record.

Task hidden suites and graders are shipped under tasks/<id>/hidden/ and tasks/<id>/grader/; they are for post-run grading and must not be visible to the agent. Reruns require Linux, Bubblewrap (bwrap), the task's documented dependencies, and your own Codex login. The host kit now includes setup-host.sh, doctor.sh, and run-controls.sh; fresh-machine verification passed for tasks 1, 5 and 7 (see Scope below). See RERUN.md. Before any scored release, follow the release checklist, including officially graded real-sandbox smoke cells for every arm.

Offline dependency snapshots

Grading runs offline. The r70 kits read prebuilt dependency templates from the host, which setup-host.sh builds from the lockfiles in `deps/`. The package managers verify the lockfile checksums.

  • Elixir: toolchains/elixir-1.20.2-otp29/{mix-home,hex-home,project}, built from deps/elixir/mix.exs and mix.lock with Hex 2.5.1 (built from its SHA-256-pinned source tag on the host OTP), deps.get --check-locked, deps.compile and the Dialyzer PLTs (MIX_ENV=test).
  • Bun: toolchains/bun-1.4.2/node-template, built by bun install --frozen-lockfile from deps/bun/.
  • Gleam: toolchains/gleam-1.18.1/cache/project, built by gleam deps download and gleam build from deps/gleam/.
  • Go: golangci-lint 2.14.0 in toolchains/go-1.27.1/bin, pinned by release-archive and binary SHA-256, plus an empty cache/gomodcache.
  • Rust: toolchains/rust-1.97.1/vendor, the Cargo directory source that the r70 Rust references' .cargo/config.toml points at. It is rebuilt from deps/rust/vendor.tsv (160 crates, each verified against its crates.io SHA-256) by deps/rust/build_vendor.py, which writes cargo vendor-identical checksum files. There is also an empty cache/cargo-home; the r70 Go kits declare no module requirements.

Each template carries a .kogen-lock-sha256 marker that binds it to the shipped lockfile bytes, and doctor.sh checks the markers. These snapshots replace the dependency sets that were provisioned by hand on the original hosts. Dependency trees are never committed.

Scope: the Linux host kit targets the r70-family kits. On a freshly installed host (9 October 2026), the admission controls admitted every stack of tasks 1, 5 and 7. Tasks 2, 3, 4, 6 and 8 are not yet rerunnable from this kit: several of their published grader entry points need repair, and only the Zig kit of task 8 is published. The Rails and Phoenix kits were graded on macOS (their grader entry points use macOS tool paths), and 18 kits have no bundled base; neither group is rerunnable on a Linux host from this kit. The published fresh-host proof runs the admission controls on the six task-1 kits (r70-1-rust, r70-1-go, r70-1-elixir, r70-1-ts-bun, r70-1-gleam and r70-1-zig); a sweep over all kits is optional for anyone rerunning older rounds.

Standard records and historical gaps

python3 reproduce/build_records.py
python3 reproduce/validate_round.py --round r53
python3 reproduce/validate_round.py --round r53 --strict

The standard-record builder replays the committed sanitized evidence snapshot. Schema 1.2 stores missing values as short codes; missing-reasons.json maps each code to its original reason and reconstruction source. Source locations that are absent from the public tree are identified as unavailable and not independently reproducible. The index records each partition's schema version, byte length, and SHA-256; unassigned records are listed in results/run-records/unassigned.jsonl. The validator checks every evidence index, record schema, declared historical gaps, duplicate identities, and empty cohorts. The --strict command requires schema 1.2, a nonempty cohort, and an exact gap declaration. A historical record set with explicitly listed protocol deviations can pass as PASS WITH DECLARED DEVIATIONS; this does not make the round release-compliant.

python3 reproduce/expand_records.py --output-dir "$TMPDIR/run-records-expanded" restores the per-round run-record files to their previous verbose schema 1.1 form. python3 reproduce/test_compact_record_round_trip.py checks byte-for-byte expansion of every published run-record row and its index.

python3 reproduce/validate_release.py checks the historical round inventory for a publication badge, a recomputation status, and a Why not VALID explanation wherever the registered status is not VALID. It then applies strict Standard validation to every round dated 2026-10-09 or later: all required capture fields must be present and no protocol deviations may be declared. Earlier rounds remain historical records with their original status and explicit limits. The publication evidence register must also be resolved. See VERIFY.md for the full verification sequence and expected limits, and RERUN.md for task reruns.

The release validator also runs record_contract.py on the checked-in synthetic cell in fixtures/record-contract/. The fixture has a frozen plan, transcript with usage, sanitized grade receipt, and host snapshots; it contains no task prompt or hidden tests. The contract calls the same standard_record.finalize writer used by run-lane.sh, then validates its output strictly against the current Standard schema with zero missing fields. This catches both schema changes the writer has not learned to fill and writer output that no longer matches the schema. The protected fixture manifest and grade receipt are read only through safe_rows.py.

audit_rounds.py explicitly regenerates historical gap declarations, measurement records, and the validation summary; it also writes round files. Review those generated changes before keeping them. The validator never writes declarations. See STANDARD.md, the schema, and the public record mapping.

The committed capture includes graded identities and surviving ungraded deliveries, separate from the older cells.* outcome export. Run, context, and ungraded input records are split by round under their respective directories, each with an index.json. Their cutoff and counts are in capture provenance. Context provenance uses stable source IDs; source paths and hardware details are withheld. Source hashes are source-reported and not independently verifiable when the underlying receipts are absent.

collect_context_metadata.py VENUE PUBLIC_ROOT --results-root DIR --event-root DIR is an optional read-only reader for explicitly public metadata roots. Configured inputs must remain under the supplied root; never pass sealed, grader, production, or private data. Keep intermediate output outside the repository. backfill_context.py converts approved, sanitized public evidence into records. Neither script authenticates, deploys, launches, or grades.

Public reproduction scripts

Script links come from tracked paths under reproduce/.

build_requests.pyreproduce/api-batch-review-landed-commits-20261104/build_requests.pyreport.pyreproduce/api-batch-review-landed-commits-20261104/report.pysubmit_batch.pyreproduce/api-batch-review-landed-commits-20261104/submit_batch.pyanalyze.pyreproduce/api-patch-screen-2nd-host/analyze.pyprobe.pyreproduce/api-prompt-cache-reuse-probe-20261104/probe.pyanalyze.pyreproduce/apply-patch-edit5/analyze.pyanalyze-round.pyreproduce/apply-patch-edit5/probes/analyze-round.pyprepare-edit5.pyreproduce/apply-patch-edit5/probes/prepare-edit5.pyrun-edit5.pyreproduce/apply-patch-edit5/probes/run-edit5.pyreport.pyreproduce/approval-delay-evidence/report.pyaudit_cell.pyreproduce/audit_cell.pyaudit_round70_evidence.pyreproduce/audit_round70_evidence.pyaudit_rounds.pyreproduce/audit_rounds.pyreport.pyreproduce/b1-base-qualification-go-eu2/report.pyreport.pyreproduce/b1-base-qualification-go-pooled/report.pybackfill_context.pyreproduce/backfill_context.pyreplay.pyreproduce/baseline-check-reuse-prototype/replay.pyanalyze.pyreproduce/batch-tool-confirm-12task/analyze.pyrun-batchconf.pyreproduce/batch-tool-confirm-12task/probes/run-batchconf.pyanalyze.pyreproduce/batch-tool-short-test/analyze.pyrun-batch-tool-short-test.pyreproduce/batch-tool-short-test/probes/run-batch-tool-short-test.pyanalyze2.pyreproduce/brief-compaction-detection-p2-20261105/analyze2.pycc2.pyreproduce/brief-compaction-detection-p2-20261105/cc2.pyextract2.pyreproduce/brief-compaction-detection-p2-20261105/extract2.pylive.pyreproduce/brief-compaction-detection-p2-20261105/live.pystub2.pyreproduce/brief-compaction-detection-p2-20261105/stub2.pyanalyze.pyreproduce/brief-marker-compaction-p1-20261105/analyze.pycc.pyreproduce/brief-marker-compaction-p1-20261105/cc.pyextract.pyreproduce/brief-marker-compaction-p1-20261105/extract.pystub.pyreproduce/brief-marker-compaction-p1-20261105/stub.pyreport.pyreproduce/build-release-mainhead-20261104/report.pyanalyze.pyreproduce/build-speed-e1-approve-batch-20261105/data/analyze.pymkneg.pyreproduce/build-speed-e1-approve-batch-20261105/data/mkneg.pyreport.pyreproduce/build-speed-e1-approve-batch-20261105/report.pyreport.pyreproduce/build-speed-e2-go-lock-20261105/report.pyanalyze_e2.pyreproduce/build-speed-e2-go-lock-20261105/scripts/analyze_e2.pye2.pyreproduce/build-speed-e2-go-lock-20261105/scripts/e2.pymix.pyreproduce/build-speed-e2-go-lock-20261105/scripts/mix.pysched.pyreproduce/build-speed-e2-go-lock-20261105/scripts/sched.pysim.pyreproduce/build-speed-e2-go-lock-20261105/scripts/sim.pyreport.pyreproduce/build-speed-e3-verify-load-20261105/report.pyanalyze_e3.pyreproduce/build-speed-e3-verify-load-20261105/scripts/analyze_e3.pye3.pyreproduce/build-speed-e3-verify-load-20261105/scripts/e3.pyreport.pyreproduce/build-speed-e6-fsevents-temp-trees-20261105/report.pyreport.pyreproduce/build-speed-e6d-assembly-single-member-extract-20261105/report.pymkc.pyreproduce/build-speed-e6d-assembly-single-member-extract-20261105/scripts/mkc.pybuild_grade_join.pyreproduce/build_grade_join.pybuild_l3b_records.pyreproduce/build_l3b_records.pybuild_records.pyreproduce/build_records.pybuild_scored_round_records.pyreproduce/build_scored_round_records.pycache_replay_tables.pyreproduce/cache_replay_tables.pycandidate_revision.pyreproduce/candidate_revision.pyanalyze.pyreproduce/cco-mb-hard-tasks/analyze.pyccmac-cost.pyreproduce/cco-mb-hard-tasks/probes/ccmac-cost.pyanalyze.pyreproduce/ccs2-mb-hard-tasks/analyze.pyccmac-cost.pyreproduce/ccs2-mb-hard-tasks/probes/ccmac-cost.pyanalyze.pyreproduce/ccs3-lm-hard-tasks/analyze.pyccmac-cost.pyreproduce/ccs3-lm-hard-tasks/probes/ccmac-cost.pycheck_standard_cell.pyreproduce/check_standard_cell.pyclassify-control-logs.pyreproduce/classify-control-logs.pycollect_context_metadata.pyreproduce/collect_context_metadata.pyanalyze3.pyreproduce/compaction-watcher-token-drop-p3a-20261105/analyze3.pydetect3.pyreproduce/compaction-watcher-token-drop-p3a-20261105/detect3.pyreal3.pyreproduce/compaction-watcher-token-drop-p3a-20261105/real3.pywatch3.pyreproduce/compaction-watcher-token-drop-p3a-20261105/watch3.pyanalyze.pyreproduce/confirm5-clean-route-8tasks/analyze.pyanalyze.pyreproduce/containment-macos-native-20261015/analyze.pycmp_mac.pyreproduce/containment-macos-native-20261015/harness/denied/cmp_mac.pymac-fixtures.pyreproduce/containment-macos-native-20261015/harness/denied/fixtures/mac-fixtures.pytest_cmp_mac.pyreproduce/containment-macos-native-20261015/harness/denied/test_cmp_mac.pycmp_mac.pyreproduce/containment-macos-native-20261015/harness/native3/cmp_mac.pymac-fixtures.pyreproduce/containment-macos-native-20261015/harness/native3/fixtures/mac-fixtures.pytest_cmp_mac.pyreproduce/containment-macos-native-20261015/harness/native3/test_cmp_mac.pycmp_mac.pyreproduce/containment-macos-native-20261015/harness/net/cmp_mac.pymac-fixtures.pyreproduce/containment-macos-native-20261015/harness/net/fixtures/mac-fixtures.pytest_cmp_mac.pyreproduce/containment-macos-native-20261015/harness/net/test_cmp_mac.pycontrol-worker.pyreproduce/control-worker.pycost_calc.pyreproduce/cost_calc.pyreport.pyreproduce/delivery-path-timing/report.pybuild_vendor.pyreproduce/deps/rust/build_vendor.pyegress-bridge.pyreproduce/egress-bridge.pyegress-proxy.pyreproduce/egress-proxy.pyexpand_records.pyreproduce/expand_records.pyexport_results.pyreproduce/export_results.pygenerate-host-plan.pyreproduce/generate-host-plan.pyanalyze.pyreproduce/h2h30-sub-route-tokens/analyze.pyanalyze.pyreproduce/hard-task-review-value/analyze.pyextract-reviews.pyreproduce/hard-task-review-value/probes/extract-reviews.pyanalyze.pyreproduce/hard-tasks-grading-correction/analyze.pyregrade.pyreproduce/hard-tasks-grading-correction/regrade/regrade.pyanalyze.pyreproduce/highwall-write-check/analyze.pyprepare-screen-extension.pyreproduce/highwall-write-check/probes/prepare-screen-extension.pyrun-write-check-short-test.pyreproduce/highwall-write-check/probes/run-write-check-short-test.pyanalyze.pyreproduce/hk2-hard-set-grading-correction/analyze.pyanalyze.pyreproduce/hk2-hard-set/analyze.pyanalyze.pyreproduce/hk3-lm-hard-tasks/analyze.pyanalyze.pyreproduce/hk3-mb-hard-tasks/analyze.pyanalyze.pyreproduce/hk4-lm-hard-tasks/analyze.pyreport.pyreproduce/kogen-aor-claude-ceiling-514dab4-eu/report.pyreport.pyreproduce/kogen-aor-claude-ceiling-514dab4-misetrust-eu/report.pyreport.pyreproduce/kogen-aor-claude-ceiling-514dab4-misetrust/report.pyreport.pyreproduce/kogen-aor-claude-ceiling-514dab4-r2/report.pyreport.pyreproduce/kogen-aor-claude-ceiling-514dab4/report.pyreport.pyreproduce/kogen-aor-control-514dab4-st2/report.pyreport.pyreproduce/kogen-aor-control-514dab4/report.pyreport.pyreproduce/kogen-aor-control-573d023/report.pyreport.pyreproduce/kogen-aor-freeze-baseline-3742abe/report.pyreport.pyreproduce/kogen-aor-kit-evaluate-ids/report.pyanalyze.pyreproduce/kogen-approve-deadline-load/analyze.pyreport.pyreproduce/kogen-bounded-shell-wait-e6f69a1/report.pyreport.pyreproduce/kogen-build-claude-route-confirm-c3-20261105/report.pyreport.pyreproduce/kogen-build-claude-supplied-intent-20261104/report.pyreport.pyreproduce/kogen-build-main-09b177e/report.pyreport.pyreproduce/kogen-build-mainhead-supplied-intent-20261104/report.pyreport.pyreproduce/kogen-build-nodefect-scope-269823f/report.pyreport.pyreproduce/kogen-coherent-main-a1a0445/report.pyreport.pyreproduce/kogen-coherent-set-c92d897-confirmation/report.pyreport.pyreproduce/kogen-coherent-set-c92d897/report.pyreport.pyreproduce/kogen-coherent-worseattr-confirmation-20261103/report.pyreport.pyreproduce/kogen-context-failure-packet-screen-20261104/report.pyreplay.pyreproduce/kogen-coordinator-lint-routing-e26a4d7/replay.pyreport.pyreproduce/kogen-coordinator-lint-routing-e26a4d7/report.pyreport.pyreproduce/kogen-executor-phase-api-2f13bd4/report.pyreport.pyreproduce/kogen-extraction-aa-573d023/report.pyreport.pyreproduce/kogen-loop-evidence-wave-three-kits-6e69ffb3/report.pyreport.pyreproduce/kogen-loop-line-cuts-eed48c4/report.pyreport.pyreproduce/kogen-loop-tool-error-classes-2fc1eb7/report.pybuild_data.pyreproduce/kogen-prompt-cache-cold-starts/build_data.pygolden_move.pyreproduce/kogen-prompt-cache-cold-starts/golden_move.pyreport.pyreproduce/kogen-prompt-cache-cold-starts/report.pyreport.pyreproduce/kogen-release-e6f69a1-vs-69d09e1/report.pyreport.pyreproduce/kogen-review-claude-route-confirm-c3-20261105/report.pyreport.pyreproduce/kogen-review-claude-supplied-intent-20261104/report.pyreport.pyreproduce/kogen-review-mainhead-confirm-20261104/report.pyreport.pyreproduce/kogen-review-mainhead-supplied-intent-20261104/report.pyreport.pyreproduce/kogen-setup-initial-e580996/report.pyreport.pyreproduce/kogen-shape-data-stores-816b0b0-st2/report.pyreport.pyreproduce/kogen-shape-data-stores-816b0b0/report.pyreport.pyreproduce/kogen-shaper-high-effort-e6f69a1/report.pyreport.pyreproduce/kogen-shaper-opus-claude-code-partial/report.pyreport.pyreproduce/kogen-shaper-product-decisions-5801b28/report.pyreport.pyreproduce/kogen-shaping-bounded-refusal-recovery-41d70052/report.pyreport.pyreproduce/kogen-shaping-claude-route-confirm-c3-20261105/report.pyreport.pyreproduce/kogen-shaping-data-stores-tool-13ff132-opus/report.pyreport.pyreproduce/kogen-shaping-data-stores-tool-13ff132/report.pyreport.pyreproduce/kogen-shaping-decide-non-ux-e572010/report.pyreport.pyreproduce/kogen-shaping-recovery-claude-20261105/report.pyreport.pyreproduce/kogen-shaping-recovery-confirmation-stop-41d70052/report.pyreport.pyreproduce/kogen-shaping-repair-before-terminal-submit-2192ae21/report.pyreport.pyreproduce/kogen-shaping-repair-completed-pairs-2192ae21/report.pyreport.pyreproduce/kogen-shaping-row3-verification-separation-0c8852e8/report.pyreport.pyreproduce/kogen-verification-caller-cancellation-f0884fd/report.pyreport.pyreproduce/kogen-verification-orphan-sweep-6ac87f0/report.pyreport.pyreproduce/kogen-verification-timeout-evidence-5070052/report.pyreport.pyreproduce/kogen-workspace-snapshot-e6f69a1/report.pylocalize_api.pyreproduce/kontext-api-localization-matched2-20261105/localize_api.pyreport.pyreproduce/kontext-api-localization-matched2-20261105/report.pybuild_bench-confirmation.pyreproduce/kontext-matched-2-verified-patch-time-20261105/scripts/build_bench-confirmation.pyqualify-m4.pyreproduce/kontext-matched-2-verified-patch-time-20261105/scripts/qualify-m4.pyreport_m4.pyreproduce/kontext-matched-2-verified-patch-time-20261105/scripts/report_m4.pylocalize_api.pyreproduce/kontext-packet-only-localization-api-20261104/localize_api.pyreport.pyreproduce/kontext-packet-only-localization-api-20261104/report.pyl2-plan-specificity.pyreproduce/l2-plan-specificity.pybuild_pack.pyreproduce/l3-stop-at-green-hard-kits/build_pack.pyreport.pyreproduce/lab-aor-kogen-c7-round2-20261107/report.pyreport.pyreproduce/lab-aor-shaping-no-answerer-gates-20261106/report.pyreport.pyreproduce/lab-aor-shaping-prompt-screen-20261106/report.pybuild_cases.pyreproduce/lab-aor-shaping-prompt-screen2-20261106/harness/build_cases.pyreport.pyreproduce/lab-aor-shaping-prompt-screen2-20261106/report.pytimeline.pyreproduce/lab-aor-shaping-prompt-screen2-20261106/scripts/timeline.pyzprobe.pyreproduce/lab-c1-linux-reaper-zombies-20261105/harness/fixture/zprobe.pyrun.pyreproduce/lab-c1-linux-reaper-zombies-20261105/harness/run.pysummarize.pyreproduce/lab-c1-linux-reaper-zombies-20261105/harness/summarize.pyzprobe.pyreproduce/lab-c1-reaper-confirm-20261105/harness/fixture/zprobe.pyrun.pyreproduce/lab-c1-reaper-confirm-20261105/harness/run.pyreport.pyreproduce/lab-c10-probe-git-copy-20261105/report.pyreport.pyreproduce/lab-c2-static-checks-20261105/report.pyanalyze_c6.pyreproduce/lab-c6-consumer-overcoupling-allowlist-20261105/harness/analyze_c6.pyrun-probes.pyreproduce/lab-c6-consumer-overcoupling-allowlist-20261105/harness/run-probes.pystatic_replay.pyreproduce/lab-c6-consumer-overcoupling-allowlist-20261105/static_replay.pycheck.pyreproduce/lab-c9-outcome-misread-reader-20261105/check.pyread-outcome.pyreproduce/lab-c9-outcome-misread-reader-20261105/read-outcome.pyreport.pyreproduce/lab-cc-layers-b-20261105/report.pyanalyze3_nonce.pyreproduce/lab-compaction-p3-newnonce-20261105/analyze3_nonce.pycc2.pyreproduce/lab-compaction-p3-newnonce-20261105/cc2.pyextract2.pyreproduce/lab-compaction-p3-newnonce-20261105/extract2.pystub2.pyreproduce/lab-compaction-p3-newnonce-20261105/stub2.pystub3.pyreproduce/lab-compaction-p3-newnonce-20261105/stub3.pyreplay.pyreproduce/lab-k-review-wide-stage0-20261105/replay.pyreport.pyreproduce/lab-k-round-full-suite-s1-20261105/report.pyreport.pyreproduce/lab-k-shape-scope-enumeration-s1-20261105/report.pymsg.pyreproduce/lab-k-verify-isolated-s1-20261105/harness/msg.pyorch.pyreproduce/lab-k-verify-isolated-s1-20261105/harness/orch.pyv1prime.pyreproduce/lab-k-verify-isolated-s1-20261105/harness/v1prime.pyreport.pyreproduce/lab-k-verify-isolated-s1-20261105/report.pyv1prime.pyreproduce/lab-k-verify-isolated-s1-20261105/v1prime.pycascade.pyreproduce/lab-loops-collide-s0-20261105/extract/cascade.pyd0.pyreproduce/lab-loops-collide-s0-20261105/extract/d0.pyextract2.pyreproduce/lab-loops-collide-s0-20261105/extract/extract2.pymake_data.pyreproduce/lab-loops-collide-s0-20261105/extract/make_data.pyparse.pyreproduce/lab-loops-collide-s0-20261105/extract/parse.pyreport.pyreproduce/lab-loops-collide-s0-20261105/report.pyreport.pyreproduce/lab-prelanding-kogen-check-20261105/report.pybuild_sample.pyreproduce/lab-prelanding-kogen-check-20261105/scripts/build_sample.pybuild_tuples.pyreproduce/lab-prelanding-kogen-check-20261105/scripts/build_tuples.pygen_results.pyreproduce/lab-prelanding-kogen-check-20261105/scripts/gen_results.pyreport.pyreproduce/lab-probe-env-shaper-vs-readiness-20261107/report.pycheck.pyreproduce/lab-rc-b-review-listed-requirements-20261106/check.pyreport.pyreproduce/lab-rc-c-decision-gate-before-readiness-20261106/report.pyanalyze.pyreproduce/lab-rc-c2-scope-bound-before-readiness-20261106/analyze.pyreport.pyreproduce/lab-rc-c2-scope-bound-before-readiness-20261106/report.pyprobe_report.pyreproduce/lab-rc-l1-walltime-20261105/probe_report.pyreport.pyreproduce/lab-rc-l1-walltime-20261105/report.pyreport.pyreproduce/lab-review-invalid-quote-and-unread/report.pyreport.pyreproduce/lab-shape-handoff-question-s0-20261105/report.pyreport.pyreproduce/lab-shaping-aor-diagnosis-20261107/report.pyreport.pyreproduce/lab-shaping-assumption-guidance-screen-20261108/report.pyreport.pyreproduce/lab-shaping-baseline-noise1-20261108/report.pyreport.pyreproduce/lab-shaping-baseline-noise2-20261108/report.pyreport.pyreproduce/lab-shaping-bytecode-screen-20261107/report.pyreport.pyreproduce/lab-shaping-carried-decision-screen-20261108/report.pyreport.pyreproduce/lab-shaping-climb-round0-20261107/report.pyreport.pyreproduce/lab-shaping-combined-guidance-screen-20261108/report.pyreport.pyreproduce/lab-shaping-converge-20261107/report.pyreport.pyreproduce/lab-shaping-decision-bisect-20261108/report.pyreport.pyreproduce/lab-shaping-dual-path-guidance-screen-20261108/report.pyreport.pyreproduce/lab-shaping-evidence-path-screen-20261107/report.pyreport.pyreproduce/lab-shaping-evidence-probe-screen-20261108/report.pyreport.pyreproduce/lab-shaping-historical-current-20261108/report.pyreport.pyreproduce/lab-shaping-initial-evidence-screen-20261107/report.pyreport.pyreproduce/lab-shaping-observed-evidence-screen-20261107/report.pyreport.pyreproduce/lab-shaping-small-requests-20261107/report.pyreport.pyreproduce/lab-slow-checks-rerun-s0-20261105/report.pyl2_project.pyreproduce/lab-tests-l2-projection-20261105/extract/l2_project.pyreport.pyreproduce/lab-tests-l2-projection-20261105/report.pyextract_inputs.pyreproduce/lab-tests-l4-critical-path-20261105/extract/extract_inputs.pyheavy_suites.pyreproduce/lab-tests-l4-critical-path-20261105/extract/heavy_suites.pyl4_model.pyreproduce/lab-tests-l4-critical-path-20261105/l4_model.pyreport.pyreproduce/lab-tests-l4-critical-path-20261105/report.pyprofile.pyreproduce/lab-tests-l4-per-test-profile-20261107/scripts/profile.pyreport.pyreproduce/lab-tests-l5-quint-replay-20261105/report.pyl5_static.pyreproduce/lab-tests-l5-quint-replay-20261105/scripts/l5_static.pyreport.pyreproduce/lab-tests-l8-step-files-20261105/report.pyl8_inventory.pyreproduce/lab-tests-l8-step-files-20261105/scripts/l8_inventory.pyl8_section6.pyreproduce/lab-tests-l8-step-files-20261105/scripts/l8_section6.pydrive-stage1.pyreproduce/lab-xprotect-launch-path-20261106/drive-stage1.pydrive-stage2.pyreproduce/lab-xprotect-launch-path-20261106/drive-stage2.pylane-codex-info.pyreproduce/lane-codex-info.pylane-plan.pyreproduce/lane-plan.pylane-receipt.pyreproduce/lane-receipt.pylane-sample.pyreproduce/lane-sample.pylane-snapshot.pyreproduce/lane-snapshot.pylane_watchdog.pyreproduce/lane_watchdog.pyanalyze_bcw.pyreproduce/loop-bounded-shell-wait-166ec7f-eu2/analyze_bcw.pydurations.pyreproduce/loop-bounded-shell-wait-166ec7f-eu2/durations.pylever_fires.pyreproduce/loop-bounded-shell-wait-166ec7f-eu2/lever_fires.pyregrade.pyreproduce/loop-bounded-shell-wait-166ec7f-eu2/regrade.pysummarize.pyreproduce/loop-bounded-shell-wait-166ec7f-eu2/summarize.pyanalyze_loopnp.pyreproduce/loop-fixes-9e90bf5-vs-pin-1a3eb18-eu2/analyze_loopnp.pybuild_runs.pyreproduce/loop-fixes-9e90bf5-vs-pin-1a3eb18-eu2/build_runs.pyregrade.pyreproduce/loop-fixes-9e90bf5-vs-pin-1a3eb18-eu2/regrade.pystage_ends.pyreproduce/loop-fixes-9e90bf5-vs-pin-1a3eb18-eu2/stage_ends.pysummarize.pyreproduce/loop-fixes-9e90bf5-vs-pin-1a3eb18-eu2/summarize.pyanalyze.pyreproduce/loop-high-reasoning-api-replay13/analyze.pyanalyze.pyreproduce/loop-high-reasoning-api/analyze.pyanalyze_loopnp.pyreproduce/loop-pin-1a3eb18-failed-row-cost-eu2/analyze_loopnp.pybuild_runs.pyreproduce/loop-pin-1a3eb18-failed-row-cost-eu2/build_runs.pysummarize.pyreproduce/loop-pin-1a3eb18-failed-row-cost-eu2/summarize.pylrepin-count.pyreproduce/loop-repin-9e90bf5-in-kogen-eu2/lrepin-count.pylrepin-pairs.pyreproduce/loop-repin-9e90bf5-in-kogen-eu2/lrepin-pairs.pyreport.pyreproduce/loop-repin-9e90bf5-in-kogen-eu2/report.pyanalyze.pyreproduce/loop-shell-tail/analyze.pyanalyze.pyreproduce/loop-shell-tail/extract/analyze.pyreport.pyreproduce/loop-shell-tail/extract/report.pyrun_fixtures.pyreproduce/loop-stream-shell-fault-fixtures/run_fixtures.pysummarize.pyreproduce/loop-stream-shell-fault-fixtures/summarize.pyanalyze.pyreproduce/loop-sub-route-honest-probe/analyze.pysubroute-probe.pyreproduce/loop-sub-route-honest-probe/probes/subroute-probe.pyto-csv.pyreproduce/loop-sub-route-honest-probe/probes/to-csv.pyanalyze.pyreproduce/loop-sub-route-throughput/analyze.pydirect-sse-probe.pyreproduce/loop-sub-route-throughput/probes/direct-sse-probe.pysubroute-probe.pyreproduce/loop-sub-route-throughput/probes/subroute-probe.pyanalyze.pyreproduce/loop-sub-route-throughput/superseding-same-host-test/analyze.pyprobe.pyreproduce/loop-sub-route-throughput/superseding-same-host-test/probe.pyanalyze_wmap.pyreproduce/loop-workspace-snapshot-166ec7f-eu/analyze_wmap.pysummarize.pyreproduce/loop-workspace-snapshot-166ec7f-eu/summarize.pyanalyze_wmap.pyreproduce/loop-workspace-snapshot-replication-3hosts/analyze_wmap.pypool_wmap.pyreproduce/loop-workspace-snapshot-replication-3hosts/pool_wmap.pyanalyze_night.pyreproduce/loopnight-instruction-levers/analyze_night.pybuild_pack.pyreproduce/loopnight-instruction-levers/build_pack.pymine_probe.pyreproduce/mine_probe.pymine_studio.pyreproduce/mine_studio.pymissing_reasons.pyreproduce/missing_reasons.pyanalyze.pyreproduce/morning-report-20261030/analyze.pyanalyze.pyreproduce/npm-preflight-cache/analyze.pyapproval-deferral-fault.pyreproduce/npm-preflight-cache/faulttests/approval-deferral-fault.pycompleted-resume.pyreproduce/npm-preflight-cache/faulttests/completed-resume.pygit-add-stall-preflight.pyreproduce/npm-preflight-cache/faulttests/git-add-stall-preflight.pynpm-stall-preflight.pyreproduce/npm-preflight-cache/faulttests/npm-stall-preflight.pyanalyze.pyreproduce/overlap3-ceiling-eu2/analyze.pyreport.pyreproduce/p2-unstated-aor-screen/report.pyreport.pyreproduce/p2-unstated-go-pooled/report.pyreport.pyreproduce/p2-unstated-go-screen/report.pyreport.pyreproduce/p3-review-probes-aor/report.pyreport.pyreproduce/p4-ledger-aor-screen/report.pyreport.pyreproduce/p4-ledger-go-pooled/report.pyreport.pyreproduce/p4-ledger-go-screen/report.pyreport.pyreproduce/p5-loss-attribution/report.pyreport.pyreproduce/p5-stop-at-green-go-pooled-eu2/report.pyreport.pyreproduce/p5-stop-at-green-go-screen/report.pypartitioned_jsonl.pyreproduce/partitioned_jsonl.pypass_counts.pyreproduce/pass_counts.pyprivacy.pyreproduce/privacy.pyprove_apparatus.pyreproduce/prove_apparatus.pypublished_hidden.pyreproduce/published_hidden.pyr70_completeness.pyreproduce/r70_completeness.pyrecompute_mined.pyreproduce/recompute_mined.pyreconcile_l0.pyreproduce/reconcile_l0.pyrecord_contract.pyreproduce/record_contract.pyanalyze.pyreproduce/replayconf-14task-high/analyze.pyrun-replayconf.pyreproduce/replayconf-14task-high/probes/run-replayconf.pyreport.pyreproduce/review-climb-saved-candidates-20261103/report.pyround_cache.pyreproduce/round_cache.pyanalyze.pyreproduce/route-same-host-throughput/analyze.pyprobe.pyreproduce/route-same-host-throughput/probe.pyrun_records.pyreproduce/run_records.pyreport.pyreproduce/s3-audit-on-go-pooled/report.pyreport.pyreproduce/s3-audit-on-go-screen/report.pysafe_rows.pyreproduce/safe_rows.pysafe_rows_path.pyreproduce/safe_rows_path.pyscore_l6.pyreproduce/score_l6.pyscrutinize.pyreproduce/scrutinize.pyreport.pyreproduce/shape-example-check-3ea2e23-screen/report.pyclassify_private_logs.pyreproduce/shape-luna-medium-refusals/classify_private_logs.pyreport.pyreproduce/shape-luna-medium-refusals/report.pyreport.pyreproduce/shape-protected-setup-evidence-screen/report.pyreport.pyreproduce/shape-retry-levers-k11/report.pyreport.pyreproduce/shape-screen-s5/report.pyreport.pyreproduce/shape-screen-terra-b/report.pyanalyze.pyreproduce/shell-yield-eu2/analyze.pyanalyze-round.pyreproduce/shell-yield-eu2/probes/analyze-round.pyprepare-shellwait.pyreproduce/shell-yield-eu2/probes/prepare-shellwait.pyrun-shellwait.pyreproduce/shell-yield-eu2/probes/run-shellwait.pystandard_record.pyreproduce/standard_record.pyreport.pyreproduce/test-authoring-go-screen/report.pytest_audit_cell.pyreproduce/test_audit_cell.pytest_candidate_revision.pyreproduce/test_candidate_revision.pytest_compact_record_round_trip.pyreproduce/test_compact_record_round_trip.pytest_context_reconstruction.pyreproduce/test_context_reconstruction.pytest_control_worker_reference.pyreproduce/test_control_worker_reference.pytest_cost_calc.pyreproduce/test_cost_calc.pytest_doctor_versions.pyreproduce/test_doctor_versions.pytest_egress_proxy.pyreproduce/test_egress_proxy.pytest_evidence_partitions.pyreproduce/test_evidence_partitions.pytest_host_layout.pyreproduce/test_host_layout.pytest_l3b_per_test_rule.pyreproduce/test_l3b_per_test_rule.pytest_lane_plan.pyreproduce/test_lane_plan.pytest_lane_watchdog.pyreproduce/test_lane_watchdog.pytest_mine_probe.pyreproduce/test_mine_probe.pytest_published_hidden.pyreproduce/test_published_hidden.pytest_recompute_mined.pyreproduce/test_recompute_mined.pytest_record_contract.pyreproduce/test_record_contract.pytest_run_lane.pyreproduce/test_run_lane.pytest_run_records.pyreproduce/test_run_records.pytest_safe_rows_path.pyreproduce/test_safe_rows_path.pytest_scrutinize.pyreproduce/test_scrutinize.pytest_split_run_records.pyreproduce/test_split_run_records.pytest_standard_record.pyreproduce/test_standard_record.pytest_validate_release.pyreproduce/test_validate_release.pytest_validation_policy.pyreproduce/test_validation_policy.pyvalidate_release.pyreproduce/validate_release.pyvalidate_repo.pyreproduce/validate_repo.pyvalidate_round.pyreproduce/validate_round.pyvalidate_spec_snapshot.pyreproduce/validate_spec_snapshot.pyvalidation_policy.pyreproduce/validation_policy.pyvendor_spec.pyreproduce/vendor_spec.pyanalyze.pyreproduce/verbosity-low-eu6/analyze.pyrun-eu6.pyreproduce/verbosity-low-eu6/probes/run-eu6.pyreport.pyreproduce/verification-diagnostic-localization-replay-20261103/report.pyanalyze_executions.pyreproduce/verification-reuse-replay/analyze_executions.pyanalyze_runs.pyreproduce/verification-reuse-replay/analyze_runs.py

Per-round reproduction inventory

One entry for every round in rounds/index.json. Full execution replay is unavailable for all entries in this snapshot.

RoundScriptInputsReplay typeFull execution replay
cache-key-api-v4None publishedProbe script and raw request receipts are not published; sanitized per-pair counters are in the round sanitized.csv and the registered rule is recomputable from them by hand.None identifiedunavailableUnavailable
cache-key-backend-v4None publishedProbe script and raw request receipts are not published; sanitized per-pair counters are in the round sanitized.csv and the registered rule is recomputable from them by hand.None identifiedunavailableUnavailable
cache-replay-mechanismreproduce/cache_replay_tables.py
python3 reproduce/cache_replay_tables.pyRecomputes every published cache-counter table from the sanitized attempt CSVs. Provider execution cannot be replayed from this public bundle.
analysis-onlyUnavailable
campfire-lane1None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
campfire-lane2None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
candidate-counterfactualNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
claim-b-rerun-1None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
claim-b-screen-1None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
confirm-combined-1None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
core4-pilotsNone publishedThe published source table and Standard records preserve available cell outcomes; raw execution receipts are incomplete and full model execution cannot be replayed.analysis-onlyUnavailable
core4b-ksub-maxNone publishedThe published source table and Standard records preserve available cell outcomes; raw execution receipts are incomplete and full model execution cannot be replayed.analysis-onlyUnavailable
ctx-storeNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
e06-contextNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
e09-tidewaveNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
frontierNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
fv-pilot-2026-10-10rounds/fv-pilot-2026-10-10/reproduce.py
python3 rounds/fv-pilot-2026-10-10/reproduce.pyRecomputes per-task and arm results from the sanitized lifecycle ledger; model calls and independent final evaluation are not replayable.
analysis-onlyUnavailable
gleamNone publishedThe public per-cell summaries and Standard-shaped records preserve the reported outcomes; original execution and complete phase receipts are not available for full replay.unavailableUnavailable
grok-frontierNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
hard1-calibrate-1None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
harness-compare-1None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
harness-pick-2026-09-29None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
hc-1None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
hc-2None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
hc-3None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
hc-4None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
hc01-small32None publishedOwner stopped the round at 25/32 graded; source receipts and remaining planned cohort do not support full execution replay.unavailableUnavailable
intent-coverage-auditNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
jev-route-ctxNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
jev-triage-ctxNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
kh-climbNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
kh-compare-2026-09-29None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
kogen-rs-ladder-luna-r74xNone publishedPer-cell outcomes, execution/hash assertions, incidents and identity audit are source-reported and not independently reproducible from the public snapshot; named lane records are absent. R74 rows are recovered internal historical context, not joined to the canonical official export. Strict release compliance is not established. Model execution and official grading are not replayed.unavailableUnavailable
l0-reconcilereproduce/reconcile_l0.py
python3 reproduce/reconcile_l0.pyRecomputes selected reconciliation CSV summaries only; it does not replay historical model executions or complete the canonical official outcome join.
analysis-onlyUnavailable
l1-task-grader-trustNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
l2-plan-specificityreproduce/l2-plan-specificity.py
python3 reproduce/l2-plan-specificity.pyRecomputes all published numeric results from the sanitized official-cell, planner-receipt, and historical-baseline CSVs. Sandbox execution and official grading cannot be replayed from this public bundle.
analysis-onlyUnavailable
l3-repairrounds/l3-repair/reproduce/l3_repair.py
python3 rounds/l3-repair/reproduce/l3_repair.pyRecomputes aggregate repair counts from six published repair rows; per-test pass sets, grade receipts, and historical model executions are unavailable.
analysis-onlyUnavailable
l3b-repair-vs-continuerounds/l3b-repair-vs-continue/reproduce/l3b_repair_vs_continue.py
python3 rounds/l3b-repair-vs-continue/reproduce/l3b_repair_vs_continue.pyRecomputes the public 24-cell aggregate outcome table, rescue counts, usage totals, and registered rule result, including per-test no-regression from the published pass/fail sets. Model execution and official grading are not replayed.
analysis-onlyUnavailable
l4-context-packetrounds/l4-context-packet/reproduce/l4-context-packet.py
python3 rounds/l4-context-packet/reproduce/l4-context-packet.pyThe public aggregate CSVs support analysis replay only; benchmark execution and official grading are not replayed.
analysis-onlyUnavailable
l4b-confirmrounds/l4b-confirm/reproduce/reproduce_us.py
python3 rounds/l4b-confirm/reproduce/reproduce_us.pyRecomputes official outcomes, paired seed results, test counts, token totals, and published tables from sanitized cell metrics. Model execution and official grading are not replayed.
analysis-onlyUnavailable
l5-sol-comparatorsrounds/l5-sol-comparators/reproduce/l5_sol_comparators.py
python3 rounds/l5-sol-comparators/reproduce/l5_sol_comparators.pyThe public cell CSVs support analysis replay only; model execution and official grading are not replayed.
analysis-onlyUnavailable
l6-auditor-replayreproduce/score_l6.py
python3 reproduce/score_l6.py --cases rounds/l6-auditor-replay/data/case_list.csv --calls rounds/l6-auditor-replay/data/calls.csvRecomputes the offline auditor metrics from the published cases and calls; it does not rerun auditor requests or recreate withheld task prompts and patches.
analysis-onlyUnavailable
lang-sol-replicationrounds/lang-sol-replication/reproduce/lang_sol_replication.py
python3 rounds/lang-sol-replication/reproduce/lang_sol_replication.pyRecomputes the round tables and post-hoc equal-task sensitivity from the 58 published cell rows; it does not replay model execution, official grading, host smoke gates, or the missing exception receipt.
analysis-onlyUnavailable
llm-latency-longrounds/llm-latency-long/reproduce.py
python3 rounds/llm-latency-long/reproduce.pyRecomputes long-probe request summaries from text-free event logs; it does not repeat model calls.
  • rounds/llm-latency-long/raw/round2-eu-*.jsonl
analysis-onlyUnavailable
llm-latency-shortrounds/llm-latency-short/reproduce.py
python3 rounds/llm-latency-short/reproduce.pyRecomputes request counts and quantiles from text-free event logs; it does not repeat model calls.
  • rounds/llm-latency-short/raw/*.jsonl
analysis-onlyUnavailable
luna-mediumNone publishedThe public per-cell summaries and Standard-shaped records preserve the reported outcomes; original execution and complete phase receipts are not available for full replay.unavailableUnavailable
night-2026-10-01None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
offline-aa-bo3None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
offline-contract-auditNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
offline-reviewNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
offline-review3None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
overstrict-replayNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
pilot73reproduce/recompute_mined.py
python3 reproduce/recompute_mined.pyRegenerates observed source-row summaries and token comparisons for the recovered pilot tag. It does not establish the registered r73 scored phase, a planned denominator, or historical model execution.
analysis-onlyUnavailable
r1None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r10None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r11None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r12None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r13None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r14None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r15None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r16None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r17None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r18None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r19None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r2None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r20None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r21None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r22None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r23None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r24None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r25None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r26None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r27None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r28None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r29None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r3None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r30None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r31None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r32None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r32aNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r32bNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r33None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r34None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r34bNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r35None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r36None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r37None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r37cNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r38None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r39None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r39bNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r4None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r40None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r41None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r42None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r43None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r44None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r45None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r46None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r47None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r48None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r48bNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r49None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r49bNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r5None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r50None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r51None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r52None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r53None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r53bNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r54None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r55None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r56None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r56bNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r56cNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r56dNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r56p2None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r57reproduce/reconcile_l0.py
python3 reproduce/reconcile_l0.pyRecomputes selected reconciliation CSV summaries only; it does not replay historical model executions or complete the canonical official outcome join.
analysis-onlyUnavailable
r57-studioNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r57-t90-studioNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r57bNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r57cNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r57dNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r57eNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r58None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r58bNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r58x-studioNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r59None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r6None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r60None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r60bNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r61None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r62None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r62bNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r63None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r63bNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r64None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r64bNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r64cNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r64dNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r64eNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r65None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r65bNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r66None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r67None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r67bNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r68None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r68bNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r69reproduce/reconcile_l0.py
python3 reproduce/reconcile_l0.pyRecomputes selected reconciliation CSV summaries only; it does not replay historical model executions or complete the canonical official outcome join.
analysis-onlyUnavailable
r7None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r70None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r70-compileNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r70-rverounds/r70-rve/reproduce.py
python3 rounds/r70-rve/reproduce.pyRecomputes the published R70 original RvE summary from its public ledgers; it does not launch cells.
analysis-onlyUnavailable
r70-rve-extNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r70-rve-rerunNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r70-rve-task8v2rounds/r70-rve-task8v2/reproduce.py
python3 rounds/r70-rve-task8v2/reproduce.pyExtracts the two v2 smoke rows and controls; it does not launch scored cells.
analysis-onlyUnavailable
r70-rve2None publishedOnly filtered grade and manifest fields, selected launcher lines, and corrected audit counts are available; no complete delivery bundle or execution replay is published.None identifiedunavailableUnavailable
r70-rve3None publishedOnly filtered grade and manifest fields, selected launcher lines, and corrected audit counts are available; no complete delivery bundle or execution replay is published.None identifiedunavailableUnavailable
r70-spot1None publishedOnly filtered grade and manifest fields, selected launcher lines, and corrected audit counts are available; no complete delivery bundle or execution replay is published.None identifiedunavailableUnavailable
r70-task8None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r71None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r72reproduce/reconcile_l0.py
python3 reproduce/reconcile_l0.pyRecomputes selected reconciliation CSV summaries only; it does not replay historical model executions or complete the canonical official outcome join.
analysis-onlyUnavailable
r73None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r74reproduce/reconcile_l0.py
python3 reproduce/reconcile_l0.pyRecomputes selected reconciliation CSV summaries only; it does not replay historical model executions or complete the canonical official outcome join.
analysis-onlyUnavailable
r8None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
r9None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
race1None publishedPublished race scores and per-case outputs are available for inspection and summary checks. Original model execution is not replayed; arm code was not published.analysis-onlyUnavailable
race2None publishedPublished race scores and per-case outputs are available for inspection and summary checks. Original model execution is not replayed; arm code was not published.analysis-onlyUnavailable
race3-sprintNone publishedThe sanitized public race export preserves supplied result tables and per-case projections for inspection; original model execution cannot be replayed because implementation source and full arm streams remain private.analysis-onlyUnavailable
race4None publishedPer-cell aggregate results are available for review; the 88-case suite and per-case results remain withheld until after Race 6, and arm code remains private.analysis-onlyUnavailable
race5rounds/race5/results/recompute.py
rounds/race5/results/recompute.pyPRIMARY medians, spreads, and published timing arithmetic can be recomputed from the aggregate per-cell records; suite-v3 cases, case IDs, per-case results, and arm code remain withheld/private.
analysis-onlyUnavailable
race6rounds/race6/results/recompute.py
rounds/race6/results/recompute.pyPer-cell scores, medians and kill-point token summaries are available for review; the suite-v3 cases, case inventories and per-case results remain withheld while L1–L3 use the same cases; full execution replay is unavailable.
analysis-onlyUnavailable
rails-catalogueNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
rtg1None publishedThe published source table and Standard records preserve available cell outcomes; raw execution receipts are incomplete and full model execution cannot be replayed.analysis-onlyUnavailable
rz1rounds/rz1/reproduce_decision.py
python3 rounds/rz1/reproduce_decision.pyRecomputes the preregistered interim and final decision from the sanitized plan receipts and 42 mined records; model execution and hidden grading are not replayed.
analysis-onlyUnavailable
shape1-wave1None publishedThe public per-cell summaries and Standard-shaped records preserve the reported outcomes; original execution and complete phase receipts are not available for full replay.unavailableUnavailable
sol-mediumNone publishedThe published source table and Standard records preserve available cell outcomes; raw execution receipts are incomplete and full model execution cannot be replayed.analysis-onlyUnavailable
stack-oneshot-conformanceNone publishedOriginal implementation worktrees, conformance result bundles, and host environments are not included; the round cites the retained source reports and result paths but cannot be replayed from this repository.None identifiedunavailableUnavailable
studio-kogenNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
studio-kogen-80a4None publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
studio-shaper-pilotNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
t100-cache-laneNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
t98-cache-smokeNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
unmappedNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
x-compactionNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
x-contextNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
x-continuationNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
x-ladderNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
x-minikogenNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
x-miningNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
x-modelsNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
x-parallelNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
x-raceNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
x-recoveryNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
x-reviewNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
x-rolesNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
x-surfaceNone publishedNo public per-round analysis script with a complete, identified input bundle is published. See the round README, MEASURED.md, and MISSING.md for recorded scope and gaps; historical execution inputs/toolchains/grading are not bundled.None identifiedunavailableUnavailable
l1-model-effortNone publishedReported aggregate FINAL/T+30 scores and token totals are preserved for eight planned language-condition slots; suite-v3 cases, per-case results, fixtures and execution artifacts are withheld.analysis-onlyUnavailable
arch-elixir-vs-goNone publishedPublished aggregate observations and Standard records are available; original execution evidence and hidden test artifacts are not included.analysis-onlyUnavailable
lab01-analysis-tooling-screenNone publishedPublished aggregate observations and Standard records are available; original execution evidence and protected scoring artifacts are not included.analysis-onlyUnavailable
containment-small-euNone publishedPublished per-repetition observations and Standard records are available; original execution evidence is not included.analysis-onlyUnavailable
containment-small-eu-v2None publishedPublished per-repetition observations, source hashes and a privacy-edited compact evidence bundle are available; the full execution archive and measurement transcript are not included.analysis-onlyUnavailable
containment-small-eu-helperNone publishedPublished per-repetition observations, measured summaries, compact public evidence and the privacy-edited harness are available; the full execution archive and transcript are not included.analysis-onlyUnavailable
review-convergence-1None publishedPublished Standard records, the Lab public export (frozen configuration, per-attempt outcomes, public test states, stage times and usage) and the Lab report are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
screen-kogen-1-stage-dNone publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
screen-kogen-2None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
screen-kogen-1-stage-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
screen-kogen-2-first-runNone publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
baseline-variance-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
review-timing-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
shaping-model-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
builder-effort-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
confirm-builder-effort-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
repair-stop-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
review-bound-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
review-bound-2None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
retention-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
review-independence-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
repair-cost-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
planner-direct-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
claude-roles-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
fixer-model-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
planner-direct-2None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
claude-builder-2None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
builder-subagents-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
review-fix-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
shape-default-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
blocker-stop-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
claude-review-fix-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
builder-subagents-2None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
repeat-screen-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
review-fix-2None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
shaping-2None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
prose-vs-gherkin-1None publishedPublic bundles and records preserve observed Lab outcomes and configuration; model execution cannot be replayed from this repository snapshot.analysis-onlyUnavailable
mechanism-split-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
review-named-defect-1None publishedPublic bundles and records preserve observed Lab outcomes and configuration; model execution cannot be replayed from this repository snapshot.analysis-onlyUnavailable
builder-low-1None publishedPublished Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.analysis-onlyUnavailable
reviewer-effort-weak-1None publishedPublic bundles and records preserve observed Lab outcomes and configuration; model execution cannot be replayed from this repository snapshot.analysis-onlyUnavailable
cheap-profile-1None publishedPublished bundles and records preserve observed Lab outcomes and configuration; model execution cannot be replayed from this repository snapshot.analysis-onlyUnavailable
confirm-cheap-profile-1None publishedPublished bundles and records preserve observed Lab outcomes and configuration; model execution cannot be replayed from this repository snapshot.analysis-onlyUnavailable
reviewer-polish-1None publishedPublished bundles and records preserve observed Lab outcomes and configuration; model execution cannot be replayed from this repository snapshot.analysis-onlyUnavailable
confirm-reviewer-medium-1None publishedPublished bundles and records preserve observed Lab outcomes and configuration; model execution cannot be replayed from this repository snapshot.analysis-onlyUnavailable
confirm-reviewer-medium-2None publishedPublished bundles and records preserve observed Lab outcomes and configuration; model execution cannot be replayed from this repository snapshot.analysis-onlyUnavailable
review-off-hard-1None publishedPublic bundles and records preserve observed Lab outcomes and configuration; model execution cannot be replayed from this repository snapshot.analysis-onlyUnavailable
intent-short-1None publishedPublic bundles and records preserve observed Lab outcomes and configuration; model execution cannot be replayed from this repository snapshot.analysis-onlyUnavailable
confirm-intent-short-1None publishedPublic bundles and records preserve observed Lab outcomes and configuration; model execution cannot be replayed from this repository snapshot.analysis-onlyUnavailable
review-off-hard-1-kogenNone publishedRecords and bundles preserve the graded outcomes; model execution is rerun with the reader's own login (lanes/reproduce.sh --rerun).analysis-onlyUnavailable
chatgpt-session-cache-1None publishedRecords and bundles preserve the graded outcomes; model execution is rerun with the reader's own login (lanes/reproduce.sh --rerun).analysis-onlyUnavailable
review-value-2rounds/review-value-2/analyze.py
python3 rounds/review-value-2/analyze.pyRecomputes the published tables from the records and per-cell bundles of the five review-value-2 rounds, with no model call. The task folders and cell trees are not published yet (the tasks are still in use), so cells cannot be re-graded; model execution cannot be replayed from this bundle.
analysis-onlyUnavailable
review-value-2-cleanrounds/review-value-2/analyze.py
python3 rounds/review-value-2/analyze.pyRecomputes the published tables from the records and per-cell bundles of the five review-value-2 rounds, with no model call. The task folders and cell trees are not published yet (the tasks are still in use), so cells cannot be re-graded; model execution cannot be replayed from this bundle.
analysis-onlyUnavailable
review-value-2-natural-offrounds/review-value-2/analyze.py
python3 rounds/review-value-2/analyze.pyRecomputes the published tables from the records and per-cell bundles of the five review-value-2 rounds, with no model call. The task folders and cell trees are not published yet (the tasks are still in use), so cells cannot be re-graded; model execution cannot be replayed from this bundle.
analysis-onlyUnavailable
review-value-2-natural-reviewedrounds/review-value-2/analyze.py
python3 rounds/review-value-2/analyze.pyRecomputes the published tables from the records and per-cell bundles of the five review-value-2 rounds, with no model call. The task folders and cell trees are not published yet (the tasks are still in use), so cells cannot be re-graded; model execution cannot be replayed from this bundle.
analysis-onlyUnavailable
review-value-2-openairounds/review-value-2/analyze.py
python3 rounds/review-value-2/analyze.pyRecomputes the published tables from the records and per-cell bundles of the five review-value-2 rounds, with no model call. The task folders and cell trees are not published yet (the tasks are still in use), so cells cannot be re-graded; model execution cannot be replayed from this bundle.
analysis-onlyUnavailable
long-tasks-reset-fix-1-beforeNone publishedRecords and bundles preserve the graded outcomes; model execution is rerun with the reader's own login (lanes/reproduce.sh --rerun).analysis-onlyUnavailable
long-tasks-reset-fix-1-afterNone publishedRecords and bundles preserve the graded outcomes; model execution is rerun with the reader's own login (lanes/reproduce.sh --rerun).analysis-onlyUnavailable
long-tasks-reset-fix-1-after-studioNone publishedRecords and bundles preserve the graded outcomes; model execution is rerun with the reader's own login (lanes/reproduce.sh --rerun).analysis-onlyUnavailable
reviewer-models-seeded-1rounds/reviewer-models-seeded-1/analyze.py
python3 rounds/reviewer-models-seeded-1/analyze.pyRecomputes the published tables from the records and per-cell bundles of this round, the clean-control round and the earlier review-value-2 and review-value-2-openai rounds, with no model call. The task folders and cell trees are not published yet (the tasks are still in use), so cells cannot be re-graded; model execution cannot be replayed from this bundle.
analysis-onlyUnavailable
reviewer-models-clean-1rounds/reviewer-models-seeded-1/analyze.py
python3 rounds/reviewer-models-seeded-1/analyze.pyRecomputes the published tables from the records and per-cell bundles of this round and the seeded round, with no model call. The task folders and cell trees are not published yet (the tasks are still in use), so cells cannot be re-graded; model execution cannot be replayed from this bundle.
analysis-onlyUnavailable
shaper-models-1rounds/shaper-models-1/analyze.py
python3 rounds/shaper-models-1/analyze.pyRecomputes the published tables from the per-cell bundle of this round, with no model call. The task folders and cell trees are not published yet (the tasks are still in use), so cells cannot be re-graded; model execution cannot be replayed from this bundle.
analysis-onlyUnavailable
terra-builder-1rounds/terra-builder-1/analyze.py
python3 rounds/terra-builder-1/analyze.pyRecomputes the published tables from the per-cell bundles and fair-tests.json of this round and terra-builder-effort-1, with no model call. The task folders and cell trees are not published yet (the tasks are still in use), so cells cannot be re-graded; model execution cannot be replayed from this bundle.
analysis-onlyUnavailable
terra-builder-effort-1rounds/terra-builder-1/analyze.py
python3 rounds/terra-builder-1/analyze.pyRecomputes the published tables from the per-cell bundles and fair-tests.json of this round and terra-builder-1, with no model call. The task folders and cell trees are not published yet (the tasks are still in use), so cells cannot be re-graded; model execution cannot be replayed from this bundle.
analysis-onlyUnavailable
loop-vs-executor-1rounds/loop-vs-executor-1/analyze.py
python3 rounds/loop-vs-executor-1/analyze.pyRecomputes the published tables from the per-cell bundles and executor-usage.json of this round, with no model call. The task folders and cell trees are not published yet (the tasks are still in use), so cells cannot be re-graded; model execution cannot be replayed from this bundle.
analysis-onlyUnavailable
executor-lineup-dry-1rounds/executor-lineup-dry-1/analyze.py
python3 rounds/executor-lineup-dry-1/analyze.pyRecomputes the published tables from the per-cell bundles, plumbing.json and executor-usage.json of this round, with no model call. The task folders and cell trees are not published yet (the tasks are still in use), so cells cannot be re-graded; model execution cannot be replayed from this bundle.
analysis-onlyUnavailable
sonnet-long-builder-1rounds/sonnet-long-builder-1/analyze.py
python3 rounds/sonnet-long-builder-1/analyze.pyRecomputes the published tables from the per-cell bundles of this round, with no model call. The task folders and cell trees are not published yet (the tasks are still in use), so cells cannot be re-graded; model execution cannot be replayed from this bundle.
analysis-onlyUnavailable
terra-vs-sonnet-builder-1rounds/terra-vs-sonnet-builder-1/analyze.py
python3 rounds/terra-vs-sonnet-builder-1/analyze.pyRecomputes the published tables from the per-cell bundles of this round, void-cells.json and comparator.json (one row per partner cell of loop-vs-executor-1, rebuildable from that round's files), with no model call. The task folders and cell trees are not published yet (the tasks are still in use), so cells cannot be re-graded; model execution cannot be replayed from this bundle.
analysis-onlyUnavailable
shape-flawed-requests-1rounds/shape-flawed-requests-1/analyze.py
python3 rounds/shape-flawed-requests-1/analyze.pyRecomputes the published tables from the per-cell bundles, grades.json (label counts per cell from the blind grader), grader-agreement.json and shape-usage.json of this round, with no model call. The task folders and cell trees are not published yet (the tasks are still in use), so cells cannot be re-graded; model execution cannot be replayed from this bundle.
analysis-onlyUnavailable
claude-code-fix-rechecks-1rounds/claude-code-fix-rechecks-1/analyze.py
python3 rounds/claude-code-fix-rechecks-1/analyze.pyRecomputes the published tables from the per-cell bundles and cell-facts.json of this round, with no model call. The task folder and cell trees are not published, so cells cannot be re-graded; model execution cannot be replayed from this bundle.
analysis-onlyUnavailable
terra-reviewer-seeded-clean-1rounds/terra-reviewer-seeded-clean-1/analyze.py
python3 rounds/terra-reviewer-seeded-clean-1/analyze.pyRecomputes the published tables from the per-cell bundles of this round, with no model call. The task folders and cell trees are not published yet (the tasks are still in use), so cells cannot be re-graded; model execution cannot be replayed from this bundle.
analysis-onlyUnavailable
h23-release-bbf56fdc-vs-891c91b3None publishedRetained numeric records only; the full model and grader rerun is unavailable.unavailableUnavailable