> Data revision: baf7c2f4307d3a989a8fb9363df8000fda3d9310
> Source files:
- [reproduce/VERIFY.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/VERIFY.md)

# Verify published records and numbers

The public command sequence below runs from the repository root with Python 3 and uses only committed files and the standard library. It does not contact a model, grading service, benchmark host, or private account.

For an Ubuntu 24.04 x86_64 worker, [`setup-host.sh`](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/setup-host.sh) installs the pinned host kit, [`doctor.sh`](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/doctor.sh) performs model-free host qualification, and [`run-controls.sh`](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/run-controls.sh) checks that each chosen task's published reference passes and its no-op base fails after manual `bench` device login. `setup-host.sh --dry-run` prints its planned actions. Fresh-machine verification of all three scripts is pending the first host rebuild; this is separate from the read-only publication checks below.

```sh
python3 reproduce/export_results.py
python3 reproduce/build_records.py
python3 reproduce/build_grade_join.py
python3 reproduce/build_l3b_records.py
python3 reproduce/r70_completeness.py --write
python3 reproduce/audit_rounds.py
python3 reproduce/validate_repo.py
python3 reproduce/validate_release.py
python3 reproduce/scrutinize.py --json
python3 -m unittest discover -s reproduce -p 'test_*.py'
```

The first six commands rebuild published tables, indexed records, joins, and audit summaries from their committed inputs. They write generated files; inspect `git diff` afterward. `audit_rounds.py` and `r70_completeness.py --write` also rewrite round pages or summaries. For a read-only check, skip the rebuilding commands and run the validators and tests against the committed outputs.

The reproduction inventory has 177 round entries: 19 have an analysis-only replay command and 158 are marked unavailable. Run the available round-specific commands below to recompute their published tables. Repeated inventory entries for the same command are shown once. An unavailable entry has no complete per-round command that recreates its published figures from the current snapshot; scoped observed-row audits may still be available.

The recovered-source token audit covers 105 mined round tags. It rebuilds observed-row summaries and token metadata comparisons from the committed mined records; it does not restore missing published values or reproduce a full historical round. Run it with:

```sh
python3 reproduce/recompute_mined.py
```

## Operator-only source recovery

`reproduce/mine_probe.py` is a recovery/import utility, not a public-snapshot reproducer. It requires an operator-owned recovery tree containing the source `results/` directory with per-cell directories and manifests, plus its separate `grades.final.jsonl`. Those inputs are not committed here; the default locations are outside this repository. When importing recovered data, pass their locations with `--results` and `--grades`; use `--output` to direct generated mined records and archives to a review copy. The public commands above consume the committed output and do not invoke this utility.

```sh
python3 reproduce/cache_replay_tables.py
python3 reproduce/reconcile_l0.py
python3 reproduce/l2-plan-specificity.py
python3 rounds/l3-repair/reproduce/l3_repair.py
python3 rounds/l3b-repair-vs-continue/reproduce/l3b_repair_vs_continue.py
python3 rounds/l4-context-packet/reproduce/l4-context-packet.py
python3 rounds/l4b-confirm/reproduce/reproduce_us.py
python3 rounds/l5-sol-comparators/reproduce/l5_sol_comparators.py
python3 reproduce/score_l6.py --cases rounds/l6-auditor-replay/data/case_list.csv --calls rounds/l6-auditor-replay/data/calls.csv
python3 rounds/lang-sol-replication/reproduce/lang_sol_replication.py
python3 rounds/llm-latency-long/reproduce.py
python3 rounds/llm-latency-short/reproduce.py
python3 rounds/r70-rve/reproduce.py
python3 rounds/r70-rve-task8v2/reproduce.py
```

`validate_release.py` and `scrutinize.py --json` are separate checks. `validate_release.py` validates historical round labels, applies strict Standard-record checks only to rounds dated 2026-10-09 or later, runs the synthetic record-writer contract against the current schema, checks the registered publication gates, and then runs `validate_repo.py`; it does not run scrutiny or inspect independent-review receipts. The record-writer contract finalizes a checked-in synthetic cell through the same writer as `run-lane.sh`, then requires a schema-valid record with zero missing fields. `scrutinize.py --json` independently recomputes configured evidence checks and returns nonzero when its report contains findings; inspect those findings rather than treating a nonzero exit as proof that no figures were recomputed.

For rounds dated 2026-10-09 or later, `VALID` requires complete Standard capture and no protocol deviations. `DESCRIPTIVE`, `INVALID`, `INCOMPLETE`, and `PILOT` may have missing Standard fields only when every missing field is listed in that round's `MISSING-DECLARED.json`, with a reason code from `results/missing-reasons.json` and a plain-language note. Their README must include the matching `What's missing and why` section; the release validator checks it against the declaration and reports how many missing fields were declared. This exception documents capture gaps and does not upgrade a round's analytical status. Historical rounds before the cutoff retain their labelled historical publication route.

| Check | What it proves | Limit |
| --- | --- | --- |
| `export_results.py` | Rebuilds official outcome exports from the grade snapshot and numeric metadata; reconciles duplicate grades and preserves missing usage as null. | Does not reconstruct ungraded delivery denominators by itself or regrade tasks. |
| `build_records.py`, `build_grade_join.py`, and `build_l3b_records.py` | Rebuild Standard records and grade-to-delivery links from indexed, sanitized inputs; the join checks exact ID coverage and equality where source rows exist. | Cannot recover records that were not retained. |
| `r70_completeness.py --write` | Recomputes the published Round 70 ledger and its summaries from the named public outcome and diagnostic inputs. | Does not infer absent costs, counters, or outcomes. |
| `audit_rounds.py` | Rebuilds the historical record and gap audit summaries from published evidence. | Does not upgrade a round's validity or make missing evidence complete. |
| `validate_repo.py` | Checks repository indexes, records, cross-references, numbers covered by its rules, privacy constraints, and authored relative links. | It is a consistency validator, not an independent rerun of historical model calls or hidden grading. |
| `validate_release.py` | Cross-checks historical round status, badges, reasons, dates, and recomputation labels; applies strict Standard-record checks only from 2026-10-09 onward; runs the synthetic record-writer contract; verifies the publication-blocker register; then runs `validate_repo.py`. | It does not run `scrutinize.py`, require independent-review receipts, or invoke official-grade round reproducers. A passing exit establishes only that the checks above passed. |
| `scrutinize.py --json` | Recomputes the configured release-round headline counts, rates, token arithmetic, record joins, and related evidence checks, and emits machine-readable findings. | Covers the configured rounds and checks only; it does not prove unobserved historical events or restore missing evidence. |
| Python tests | Exercise record transformations, validation policies, round-trip behavior, scrutiny arithmetic, and release-gate predicates. | Tests prove these code paths behave as specified, not that every historical input is complete. |

Numbers are recomputable only to the extent their source rows and formulas are present. In particular, the published cost estimates and cost comparisons are not re-derivable from this snapshot because their calculator and analysis inputs are absent. Source-reported hashes cannot be independently checked when their underlying receipts are absent. The round pages and `reproduce/round-inventory.json` state these limits; missing values remain missing.
