> Data revision: baf7c2f4307d3a989a8fb9363df8000fda3d9310
> Source files:
- [hypotheses/f06-checks-review-recovery.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/hypotheses/f06-checks-review-recovery.md)
- [hypotheses/README.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/hypotheses/README.md)

# F06 — Checks, review, and recovery

> Family source text is narrative from the repository. Any numeric value in that text remains source-reported unless an exact claim ledger entry supplies it.

# F06 Checks, review, and recovery

This page records evidence for H62–H74. The propositions below are hypotheses, not established findings. Published measurements remain attached to their individual runs and claim IDs. Where a B1 claim has no hypothesis mapping, it is cited as contextual evidence without changing that ledger mapping.

## Hypotheses and current status

| ID | Falsifiable proposition | Current evidence status |
|---|---|---|
| H62 | A selective, faster check set can preserve detection of real failures while reducing check time or cost. | NOT ESTABLISHED. No public matched before/after result measures both failure detection and work or time. |
| H63 | A catalog of safe parallel tests can reduce verification wall time without changing results. | NOT ESTABLISHED. No public controlled catalog or concurrency comparison is identified. |
| H64 | Targeted tests of Build mechanics can preserve defect detection at lower cost than repeated full shaping checks. | NOT ESTABLISHED. No matched mechanics-test versus full-check result is published. |
| H65 | Clear handoff, process custody, failure classes, and visible stage status can prevent repeated expensive Build attempts. | NOT ESTABLISHED. Public records include blocked and smoke-only workflows, but no isolated live recovery comparison. |
| H66 | Strong deterministic checks can enable a low-effort model to complete tasks that weaker checks miss. | NOT ESTABLISHED. No check-ablation comparison is joined to official outcomes. |
| H67 | A scoped reviewer veto improves acceptance quality enough to justify its review and repair cost. | INTERIM, OFFLINE ONLY. L6 misses both targets in the selected sample and is not a prospective live veto test. |
| H68 | Executable review catches small-diff false accepts that other review methods miss. | INTERIM, OFFLINE ONLY. The public summaries do not include the patch-level decision ledger. |
| H69 | Contract audits detect missing requested behavior and can safely gate completion or trigger repair. | INTERIM, OFFLINE ONLY. The published audit did not run a repair or Build rerun. |
| H70 | Explicit behavior checks outperform source-text or file-existence checks without losing relevant coverage. | NOT ESTABLISHED. Offline audits identify test-quality concerns, but do not compare those check designs prospectively. |
| H71 | The proposed static and domain-specific checks are accurate and fast enough for the default gate. | NOT ESTABLISHED. The study census lists check research, but no public default-gate portfolio comparison is available. |
| H72 | A post-change fix loop can repair a static-check failure rather than reject functionally passing work. | NOT ESTABLISHED. No matched repair-loop outcome is identified in the public records. |
| H73 | Verification should score observable behavior even when an implementation reports its own checks as successful. | PARTIAL, OFFLINE ONLY. Saved-output and candidate audits are counterfactual evidence, not a prospective grading-policy test. |
| H74 | Reusable duplicate, error, and type checks provide value beyond the Kogen project. | NOT ESTABLISHED. No cross-project transfer or maintenance comparison is published. |

## Run and claim record

**Live counts** refers to planned, started, finished, graded, and ITT Build cells. Offline rows have no live Build population. Where a value is reported but its underlying cases are absent, the linked claim remains INTERIM and is labeled source-reported, not reproducible from public data.

| experiment_id | arm and task set | model / effort | Kogen and harness revision | host / epoch | planned / started / finished / graded / ITT | outcome | status and correction | claim ID | public receipt and raw records |
|---|---|---|---|---|---|---|---|---|---|
| x-review-checklist | Checklist and risk-directed review; exact diagnostic cases unavailable | Exact model and effort unavailable | Offline review; no live Kogen or harness revision | Not applicable | Offline analysis; live counts not applicable | Each method was reported at 0/14 false accepts, 0/10 false blocks, and 14/14 supported-defect recall; case-level join unavailable. | INTERIM; source-reported, not reproducible from public data. | CL-WA-x-review-reported-x-review-checklist | [round page](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/x-review/README.md), [reported data](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/x-review/reported-data.json), [claim ledger](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl) |
| x-review-repair | Partial review-and-repair workflow; exact tasks unavailable | Exact model and effort unavailable | No live Kogen revision recovered | Not applicable | Offline workflow; live counts not applicable | 1/4 sessions reportedly emitted the required final result; 0/8 fresh confirmation calls occurred. | INTERIM; partial repair workflow, not task completion efficacy; source-reported, not reproducible from public data. | CL-WA-x-review-reported-x-review-repair | [round page](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/x-review/README.md), [reported data](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/x-review/reported-data.json), [claim ledger](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl) |
| x-review-semantic-repair | Separate semantic-repair screen; exact task and arm mapping unavailable | Sol high is reported; exact model build unavailable | No live Kogen revision recovered | Not recovered | Small source-reported screen; live counts not applicable | Complete deliveries reportedly failed the confirmation gate at 1/2 versus 0/2; do not combine with the diagnostic case set. | INTERIM; source-reported, not reproducible from public data. | CL-WA-x-review-reported-x-review-semantic-repair | [round page](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/x-review/README.md), [reported data](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/x-review/reported-data.json), [claim ledger](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl) |
| offline-review3 | Tool-less versus executable final-patch review; frozen patch IDs unavailable | Sol high is reported for both arms; exact model build unavailable | Offline review; no live Kogen revision | Not applicable | Offline analysis of a planned patch set; live counts not applicable | Reported paired first runs: 63/80. Tool-less: precision 19/23, recall 19/32, false vetoes 4/31. Executable: precision 21/29, recall 21/32, false vetoes 8/31. | INTERIM; budget-stopped partial study; source-reported, not reproducible from public data. Earlier interim summary superseded. | CL-WA-offline-review3-reported-observation | [round page](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/offline-review3/README.md), [claim ledger](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl) |
| offline-contract-audit | Sol medium and Luna max audits of known false accepts and passing controls; exact case IDs unavailable | Sol medium and Luna max; model builds unavailable | Offline audit; no live Kogen revision | Not applicable | Offline analysis; live counts not applicable | Exact defect found in 1/5 versus 0/5 cases; passing controls flagged in 10/10 versus 6/10. No repair or Build rerun followed. | INTERIM; source-reported, not reproducible from public data. | CL-WA-offline-contract-audit-reported-observation | [round page](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/offline-contract-audit/README.md), [claim ledger](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl) |
| intent-coverage-audit | Saved failure outputs and controls from R58, R61, R62, and R62b; no new model arm | No new model execution | Offline audit; original implementation revisions unavailable | Not applicable | Offline analysis; live counts not applicable | The report attributes 8/24 failed-test occurrences to dropped or weakened requirements, 15/24 to requirements retained but still failing, and 1/24 to stricter-than-request behavior. | INTERIM; descriptive saved-output audit, not a shaping effect. Source-reported, not reproducible from public data. | CL-WA-intent-coverage-audit-reported-observation | [round page](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/intent-coverage-audit/README.md), [claim ledger](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl) |
| overstrict-replay | Recovered ExUnit tests replayed against saved implementations from R61–R71 | No model execution | Offline replay; original source revisions unavailable | Not applicable | Offline test replay; live counts not applicable | Reported 168/1,160 replayed tests as over-strict; the provided-R71 subset was 9/23. | INTERIM; source-reported, not reproducible from public data. The replay does not show that a particular Build stop was caused by an over-strict test. | CL-WA-overstrict-replay-reported-observation | [round page](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/overstrict-replay/README.md), [claim ledger](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl) |
| candidate-counterfactual | Saved candidate diffs from R67b and R71; not official final outcomes | No new model execution | Offline counterfactual audit; original cell revisions not supplied | Not applicable | Offline candidate grades; not a live Build cohort | Seven saved candidates were graded and all seven passed; the audit classified six cells as over-blocked and found zero genuine misses among those six. | INTERIM; counterfactual only, not an official pass or ITT result; source-reported, not reproducible from public data. | CL-WA-candidate-counterfactual-reported-observation | [round page](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/candidate-counterfactual/README.md), [claim ledger](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl) |
| l6-auditor-replay | Request-scoped auditor on 40 selected r70 patches; two auditor models | Luna max and Sol low | Offline request runner; fingerprints and source-case IDs are in the round page | 80 requests; 79 valid verdicts; one timeout; live grade and ITT counts are zero | Neither auditor meets both predeclared precision and false-demotion targets; advisory only for this sample. | VALID offline replay; not a live gate qualification. | `CL-L6-AUDITOR-ADVISORY-REPLAY` (**STATUS_ONLY**) | [L6 auditor replay](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l6-auditor-replay/README.md), [claim ledger](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl) |
| x-continuation-live | Live continuation path; task IDs unavailable | No real-model attempt reported | Exact Kogen and harness revisions unavailable | Not recovered | Blocked before model execution; no scored live population | No live continuation result. | NOT-RUN; source-reported, not reproducible from public data. | CL-WA-x-continuation-reported-x-continuation-live | [round page](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/x-continuation/README.md), [reported data](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/x-continuation/reported-data.json), [claim ledger](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl) |
| x-continuation-scripted-provider | Scripted-provider mechanical path; exact task IDs unavailable | Scripted provider; no live model | Exact Kogen and harness revisions unavailable | Not recovered | Offline mechanical test; live counts not applicable | A source summary reports 3/6 checks before and 6/6 after a mechanical change; this is not a task outcome. | NOT-RUN for live efficacy; source-reported, not reproducible from public data. | CL-WA-x-continuation-reported-x-continuation-scripted-provider | [round page](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/x-continuation/README.md), [reported data](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/x-continuation/reported-data.json), [claim ledger](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl) |
| x-recovery | Recovery lifecycle; exact task set unavailable | Model and effort unavailable | Exact Kogen and harness revisions unavailable | Not recovered | Planned, started, finished, graded, and ITT counts unavailable | Ledger identified; per-arm extraction remains required. | INTERIM; no outcome estimate. | CL-WA-x-recovery-reported-x-recovery | [round page](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/x-recovery/README.md), [reported data](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/x-recovery/reported-data.json), [claim ledger](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl) |

## Design implications

The public evidence supports treating test quality and review false blocks as risks to measure. It does not qualify a default review veto, a fixed list of fast checks, automatic test demotion, or an automatic contract-audit gate. Keep offline diagnostics, saved-candidate counterfactuals, and live Build outcomes separate.

## What this evidence does not show

It does not establish that a reviewer improves live acceptance, that additional checks preserve defect detection while reducing time, or that a recovery loop increases completion. Offline counts are not live Build rates. A candidate that passes a saved grading counterfactual is not an official round pass.

## Smallest open tests

Freeze a blinded set of true defects, valid alternatives, and passing controls; compare each proposed check or reviewer with a matched baseline; record exact case IDs, false accepts, false blocks, time, cost, and fresh confirmation outcomes. For H65 and H72, run a prospective recovery test that records process custody and stage transitions and counts Kogen-internal stops under the declared ITT rule.

## Evidence links

Related round histories: [R37](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r37/README.md), [R48b](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r48b/README.md), [R56](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r56/README.md), [R67b](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r67b/README.md). The [research census](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/research/README.md) records related check and review studies as index metadata only; no per-study synthesis is used here.


## Related structured records

### Related hypotheses

- [H62](/hypotheses/h62/)
- [H63](/hypotheses/h63/)
- [H64](/hypotheses/h64/)
- [H65](/hypotheses/h65/)
- [H66](/hypotheses/h66/)
- [H67](/hypotheses/h67/)
- [H68](/hypotheses/h68/)
- [H69](/hypotheses/h69/)
- [H70](/hypotheses/h70/)
- [H71](/hypotheses/h71/)
- [H72](/hypotheses/h72/)
- [H73](/hypotheses/h73/)
- [H74](/hypotheses/h74/)

### Related experiments

- [l6-auditor-replay](/rounds/l6-auditor-replay/)
- [studio-kogen](/rounds/studio-kogen/)
- [studio-kogen-80a4](/rounds/studio-kogen-80a4/)
- [studio-shaper-pilot](/rounds/studio-shaper-pilot/)
- [x-continuation](/rounds/x-continuation/)
- [x-recovery](/rounds/x-recovery/)
- [x-review](/rounds/x-review/)

### Related claims

- [CL-L6-AUDITOR-ADVISORY-REPLAY](/claims/cl-l6-auditor-advisory-replay/)
- [CL-WA-studio-kogen-80a4-lifecycle-and-outcome](/claims/cl-wa-studio-kogen-80a4-lifecycle-and-outcome/)
- [CL-WA-studio-kogen-lifecycle-and-outcome](/claims/cl-wa-studio-kogen-lifecycle-and-outcome/)
- [CL-WA-studio-shaper-pilot-lifecycle-and-outcome](/claims/cl-wa-studio-shaper-pilot-lifecycle-and-outcome/)
- [CL-WA-x-continuation-reported-x-continuation-live](/claims/cl-wa-x-continuation-reported-x-continuation-live/)
- [CL-WA-x-continuation-reported-x-continuation-scripted-provider](/claims/cl-wa-x-continuation-reported-x-continuation-scripted-provider/)
- [CL-WA-x-recovery-reported-x-recovery](/claims/cl-wa-x-recovery-reported-x-recovery/)
- [CL-WA-x-review-reported-x-review-checklist](/claims/cl-wa-x-review-reported-x-review-checklist/)
- [CL-WA-x-review-reported-x-review-repair](/claims/cl-wa-x-review-reported-x-review-repair/)
- [CL-WA-x-review-reported-x-review-semantic-repair](/claims/cl-wa-x-review-reported-x-review-semantic-repair/)
