> Data revision: baf7c2f4307d3a989a8fb9363df8000fda3d9310
> Source files:
- [hypotheses/f01-shape-intent-plan.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/hypotheses/f01-shape-intent-plan.md)
- [hypotheses/README.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/hypotheses/README.md)

# F01 — Shape, Intent and plan

> Family source text is narrative from the repository. Any numeric value in that text remains source-reported unless an exact claim ledger entry supplies it.

# F01 — Shape, Intent and plan

Coverage: H01–H15. The propositions below are testable framings of the registered questions; they do not report results. The round receipts remain authoritative for design and run history. Claim IDs point to [`results/claim-ledger.jsonl`](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl). A claim marked **INTERIM**, **CONFOUNDED**, or **WITHDRAWN** does not establish a measured advantage.

## Hypotheses

| ID | Falsifiable proposition | Current state |
|---|---|---|
| H01 | On matched tasks and model settings, the full Kogen Shape/Build/Review route has a higher pass rate than direct Codex. | UNRESOLVED; available same-model summaries are not a joined public comparison. |
| H02 | Shaping improves final task completion enough to offset its added cost and time. | UNRESOLVED; no public causal estimate isolates shaping. |
| H03 | For a defined population of approved Intents that pass specified probes, Build completion can approach the declared reliability target. | NOT ESTABLISHED; no validated guarantee cohort is available. |
| H04 | A bounded Shaping time and human-input budget can minimize total cost while preserving completion and resolving material unknowns. | UNRESOLVED; no public duration-by-human-input comparison. |
| H05 | Running probes that settle stated assumptions before Build reduces failed or stopped Builds against a matched no-probe condition. | UNRESOLVED; no isolated public contrast. |
| H06 | A task-granularity rule for Intents improves completion per unit of integration and coordination cost over alternative sizes. | UNRESOLVED; no public granularity comparison. |
| H07 | More specific Intent detail raises builder success, with a larger effect for weaker builders than for stronger builders. | UNRESOLVED; related records are exploratory or have unreconciled outcome joins. |
| H08 | A compact context pack and a strong-model plan allow a cheap builder to outperform its matched direct baseline without exceeding a direct strong-model cost target. | UNRESOLVED; no joined public route comparison. |
| H09 | One strong-model plan can guide multiple Luna-low builders and a low-cost integrator with equal or higher completion at lower total cost than a matched alternative. | NOT ESTABLISHED; no complete public comparison identified. |
| H10 | Giving a weak builder access to a stronger advisor after a defined failure state improves completion per total cost over retrying or restarting with a strong model. | NOT ESTABLISHED; adjacent escalation records do not test this advisor contrast. |
| H11 | Escalation triggered by observed failure matches or exceeds text-based pre-routing on completion while reducing total cost. | UNRESOLVED; cascade records do not yield a reconciled public verdict. |
| H12 | Checks, review, and actionable retries increase final completion over a reject-only gate under matched tasks and attempt budgets. | NOT ESTABLISHED; no prospective repair-benefit estimate. |
| H13 | Two complementary shapers resolve failures missed by either single shaper at an acceptable added cost. | NOT ESTABLISHED; no matched public comparison identified. |
| H14 | Sol planning with Luna-max building improves the success-cost frontier over an all-Luna route on matched tasks. | UNRESOLVED; available R74 record is WITHDRAWN. |
| H15 | More complete Shaping plus executable specifications improves success or cost over a matched direct Codex or Grok Build. | NOT ESTABLISHED; no controlled outcome comparison. |

## Experiment register

Fields left as “not recovered” were not inferred from adjacent rounds or source summaries. Lifecycle counts in a cited round can describe separate populations; read the round page and its `MEASURED.md`/`MISSING.md` before using them.

| experiment_id | Arm / task set | Model / effort | Revision; host / epoch | Planned / started / finished / graded / ITT n | Outcome, status, correction or supersession | Claim ID | Public receipt |
|---|---|---|---|---|---|---|---|
| `r7` | Planning and builder pilot; exact scored arm/task mapping not reconstructed. | See receipt; not fully joined. | See receipt; environment epoch not established. | Not reconciled. | **INTERIM**; previous captured pass summary removed after disagreement with official outcomes. | No H01–H15-linked claim ID. | [r7](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r7/README.md) |
| `r8`–`r10` | Early route/plan records; the surviving public pages do not establish a complete testable contrast. | Not recovered. | Not recovered. | Not reconciled. | **INTERIM** documentary records; no efficacy conclusion. | No H01–H15-linked claim ID. | [r8](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r8/README.md), [r9](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r9/README.md), [r10](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r10/README.md) |
| `r49` / `r49b` | Model-cascade arms; see the separate round tables for exported tasks and arms. | Luna and Sol settings are listed in the receipts; requested and effective settings require care. | See receipts; do not pool the two rounds. | Not reconciled as one cohort. | **INTERIM**; arm/outcome mapping and decision rule are incomplete. No advisor-after-stall effect is established. | No H01–H15-linked claim ID. | [r49](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r49/README.md), [r49b](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r49b/README.md) |
| `r51` | Planning-step variants; exact task and arm interpretation is in the round receipt. | See receipt. | See receipt. | Historical denominator is not reproduced as a public exact-cell cohort. | **INTERIM**; the public decisions page recomputes the plan-versus-none test and rejects the earlier unreproducible p-value. Treat this as exploratory, not a general shaping effect. | No H01–H15-linked claim ID. | [r51](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r51/README.md), [public correction](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/decisions/README.md) |
| `r56` / `r56b` | Stage ablations and context arms; task sets and arm labels are on each receipt. | Per-cell model and effective effort are incomplete or variable; see receipts. | Receipts document the available fingerprints and epochs. | Public delivery and official-grade counts differ; valid-grade filter and ITT are not reconciled. | **INTERIM**; historic filtered contrasts are not reproduced by public outcome rows. Keep variants separate. | `CL-WA-r56-public-export-coverage`; `CL-WA-r56-source-reported-denominator-variants`; `CL-WA-r56b-public-export-coverage` (all **INTERIM**). | [r56](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r56/README.md), [r56b](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r56b/README.md), [claim ledger](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl) |
| `r69` | SPEC / APPROACH / ACCEPTANCE / NONE; task set and design are in the receipt. | Luna builder requested at max; the receipt records an effective xhigh clamp. | Source pin and desktop Mac cohort are described in the receipt; exact cell joins are incomplete. | Planned design is recorded; public captured, official-grade, and historical ITT counts are different populations. | **INTERIM**; prior grouped outcome table was removed because arm outcomes do not match the public official export. Historical four-arm totals remain **source-reported, not reproducible from public data**. | `CL-WA-r69-public-export-coverage`; `CL-WA-r69-source-reported-denominator-variants` (both **INTERIM**). | [r69](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r69/README.md), [claim ledger](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl) |
| `r71` | Provided frozen SPEC Intent versus Kogen-shaped Intent on the listed eight-task set. | Luna max builder with Sol-high planning; the page notes mixed effective effort. | The round page records a Kogen revision; harness revision is missing. Host placement and epoch are in the receipt. | Planned 48; the receipt separates 50 captured deliveries from 18 official outcome rows. Full started/finished/graded/ITT mapping is unresolved. | **INTERIM**; official outcomes do not match the earlier shaped-arm summary. The historic provided-versus-shaped contrast is **source-reported, not reproducible from public data**. | `CL-WA-r71-public-export-coverage`; `CL-WA-r71-source-reported-denominator-variants` (both **INTERIM**). | [r71](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r71/README.md), [claim ledger](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl) |
| `r72` | Same-model ladder comparison; P1/P2/P3 are separate; exact task IDs are not recovered. | P1 is reported as Luna max; P3 is a different Sol-high comparison. | The round receipt records a superseded Kogen revision; comparator harness revision is not recovered. | Source-reported plan and graded subsets do not provide a reconciled overall ITT partition. | **INTERIM**, descriptive only; no general Kogen-versus-Codex result. The lifecycle/outcome claim is not linked to H01 in Wave B. | `CL-WA-r72-lifecycle-and-outcome` (**INTERIM**, linked by Wave B to H113/H136/H138, not H01). | [r72](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r72/README.md), [claim ledger](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl) |
| `r73` | Sol Shaping with ledger/probes/auditor versus Sol Shaping alone; confirmatory design. | Sol; exact request configuration is in the frozen rule. | Frozen rule is linked on the receipt. | Planned screen/confirm/conditional extension are documented; no scored execution is recorded. | **NOT-RUN**; there is no outcome. | No claim ID. | [r73](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r73/README.md) |
| `r74` | Four Kogen configurations and matched direct Codex settings on the registered task panel. | Sol-high Shaping and several builder settings are specified in the rule; no per-cell public model rows. | The page records the planned Kogen pin; delivered-cell pin is unverified. Harness SHA and original launch command are not public. | Planned design is documented; the public exports contain no r74-tagged deliveries or outcomes. | **WITHDRAWN** after the measured harness was retired; no conclusion. Do not revive interim source totals. | `CL-WA-r74-public-export-coverage`; `CL-WA-r74-source-reported-denominator-variants` (both **WITHDRAWN**). | [r74](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r74/README.md), [claim ledger](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl) |
| `planning-effect` | Plan/detail contrasts; the R3 inventory associates the study with H02, H07–H08, H14–H15. No public task-by-arm result file is linked. | Not recovered in public data. | Not recovered. | Not recoverable from the research index. | Study evidence remains **source-reported, not reproducible from public data**; no public study synthesis exists. | No claim ID assigned. | [research census](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/research/README.md); related [r51](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/r51/README.md) |
| `intent-coverage-audit` | Offline audit of saved Intents and outputs; not a Build treatment arm. | Not applicable. | See study page; audit source records are not included. | Live Build counts are zero; audit units are separate. | Offline descriptive audit; no causal Shaping effect. Figures on the receipt are **source-reported, not reproducible from public data**. | No claim ID assigned. | [Intent coverage audit](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/intent-coverage-audit/README.md) |
| `overstrict-replay` | Offline test replay on saved implementations; not a Build treatment arm. | Not applicable. | See study page; raw replay records are not included. | Live Build counts are zero; replay case counts are separate. | Descriptive test-quality evidence; does not measure Build completion or shaping efficacy. | No claim ID assigned. | [over-strict replay](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/overstrict-replay/README.md) |
| `candidate-counterfactual` | Re-grade of saved candidate diffs; not official final-diff outcomes. | Not applicable. | See study page; candidate artifacts are not included. | Live Build counts are zero; counterfactual candidate grades are separate. | Counterfactual only; never add these outcomes to official ITT or treat them as an approved-Intent guarantee. | No claim ID assigned. | [candidate counterfactual](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/candidate-counterfactual/README.md) |
| `l3-repair` | Fixed self-verification repair on six selected failed implementations. | Direct Codex, gpt-6-luna max. | Variant base SHAs are recorded on the round page; per-test grade receipts are absent. | Planned / started / finished / graded / ITT: 6 / 6 / 6 / 6 / 6. | Two repairs rescued; the three-rescue threshold was not met. Per-test pass-set preservation is **unverifiable** from the aggregate CSV. | **DESCRIPTIVE**, not a causal repair estimate; no-regression remains unverified. | `CL-L3-REPAIR-RESCUE-RATE` (**STATUS_ONLY**) | [L3 repair](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/l3-repair/README.md), [claim ledger](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/claim-ledger.jsonl) |

## Design implications

Keep Shaping and Build as separate measured stages when the question concerns shaping cost, probes, detail, or approved-Intent reliability. State the builder model and effective effort, distinguish shaped from provided Intent, and score every assigned cell using a predeclared ITT rule. A stop, test gate, provider timeout, setup fault, and official grade are separate outcomes.

## What this evidence does not show

The listed pages do not establish that more detail always helps, that a witness guarantees a successful Build, that shaping is worth its cost, or that any one plan recipe beats direct Codex or Sol. The descriptive R72 summary and the withdrawn R74 record are not current-version superiority evidence. Offline audits of Intent coverage, test strictness, and saved candidates do not estimate a live shaping effect.

## Smallest open tests

Run a preregistered, same-task comparison of provided and shaped Intents with a fixed builder, effective effort, harness, checks, and cost accounting; record the exact cell IDs and failed/invalid dispositions. Separately randomize a fixed probe budget and a no-probe condition, and test a specified approval rule prospectively before making any Build-reliability guarantee.


## Related structured records

### Related hypotheses

- [H01](/hypotheses/h01/)
- [H02](/hypotheses/h02/)
- [H03](/hypotheses/h03/)
- [H04](/hypotheses/h04/)
- [H05](/hypotheses/h05/)
- [H06](/hypotheses/h06/)
- [H07](/hypotheses/h07/)
- [H08](/hypotheses/h08/)
- [H09](/hypotheses/h09/)
- [H10](/hypotheses/h10/)
- [H11](/hypotheses/h11/)
- [H12](/hypotheses/h12/)
- [H13](/hypotheses/h13/)
- [H14](/hypotheses/h14/)
- [H15](/hypotheses/h15/)

### Related experiments

- [l3-repair](/rounds/l3-repair/)

### Related claims

- [CL-L3-REPAIR-RESCUE-RATE](/claims/cl-l3-repair-rescue-rate/)
