F01

F01 — Shape, Intent and plan

The authored family record is reproduced below. Structured links connect its hypotheses, rounds, and claim-ledger rows.

Source-reported narrativeNumeric statements in this family document remain source-reported unless an exact claim-ledger row is linked and its status allows that interpretation.

Family source document · hypotheses/f01-shape-intent-plan.md

F01 — Shape, Intent and plan

Coverage: H01–H15. The propositions below are testable framings of the registered questions; they do not report results. The round receipts remain authoritative for design and run history. Claim IDs point to `results/claim-ledger.jsonl`. A claim marked INTERIM, CONFOUNDED, or WITHDRAWN does not establish a measured advantage.

Hypotheses

ID Falsifiable proposition Current state
H01 On matched tasks and model settings, the full Kogen Shape/Build/Review route has a higher pass rate than direct Codex. UNRESOLVED; available same-model summaries are not a joined public comparison.
H02 Shaping improves final task completion enough to offset its added cost and time. UNRESOLVED; no public causal estimate isolates shaping.
H03 For a defined population of approved Intents that pass specified probes, Build completion can approach the declared reliability target. NOT ESTABLISHED; no validated guarantee cohort is available.
H04 A bounded Shaping time and human-input budget can minimize total cost while preserving completion and resolving material unknowns. UNRESOLVED; no public duration-by-human-input comparison.
H05 Running probes that settle stated assumptions before Build reduces failed or stopped Builds against a matched no-probe condition. UNRESOLVED; no isolated public contrast.
H06 A task-granularity rule for Intents improves completion per unit of integration and coordination cost over alternative sizes. UNRESOLVED; no public granularity comparison.
H07 More specific Intent detail raises builder success, with a larger effect for weaker builders than for stronger builders. UNRESOLVED; related records are exploratory or have unreconciled outcome joins.
H08 A compact context pack and a strong-model plan allow a cheap builder to outperform its matched direct baseline without exceeding a direct strong-model cost target. UNRESOLVED; no joined public route comparison.
H09 One strong-model plan can guide multiple Luna-low builders and a low-cost integrator with equal or higher completion at lower total cost than a matched alternative. NOT ESTABLISHED; no complete public comparison identified.
H10 Giving a weak builder access to a stronger advisor after a defined failure state improves completion per total cost over retrying or restarting with a strong model. NOT ESTABLISHED; adjacent escalation records do not test this advisor contrast.
H11 Escalation triggered by observed failure matches or exceeds text-based pre-routing on completion while reducing total cost. UNRESOLVED; cascade records do not yield a reconciled public verdict.
H12 Checks, review, and actionable retries increase final completion over a reject-only gate under matched tasks and attempt budgets. NOT ESTABLISHED; no prospective repair-benefit estimate.
H13 Two complementary shapers resolve failures missed by either single shaper at an acceptable added cost. NOT ESTABLISHED; no matched public comparison identified.
H14 Sol planning with Luna-max building improves the success-cost frontier over an all-Luna route on matched tasks. UNRESOLVED; available R74 record is WITHDRAWN.
H15 More complete Shaping plus executable specifications improves success or cost over a matched direct Codex or Grok Build. NOT ESTABLISHED; no controlled outcome comparison.

Experiment register

Fields left as “not recovered” were not inferred from adjacent rounds or source summaries. Lifecycle counts in a cited round can describe separate populations; read the round page and its MEASURED.md/MISSING.md before using them.

experiment_id Arm / task set Model / effort Revision; host / epoch Planned / started / finished / graded / ITT n Outcome, status, correction or supersession Claim ID Public receipt
r7 Planning and builder pilot; exact scored arm/task mapping not reconstructed. See receipt; not fully joined. See receipt; environment epoch not established. Not reconciled. INTERIM; previous captured pass summary removed after disagreement with official outcomes. No H01–H15-linked claim ID. r7
r8–r10 Early route/plan records; the surviving public pages do not establish a complete testable contrast. Not recovered. Not recovered. Not reconciled. INTERIM documentary records; no efficacy conclusion. No H01–H15-linked claim ID. r8, r9, r10
r49 / r49b Model-cascade arms; see the separate round tables for exported tasks and arms. Luna and Sol settings are listed in the receipts; requested and effective settings require care. See receipts; do not pool the two rounds. Not reconciled as one cohort. INTERIM; arm/outcome mapping and decision rule are incomplete. No advisor-after-stall effect is established. No H01–H15-linked claim ID. r49, r49b
r51 Planning-step variants; exact task and arm interpretation is in the round receipt. See receipt. See receipt. Historical denominator is not reproduced as a public exact-cell cohort. INTERIM; the public decisions page recomputes the plan-versus-none test and rejects the earlier unreproducible p-value. Treat this as exploratory, not a general shaping effect. No H01–H15-linked claim ID. r51, public correction
r56 / r56b Stage ablations and context arms; task sets and arm labels are on each receipt. Per-cell model and effective effort are incomplete or variable; see receipts. Receipts document the available fingerprints and epochs. Public delivery and official-grade counts differ; valid-grade filter and ITT are not reconciled. INTERIM; historic filtered contrasts are not reproduced by public outcome rows. Keep variants separate. CL-WA-r56-public-export-coverage; CL-WA-r56-source-reported-denominator-variants; CL-WA-r56b-public-export-coverage (all INTERIM). r56, r56b, claim ledger
r69 SPEC / APPROACH / ACCEPTANCE / NONE; task set and design are in the receipt. Luna builder requested at max; the receipt records an effective xhigh clamp. Source pin and desktop Mac cohort are described in the receipt; exact cell joins are incomplete. Planned design is recorded; public captured, official-grade, and historical ITT counts are different populations. INTERIM; prior grouped outcome table was removed because arm outcomes do not match the public official export. Historical four-arm totals remain source-reported, not reproducible from public data. CL-WA-r69-public-export-coverage; CL-WA-r69-source-reported-denominator-variants (both INTERIM). r69, claim ledger
r71 Provided frozen SPEC Intent versus Kogen-shaped Intent on the listed eight-task set. Luna max builder with Sol-high planning; the page notes mixed effective effort. The round page records a Kogen revision; harness revision is missing. Host placement and epoch are in the receipt. Planned 48; the receipt separates 50 captured deliveries from 18 official outcome rows. Full started/finished/graded/ITT mapping is unresolved. INTERIM; official outcomes do not match the earlier shaped-arm summary. The historic provided-versus-shaped contrast is source-reported, not reproducible from public data. CL-WA-r71-public-export-coverage; CL-WA-r71-source-reported-denominator-variants (both INTERIM). r71, claim ledger
r72 Same-model ladder comparison; P1/P2/P3 are separate; exact task IDs are not recovered. P1 is reported as Luna max; P3 is a different Sol-high comparison. The round receipt records a superseded Kogen revision; comparator harness revision is not recovered. Source-reported plan and graded subsets do not provide a reconciled overall ITT partition. INTERIM, descriptive only; no general Kogen-versus-Codex result. The lifecycle/outcome claim is not linked to H01 in Wave B. CL-WA-r72-lifecycle-and-outcome (INTERIM, linked by Wave B to H113/H136/H138, not H01). r72, claim ledger
r73 Sol Shaping with ledger/probes/auditor versus Sol Shaping alone; confirmatory design. Sol; exact request configuration is in the frozen rule. Frozen rule is linked on the receipt. Planned screen/confirm/conditional extension are documented; no scored execution is recorded. NOT-RUN; there is no outcome. No claim ID. r73
r74 Four Kogen configurations and matched direct Codex settings on the registered task panel. Sol-high Shaping and several builder settings are specified in the rule; no per-cell public model rows. The page records the planned Kogen pin; delivered-cell pin is unverified. Harness SHA and original launch command are not public. Planned design is documented; the public exports contain no r74-tagged deliveries or outcomes. WITHDRAWN after the measured harness was retired; no conclusion. Do not revive interim source totals. CL-WA-r74-public-export-coverage; CL-WA-r74-source-reported-denominator-variants (both WITHDRAWN). r74, claim ledger
planning-effect Plan/detail contrasts; the R3 inventory associates the study with H02, H07–H08, H14–H15. No public task-by-arm result file is linked. Not recovered in public data. Not recovered. Not recoverable from the research index. Study evidence remains source-reported, not reproducible from public data; no public study synthesis exists. No claim ID assigned. research census; related r51
intent-coverage-audit Offline audit of saved Intents and outputs; not a Build treatment arm. Not applicable. See study page; audit source records are not included. Live Build counts are zero; audit units are separate. Offline descriptive audit; no causal Shaping effect. Figures on the receipt are source-reported, not reproducible from public data. No claim ID assigned. Intent coverage audit
overstrict-replay Offline test replay on saved implementations; not a Build treatment arm. Not applicable. See study page; raw replay records are not included. Live Build counts are zero; replay case counts are separate. Descriptive test-quality evidence; does not measure Build completion or shaping efficacy. No claim ID assigned. over-strict replay
candidate-counterfactual Re-grade of saved candidate diffs; not official final-diff outcomes. Not applicable. See study page; candidate artifacts are not included. Live Build counts are zero; counterfactual candidate grades are separate. Counterfactual only; never add these outcomes to official ITT or treat them as an approved-Intent guarantee. No claim ID assigned. candidate counterfactual
l3-repair Fixed self-verification repair on six selected failed implementations. Direct Codex, gpt-6-luna max. Variant base SHAs are recorded on the round page; per-test grade receipts are absent. Planned / started / finished / graded / ITT: 6 / 6 / 6 / 6 / 6. Two repairs rescued; the three-rescue threshold was not met. Per-test pass-set preservation is unverifiable from the aggregate CSV. DESCRIPTIVE, not a causal repair estimate; no-regression remains unverified. CL-L3-REPAIR-RESCUE-RATE (STATUS_ONLY)

Design implications

Keep Shaping and Build as separate measured stages when the question concerns shaping cost, probes, detail, or approved-Intent reliability. State the builder model and effective effort, distinguish shaped from provided Intent, and score every assigned cell using a predeclared ITT rule. A stop, test gate, provider timeout, setup fault, and official grade are separate outcomes.

What this evidence does not show

The listed pages do not establish that more detail always helps, that a witness guarantees a successful Build, that shaping is worth its cost, or that any one plan recipe beats direct Codex or Sol. The descriptive R72 summary and the withdrawn R74 record are not current-version superiority evidence. Offline audits of Intent coverage, test strictness, and saved candidates do not estimate a live shaping effect.

Smallest open tests

Run a preregistered, same-task comparison of provided and shaped Intents with a fixed builder, effective effort, harness, checks, and cost accounting; record the exact cell IDs and failed/invalid dispositions. Separately randomize a fixed probe budget and a no-probe condition, and test a specified approval rule prospectively before making any Build-reliability guarantee.