Round intent-short-1
Short intent with complete acceptance scenarios
Source-reported narrativeRound README and results text are source material. Numeric values that are not tied to a claim ledger record remain source-reported.
Round source README · rounds/intent-short-1/README.md
A descriptive development screen. The round is approximate: Lab verified the code in the builder's workspace, not in a clean copy of the candidate as Kogen now does.
Question and scope
When acceptance scenarios cover all requirements checked by hidden tests, can a short intent keep the default builder's correctness against full prose while reducing input?
What each arm received
The full arm received the complete prose specification plus an extended Gherkin acceptance feature and its step definitions. The short arm received an Intent frame with a title, purpose, command-line synopsis, option names, and a 19–23-word prose body, plus the same extended feature and steps. The feature was checked to include every requirement exercised by the hidden tests. The two arms used identical acceptance scenarios for each task.
A rescue means review changes a tree that failed one or more hidden tests into a tree that passes them all. Three reviewer edits were made, all on code that already passed before review, so this screen recorded no rescues.
Fixed rule
SHORT HOLDS if the short arm has no more wrong or stopped attempts than the full arm, and no more wrong first-green builds. SHORT LOSES if short has at least two more wrong or stopped attempts, or at least two more wrong first-green builds. Otherwise the result is inconclusive.
Results
A first-green build is the first code snapshot that passed the workflow's verification checks; hidden tests determine whether it was correct.
The run scored 16 of 16 attempts, with two repetitions on each of four development tasks.
| Arm | Correct submissions | Wrong submissions | Builder stops | Provider refusals | Wrong first-green builds | Reviewer edits | Builder input tokens | Reviewer input tokens | Total input tokens | Sum of review-stage durations (Lab bundle) |
|---|---|---|---|---|---|---|---|---|---|---|
| Full prose | 8 of 8 | 0 | 0 | 0 | 0 of 8 | 2 | 0.56M | 0.70M | 1.26M | 731.4 s |
| Short intent | 8 of 8 | 0 | 0 | 0 | 0 of 8 | 1 | 0.49M | 0.53M | 1.02M | 652.8 s |
| Task ID | Full correct submissions | Short correct submissions |
|---|---|---|
| deps-ex | 2 of 2 | 2 of 2 |
| deps-go | 2 of 2 | 2 of 2 |
| interval-ex | 2 of 2 | 2 of 2 |
| interval-go | 2 of 2 | 2 of 2 |
The fixed rule returns SHORT HOLDS. Both arms had 8 of 8 correct submissions, no wrong submissions, no stops, and no wrong first-green builds. The short arm used 19% less input (1.02M versus 1.26M). All three pre-review trees listed in PRE-REVIEW-GRADES.json were already correct.
Limits
This is a small screen on four development tasks with two repetitions per task and the default builder. Weaker builders and less fully specified work were not tested. The extended features needed 32–35 extra scenarios per specification to carry every tested requirement, so shorter prose moved detail into the scenarios.
Required reproduction metadata
The frozen settings, resolved configurations, and per-attempt outcomes are in the public bundle. Lab outcomes are descriptive; no official benchmark grade was assigned. The attempt records use Standard 1.2, with measured fields in MEASURED.md and declared gaps in MISSING.md and MISSING-DECLARED.json. PRE-REVIEW-GRADES.json contains the three model-free pre-review grades. The bundle excludes raw conversations, tool output, and candidate code.
Metadata
Round date: 2026-10-21 Label: DESCRIPTIVE Publication badge: DESCRIPTIVE; KEPT FOR AUDIT Recomputation status: OBSERVED SOURCE ONLY Why not VALID: DESCRIPTIVE_ONLY, PROTOCOL_DEVIATION, INCOMPLETE_CAPTURE — development screen, approximate execution, no official grades, and missing Standard capture fields declared below. STATUS: DESCRIPTIVE
Execution record
Lab build: 3bf295ef Kogen commit: b45ba32bacf26e2b17cb0bb81015ba56d27a4b0b Model and effort: the default builder and reviewer settings. Task IDs: deps-ex, deps-go, interval-ex, interval-go. Harness commit: not recorded as a public Git commit; the execution build is identified above. Command: the experiment was run and exported in Lab before publication. Raw records: records.jsonl and the public bundle preserve attempt outcomes and configuration. The run scored 16 of 16 attempts and used 2.28M input tokens of the 6M budget. The short arm was 8 of 8 correct against 8 of 8 for full; it used 19% less input. The public bundle retains the frozen configuration and per-attempt outcomes.
What's missing and why
circumstances.cap_end(16 missing): The Lab export does not carry this Standard field. Boundary telemetry not retained. Reason: Boundary telemetry not retained (m08).circumstances.cap_start(16 missing): The Lab export does not carry this Standard field. Boundary telemetry not retained. Reason: Boundary telemetry not retained (m08).circumstances.concurrent_cells_end(16 missing): The Lab export does not carry this Standard field. Host-wide concurrent cell count was not captured. Reason: Host-wide concurrent cell count was not captured (m66).circumstances.concurrent_cells_start(16 missing): The Lab export does not carry this Standard field. Host-wide concurrent cell count was not captured. Reason: Host-wide concurrent cell count was not captured (m66).circumstances.dispatcher_id(16 missing): The Lab export does not carry this Standard field. Exact dispatcher revision is not present in the public receipt extract. Reason: Exact dispatcher revision is not present in the public receipt extract (m63).circumstances.incidents(16 missing): The Lab export does not carry this Standard field. Per-cell incident receipt was not captured. Reason: Per-cell incident receipt was not captured (m78).circumstances.load1_end(16 missing): The Lab export does not carry this Standard field. Host boundary load was not captured in the public round data. Reason: Host boundary load was not captured in the public round data (m65).circumstances.load1_start(16 missing): The Lab export does not carry this Standard field. Host boundary load was not captured in the public round data. Reason: Host boundary load was not captured in the public round data (m65).circumstances.load_samples(16 missing): The Lab export does not carry this Standard field. Timestamped host-wide load samples were not captured. Reason: Timestamped host-wide load samples were not captured (m93).circumstances.queue(16 missing): The Lab export does not carry this Standard field. Queue receipt was not captured. Reason: Queue receipt was not captured (m91).cost.accounting(16 missing): The Lab export does not carry this Standard field. Per-cell cost and pricing inputs are not present in the public round data. Reason: Per-cell cost and pricing inputs are not present in the public round data (m70).cost.calculator_version(16 missing): The Lab export does not carry this Standard field. Cost calculator version not recorded. Reason: Cost calculator version not recorded (m61).cost.long_context_reconciled(16 missing): The Lab export does not carry this Standard field. Boolean receipt not recorded. Reason: Boolean receipt not recorded (m07).cost.price_table_version(16 missing): The Lab export does not carry this Standard field. Price table version not recorded. Reason: Price table version not recorded (m89).cost.usd(16 missing): The Lab export does not carry this Standard field. Per-cell cost and pricing inputs are not present in the public round data. Reason: Per-cell cost and pricing inputs are not present in the public round data (m70).effort.effective(16 missing): The Lab export does not carry this Standard field. Per-cell effective-effort receipt not published. Reason: Per-cell effective-effort receipt not published (m73).effort.requested(16 missing): The Lab export does not carry this Standard field. Not recorded in available public metadata. Reason: Not recorded in available public metadata (m39).effort.runner_requested(16 missing): The Lab export does not carry this Standard field. Not recorded in available public metadata. Reason: Not recorded in available public metadata (m39).environment.account_class(16 missing): The Lab export does not carry this Standard field. No owner-approved class-only account receipt is available for the cell. Reason: No owner-approved class-only account receipt is available for the cell (m100).environment.network.allowlist_hosts(16 missing): The Lab export does not carry this Standard field. Per-cell network allowlist not present in the public round data. Reason: Per-cell network allowlist not present in the public round data (m79).environment.network.profile(16 missing): The Lab export does not carry this Standard field. Per-cell network profile not present in the public round data. Reason: Per-cell network profile not present in the public round data (m80).environment.toolchains.erlang(16 missing): The Lab export does not carry this Standard field. Toolchain version not recorded. Reason: Toolchain version not recorded (m95).environment.toolchains.other_inventory(16 missing): The Lab export does not carry this Standard field. Full per-cell toolchain inventory not recorded. Reason: Full per-cell toolchain inventory not recorded (m64).environment.toolchains.ruby(16 missing): The Lab export does not carry this Standard field. Toolchain version not recorded. Reason: Toolchain version not recorded (m95).environment.toolchains.rust(16 missing): The Lab export does not carry this Standard field. Toolchain version not recorded. Reason: Toolchain version not recorded (m95).grade.grader(16 missing): The Lab export does not carry this Standard field. Per-cell grader software version is not in the public data. Reason: Per-cell grader software version is not in the public data (m77).grade.timestamp(16 missing): The Lab export does not carry this Standard field. No official grade in the public snapshot. Reason: No official grade in the public snapshot (m32).host.id(16 missing): The Lab export does not carry this Standard field. Not recorded in available public metadata. Reason: Not recorded in available public metadata (m39).itt.class(16 missing): The Lab export does not carry this Standard field. No evidence-backed ITT classification. Reason: No evidence-backed ITT classification (m28).itt.cohort(16 missing): The Lab export does not carry this Standard field. No evidence-backed ITT classification. Reason: No evidence-backed ITT classification (m28).outcome(16 missing): The Lab export does not carry this Standard field. Not available for ungraded delivery. Reason: Not available for ungraded delivery (m38).provenance.ledger_sha256(16 missing): The Lab export does not carry this Standard field. Not recorded in available public metadata. Reason: Not recorded in available public metadata (m39).sandbox.egress_allow(16 missing): The Lab export does not carry this Standard field. Per-cell egress allowlist is not present in the public round data. Reason: Per-cell egress allowlist is not present in the public round data (m75).sandbox.egress_profile(16 missing): The Lab export does not carry this Standard field. Per-cell egress profile is not present in the public round data. Reason: Per-cell egress profile is not present in the public round data (m76).sandbox.profile(16 missing): The Lab export does not carry this Standard field. Per-cell sandbox profile is not present in the public round data. Reason: Per-cell sandbox profile is not present in the public round data (m87).sandbox.profile_sha256(16 missing): The Lab export does not carry this Standard field. Per-cell sandbox profile fingerprint is not present in the public round data. Reason: Per-cell sandbox profile fingerprint is not present in the public round data (m86).setup.adapter_harness_sha(16 missing): The Lab export does not carry this Standard field. Not recorded in available public metadata. Reason: Not recorded in available public metadata (m39).setup.deps_source(16 missing): The Lab export does not carry this Standard field. Per-cell dependency source is not recorded. Reason: Per-cell dependency source is not recorded (m71).setup.sandbox_mode(16 missing): The Lab export does not carry this Standard field. Per-cell sandbox mode is not present in the public round data. Reason: Per-cell sandbox mode is not present in the public round data (m85).setup.sandbox_profile_sha256(16 missing): The Lab export does not carry this Standard field. Per-cell sandbox fingerprint is not present in the public round data. Reason: Per-cell sandbox fingerprint is not present in the public round data (m84).setup.task_base.hash(16 missing): The Lab export does not carry this Standard field. Original base commit/tree hash absent; fresh_base_commit is not a base hash. Reason: Original base commit/tree hash absent; fresh_base_commit is not a base hash (m42).setup.task_base.kind(16 missing): The Lab export does not carry this Standard field. Original base revision type not recorded per cell. Reason: Original base revision type not recorded per cell (m43).task.base_repo(16 missing): The Lab export does not carry this Standard field. Not recorded in available public metadata. Reason: Not recorded in available public metadata (m39).task.base_revision.hash(16 missing): The Lab export does not carry this Standard field. Original base commit/tree hash absent; fresh_base_commit is not a base hash. Reason: Original base commit/tree hash absent; fresh_base_commit is not a base hash (m42).task.base_revision.kind(16 missing): The Lab export does not carry this Standard field. Original base revision type not recorded per cell. Reason: Original base revision type not recorded per cell (m43).timestamps.phases.develop.end_utc(16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.develop.start_utc(16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.gate.end_utc(16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.gate.start_utc(16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.grade.end_utc(16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.grade.start_utc(16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.plan.end_utc(16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.plan.start_utc(16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.review.end_utc(16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.review.start_utc(16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.setup.end_utc(16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.setup.start_utc(16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.shape.end_utc(16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.shape.start_utc(16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timing.phases_s.gate(16 missing): The Lab export does not carry this Standard field. Phase wall not emitted or not separable. Reason: Phase wall not emitted or not separable (m45).timing.phases_s.setup(16 missing): The Lab export does not carry this Standard field. Phase wall not emitted or not separable. Reason: Phase wall not emitted or not separable (m45).timing.timeout_cap_s(16 missing): The Lab export does not carry this Standard field. Timeout receipt unavailable. Reason: Timeout receipt unavailable (m53).tools.grader(16 missing): The Lab export does not carry this Standard field. Per-cell grader software version is not in the public data. Reason: Per-cell grader software version is not in the public data (m77).tools.runner(16 missing): The Lab export does not carry this Standard field. Per-cell runner version not recorded in the public round data. Reason: Per-cell runner version not recorded in the public round data (m83).tools.toolchains.erlang(16 missing): The Lab export does not carry this Standard field. Toolchain version not recorded. Reason: Toolchain version not recorded (m95).tools.toolchains.other_inventory(16 missing): The Lab export does not carry this Standard field. Full per-cell toolchain inventory not recorded. Reason: Full per-cell toolchain inventory not recorded (m64).tools.toolchains.ruby(16 missing): The Lab export does not carry this Standard field. Toolchain version not recorded. Reason: Toolchain version not recorded (m95).tools.toolchains.rust(16 missing): The Lab export does not carry this Standard field. Toolchain version not recorded. Reason: Toolchain version not recorded (m95).
Captured run records
Standard-schema records assigned to this round in the indexed snapshot.
No per-round file is listed. Check the run-record index for the complete partition list.
Reproduction inventory
Published analysis replay and full experiment replay are reported separately.
- Replay type
analysis-only- Script
- No per-round analysis script is published.
- Inputs
rounds/intent-short-1/MEASURED.mdrounds/intent-short-1/MISSING-DECLARED.jsonrounds/intent-short-1/MISSING.mdrounds/intent-short-1/PLAN.jsonrounds/intent-short-1/PRE-REVIEW-GRADES.jsonrounds/intent-short-1/README.mdrounds/intent-short-1/bundles/is1/EXCLUDED.jsonrounds/intent-short-1/bundles/is1/attempts/deps-ex/r0/full/a1/attempt.jsonrounds/intent-short-1/bundles/is1/attempts/deps-ex/r0/full/a1/evaluation.jsonrounds/intent-short-1/bundles/is1/attempts/deps-ex/r0/short/a1/attempt.jsonrounds/intent-short-1/bundles/is1/attempts/deps-ex/r0/short/a1/evaluation.jsonrounds/intent-short-1/bundles/is1/attempts/deps-ex/r1/full/a1/attempt.jsonrounds/intent-short-1/bundles/is1/attempts/deps-ex/r1/full/a1/evaluation.jsonrounds/intent-short-1/bundles/is1/attempts/deps-ex/r1/short/a1/attempt.jsonrounds/intent-short-1/bundles/is1/attempts/deps-ex/r1/short/a1/evaluation.jsonrounds/intent-short-1/bundles/is1/attempts/deps-go/r0/full/a1/attempt.jsonrounds/intent-short-1/bundles/is1/attempts/deps-go/r0/full/a1/evaluation.jsonrounds/intent-short-1/bundles/is1/attempts/deps-go/r0/short/a1/attempt.jsonrounds/intent-short-1/bundles/is1/attempts/deps-go/r0/short/a1/evaluation.jsonrounds/intent-short-1/bundles/is1/attempts/deps-go/r1/full/a1/attempt.jsonrounds/intent-short-1/bundles/is1/attempts/deps-go/r1/full/a1/evaluation.jsonrounds/intent-short-1/bundles/is1/attempts/deps-go/r1/short/a1/attempt.jsonrounds/intent-short-1/bundles/is1/attempts/deps-go/r1/short/a1/evaluation.jsonrounds/intent-short-1/bundles/is1/attempts/interval-ex/r0/full/a1/attempt.jsonrounds/intent-short-1/bundles/is1/attempts/interval-ex/r0/full/a1/evaluation.jsonrounds/intent-short-1/bundles/is1/attempts/interval-ex/r0/short/a1/attempt.jsonrounds/intent-short-1/bundles/is1/attempts/interval-ex/r0/short/a1/evaluation.jsonrounds/intent-short-1/bundles/is1/attempts/interval-ex/r1/full/a1/attempt.jsonrounds/intent-short-1/bundles/is1/attempts/interval-ex/r1/full/a1/evaluation.jsonrounds/intent-short-1/bundles/is1/attempts/interval-ex/r1/short/a1/attempt.jsonrounds/intent-short-1/bundles/is1/attempts/interval-ex/r1/short/a1/evaluation.jsonrounds/intent-short-1/bundles/is1/attempts/interval-go/r0/full/a1/attempt.jsonrounds/intent-short-1/bundles/is1/attempts/interval-go/r0/full/a1/evaluation.jsonrounds/intent-short-1/bundles/is1/attempts/interval-go/r0/short/a1/attempt.jsonrounds/intent-short-1/bundles/is1/attempts/interval-go/r0/short/a1/evaluation.jsonrounds/intent-short-1/bundles/is1/attempts/interval-go/r1/full/a1/attempt.jsonrounds/intent-short-1/bundles/is1/attempts/interval-go/r1/full/a1/evaluation.jsonrounds/intent-short-1/bundles/is1/attempts/interval-go/r1/short/a1/attempt.jsonrounds/intent-short-1/bundles/is1/attempts/interval-go/r1/short/a1/evaluation.jsonrounds/intent-short-1/bundles/is1/frozen/experiment.tomlrounds/intent-short-1/bundles/is1/frozen/manifest.jsonrounds/intent-short-1/bundles/is1/frozen/resolved/full.tomlrounds/intent-short-1/bundles/is1/frozen/resolved/short.tomlrounds/intent-short-1/bundles/is1/manifest.jsonrounds/intent-short-1/bundles/is1/report.mdrounds/intent-short-1/records.jsonl
- Full execution replay
- Unavailable
- Scope
- Public bundles and records preserve observed Lab outcomes and configuration; model execution cannot be replayed from this repository snapshot.
View the complete per-round reproduction inventory · Repository reproduction guide