> Data revision: baf7c2f4307d3a989a8fb9363df8000fda3d9310
> Source files:
- [rounds/builder-low-1/README.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/builder-low-1/README.md)
- [results/run-records/index.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/run-records/index.json)
- [rounds/index.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/index.json)
- [results/source-crosswalk/index.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/results/source-crosswalk/index.json)
- [rounds/STATUS.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/STATUS.md)
- [reproduce/round-inventory.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/round-inventory.json)

# Builder effort at low vs medium: correctness held, total input fell 4%

**Round ID:** builder-low-1  
**Status:** DESCRIPTIVE

> Reviewed status from rounds/STATUS.md: DESCRIPTIVE.

> Round README and results text are source material. Numeric values not tied to a claim ledger record remain source-reported.

A descriptive screen with two repetitions per task. The round is approximate: Lab verified the code in the builder's workspace, not in a clean copy of the candidate as Kogen now does.

## Question and scope

Does a low-effort builder keep the default medium-effort builder's correctness on harder tasks with less input? The screen compares Kogen's default medium-effort builder with the same model at low effort. Both settings use prose plus Gherkin specifications and four public development tasks, with two repetitions per task and setting.

## Results

| Task | Arm | Correct submissions |
| --- | --- | ---: |
| r70-1-go | sol-medium | 2/2 |
| r70-1-go | sol-low | 2/2 |
| r70-4-elixir-2 | sol-medium | 2/2 |
| r70-4-elixir-2 | sol-low | 2/2 |
| r70-4-go | sol-medium | 2/2 |
| r70-4-go | sol-low | 2/2 |
| r70-6-elixir-2 | sol-medium | 2/2 |
| r70-6-elixir-2 | sol-low | 2/2 |

| Effort setting | Scored attempts | Correct submissions | Wrong submissions | Repairs | Builder input | Reviewer input | Total input | Build elapsed time | Review elapsed time |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| sol-medium (default) | 8 | 8 | 0 | 0 | 0.65M | 1.77M | 2.42M | 1765 s | 2855 s |
| sol-low | 8 | 8 | 0 | 0 | 0.41M | 1.92M | 2.33M | 1441 s | 3088 s |

Both settings scored 8 of 8, with no repairs and no wrong submissions. Builder input fell about 38% (651,878 to 407,155 tokens), and build elapsed time fell from 1765 s to 1441 s. Reviewer input rose from 1.77M to 1.92M, limiting the total input reduction to about 4% (2.42M to 2.33M). Reviewer input was about 73% of total input for sol-medium and about 83% for sol-low.

## Limits

This is a small screen with two repetitions per task. It does not establish that low effort keeps correctness on other tasks or in other runs.

## Required reproduction metadata

- The settings and resolved configurations are in the public bundle. They use the same prose and Gherkin specifications; builder effort differs.
- The task identifiers in the table are public development tasks. Lab hidden-test outcomes are reported; no official benchmark grade was assigned.
- [Records](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/builder-low-1/records.jsonl) contains one Standard 1.2 record per started attempt (16). [Measured fields](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/builder-low-1/MEASURED.md), [declared gaps](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/builder-low-1/MISSING.md), and [missing-field data](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/builder-low-1/MISSING-DECLARED.json) describe record coverage.
- [Public exports](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/rounds/builder-low-1/bundles/) contain the frozen settings and per-attempt outcomes. Candidate code is not included.

## Metadata

Round date: 2026-10-20
Publication badge: DESCRIPTIVE; KEPT FOR AUDIT
Recomputation status: OBSERVED SOURCE ONLY
Why not VALID: DESCRIPTIVE_ONLY, PROTOCOL_DEVIATION, INCOMPLETE_CAPTURE — exploratory screen with two repetitions per task, small counts, approximate execution, and no official grades; Standard capture fields are declared missing.
STATUS: **DESCRIPTIVE**

## Execution record

- Kogen commit: b45ba32bacf26e2b17cb0bb81015ba56d27a4b0b.
- Harness commit: not recorded as a public Git commit; the execution build is identified below.
- Model and effort: Builder gpt-6.1-sol at medium or low effort; reviewer-fixer gpt-6.1-sol at high effort.
- Task IDs: r70-1-go, r70-4-elixir-2, r70-4-go, and r70-6-elixir-2.
- The experiment was run and exported with Kogen Lab build 3bf295ef, set to Kogen commit b45ba32b.
- Command: The experiment was run and exported in Kogen Lab before publication.
- The run scored 16 of 16 attempts. Recorded input usage was 4.76M tokens.
- Raw records and the public bundles are linked above.

## What's missing and why

- `circumstances.cap_end` (16 missing): The Lab export does not carry this Standard field. Boundary telemetry not retained. Reason: Boundary telemetry not retained (`m08`).
- `circumstances.cap_start` (16 missing): The Lab export does not carry this Standard field. Boundary telemetry not retained. Reason: Boundary telemetry not retained (`m08`).
- `circumstances.concurrent_cells_end` (16 missing): The Lab export does not carry this Standard field. Host-wide concurrent cell count was not captured. Reason: Host-wide concurrent cell count was not captured (`m66`).
- `circumstances.concurrent_cells_start` (16 missing): The Lab export does not carry this Standard field. Host-wide concurrent cell count was not captured. Reason: Host-wide concurrent cell count was not captured (`m66`).
- `circumstances.dispatcher_id` (16 missing): The Lab export does not carry this Standard field. Exact dispatcher revision is not present in the public receipt extract. Reason: Exact dispatcher revision is not present in the public receipt extract (`m63`).
- `circumstances.incidents` (16 missing): The Lab export does not carry this Standard field. Per-cell incident receipt was not captured. Reason: Per-cell incident receipt was not captured (`m78`).
- `circumstances.load1_end` (16 missing): The Lab export does not carry this Standard field. Host boundary load was not captured in the public round data. Reason: Host boundary load was not captured in the public round data (`m65`).
- `circumstances.load1_start` (16 missing): The Lab export does not carry this Standard field. Host boundary load was not captured in the public round data. Reason: Host boundary load was not captured in the public round data (`m65`).
- `circumstances.load_samples` (16 missing): The Lab export does not carry this Standard field. Timestamped host-wide load samples were not captured. Reason: Timestamped host-wide load samples were not captured (`m93`).
- `circumstances.queue` (16 missing): The Lab export does not carry this Standard field. Queue receipt was not captured. Reason: Queue receipt was not captured (`m91`).
- `cost.accounting` (16 missing): The Lab export does not carry this Standard field. Per-cell cost and pricing inputs are not present in the public round data. Reason: Per-cell cost and pricing inputs are not present in the public round data (`m70`).
- `cost.calculator_version` (16 missing): The Lab export does not carry this Standard field. Cost calculator version not recorded. Reason: Cost calculator version not recorded (`m61`).
- `cost.long_context_reconciled` (16 missing): The Lab export does not carry this Standard field. Boolean receipt not recorded. Reason: Boolean receipt not recorded (`m07`).
- `cost.price_table_version` (16 missing): The Lab export does not carry this Standard field. Price table version not recorded. Reason: Price table version not recorded (`m89`).
- `cost.usd` (16 missing): The Lab export does not carry this Standard field. Per-cell cost and pricing inputs are not present in the public round data. Reason: Per-cell cost and pricing inputs are not present in the public round data (`m70`).
- `effort.effective` (16 missing): The Lab export does not carry this Standard field. Per-cell effective-effort receipt not published. Reason: Per-cell effective-effort receipt not published (`m73`).
- `effort.requested` (16 missing): The Lab export does not carry this Standard field. Not recorded in available public metadata. Reason: Not recorded in available public metadata (`m39`).
- `effort.runner_requested` (16 missing): The Lab export does not carry this Standard field. Not recorded in available public metadata. Reason: Not recorded in available public metadata (`m39`).
- `environment.account_class` (16 missing): The Lab export does not carry this Standard field. No owner-approved class-only account receipt is available for the cell. Reason: No owner-approved class-only account receipt is available for the cell (`m100`).
- `environment.network.allowlist_hosts` (16 missing): The Lab export does not carry this Standard field. Per-cell network allowlist not present in the public round data. Reason: Per-cell network allowlist not present in the public round data (`m79`).
- `environment.network.profile` (16 missing): The Lab export does not carry this Standard field. Per-cell network profile not present in the public round data. Reason: Per-cell network profile not present in the public round data (`m80`).
- `environment.toolchains.erlang` (16 missing): The Lab export does not carry this Standard field. Toolchain version not recorded. Reason: Toolchain version not recorded (`m95`).
- `environment.toolchains.other_inventory` (16 missing): The Lab export does not carry this Standard field. Full per-cell toolchain inventory not recorded. Reason: Full per-cell toolchain inventory not recorded (`m64`).
- `environment.toolchains.ruby` (16 missing): The Lab export does not carry this Standard field. Toolchain version not recorded. Reason: Toolchain version not recorded (`m95`).
- `environment.toolchains.rust` (16 missing): The Lab export does not carry this Standard field. Toolchain version not recorded. Reason: Toolchain version not recorded (`m95`).
- `grade.grader` (16 missing): The Lab export does not carry this Standard field. Per-cell grader software version is not in the public data. Reason: Per-cell grader software version is not in the public data (`m77`).
- `grade.timestamp` (16 missing): The Lab export does not carry this Standard field. No official grade in the public snapshot. Reason: No official grade in the public snapshot (`m32`).
- `host.id` (16 missing): The Lab export does not carry this Standard field. Not recorded in available public metadata. Reason: Not recorded in available public metadata (`m39`).
- `itt.class` (16 missing): The Lab export does not carry this Standard field. No evidence-backed ITT classification. Reason: No evidence-backed ITT classification (`m28`).
- `itt.cohort` (16 missing): The Lab export does not carry this Standard field. No evidence-backed ITT classification. Reason: No evidence-backed ITT classification (`m28`).
- `outcome` (16 missing): The Lab export does not carry this Standard field. Not available for ungraded delivery. Reason: Not available for ungraded delivery (`m38`).
- `provenance.ledger_sha256` (16 missing): The Lab export does not carry this Standard field. Not recorded in available public metadata. Reason: Not recorded in available public metadata (`m39`).
- `sandbox.egress_allow` (16 missing): The Lab export does not carry this Standard field. Per-cell egress allowlist is not present in the public round data. Reason: Per-cell egress allowlist is not present in the public round data (`m75`).
- `sandbox.egress_profile` (16 missing): The Lab export does not carry this Standard field. Per-cell egress profile is not present in the public round data. Reason: Per-cell egress profile is not present in the public round data (`m76`).
- `sandbox.profile` (16 missing): The Lab export does not carry this Standard field. Per-cell sandbox profile is not present in the public round data. Reason: Per-cell sandbox profile is not present in the public round data (`m87`).
- `sandbox.profile_sha256` (16 missing): The Lab export does not carry this Standard field. Per-cell sandbox profile fingerprint is not present in the public round data. Reason: Per-cell sandbox profile fingerprint is not present in the public round data (`m86`).
- `setup.adapter_harness_sha` (16 missing): The Lab export does not carry this Standard field. Not recorded in available public metadata. Reason: Not recorded in available public metadata (`m39`).
- `setup.deps_source` (16 missing): The Lab export does not carry this Standard field. Per-cell dependency source is not recorded. Reason: Per-cell dependency source is not recorded (`m71`).
- `setup.sandbox_mode` (16 missing): The Lab export does not carry this Standard field. Per-cell sandbox mode is not present in the public round data. Reason: Per-cell sandbox mode is not present in the public round data (`m85`).
- `setup.sandbox_profile_sha256` (16 missing): The Lab export does not carry this Standard field. Per-cell sandbox fingerprint is not present in the public round data. Reason: Per-cell sandbox fingerprint is not present in the public round data (`m84`).
- `setup.task_base.hash` (16 missing): The Lab export does not carry this Standard field. Original base commit/tree hash absent; fresh_base_commit is not a base hash. Reason: Original base commit/tree hash absent; fresh_base_commit is not a base hash (`m42`).
- `setup.task_base.kind` (16 missing): The Lab export does not carry this Standard field. Original base revision type not recorded per cell. Reason: Original base revision type not recorded per cell (`m43`).
- `task.base_repo` (16 missing): The Lab export does not carry this Standard field. Not recorded in available public metadata. Reason: Not recorded in available public metadata (`m39`).
- `task.base_revision.hash` (16 missing): The Lab export does not carry this Standard field. Original base commit/tree hash absent; fresh_base_commit is not a base hash. Reason: Original base commit/tree hash absent; fresh_base_commit is not a base hash (`m42`).
- `task.base_revision.kind` (16 missing): The Lab export does not carry this Standard field. Original base revision type not recorded per cell. Reason: Original base revision type not recorded per cell (`m43`).
- `timestamps.phases.develop.end_utc` (16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (`m88`).
- `timestamps.phases.develop.start_utc` (16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (`m88`).
- `timestamps.phases.gate.end_utc` (16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (`m88`).
- `timestamps.phases.gate.start_utc` (16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (`m88`).
- `timestamps.phases.grade.end_utc` (16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (`m88`).
- `timestamps.phases.grade.start_utc` (16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (`m88`).
- `timestamps.phases.plan.end_utc` (16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (`m88`).
- `timestamps.phases.plan.start_utc` (16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (`m88`).
- `timestamps.phases.review.end_utc` (16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (`m88`).
- `timestamps.phases.review.start_utc` (16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (`m88`).
- `timestamps.phases.setup.end_utc` (16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (`m88`).
- `timestamps.phases.setup.start_utc` (16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (`m88`).
- `timestamps.phases.shape.end_utc` (16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (`m88`).
- `timestamps.phases.shape.start_utc` (16 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (`m88`).
- `timing.phases_s.develop` (16 missing): The Lab export does not carry this Standard field. Phase wall not emitted or not separable. Reason: Phase wall not emitted or not separable (`m45`).
- `timing.phases_s.gate` (16 missing): The Lab export does not carry this Standard field. Phase wall not emitted or not separable. Reason: Phase wall not emitted or not separable (`m45`).
- `timing.phases_s.setup` (16 missing): The Lab export does not carry this Standard field. Phase wall not emitted or not separable. Reason: Phase wall not emitted or not separable (`m45`).
- `timing.timeout_cap_s` (16 missing): The Lab export does not carry this Standard field. Timeout receipt unavailable. Reason: Timeout receipt unavailable (`m53`).
- `tools.grader` (16 missing): The Lab export does not carry this Standard field. Per-cell grader software version is not in the public data. Reason: Per-cell grader software version is not in the public data (`m77`).
- `tools.runner` (16 missing): The Lab export does not carry this Standard field. Per-cell runner version not recorded in the public round data. Reason: Per-cell runner version not recorded in the public round data (`m83`).
- `tools.toolchains.erlang` (16 missing): The Lab export does not carry this Standard field. Toolchain version not recorded. Reason: Toolchain version not recorded (`m95`).
- `tools.toolchains.other_inventory` (16 missing): The Lab export does not carry this Standard field. Full per-cell toolchain inventory not recorded. Reason: Full per-cell toolchain inventory not recorded (`m64`).
- `tools.toolchains.ruby` (16 missing): The Lab export does not carry this Standard field. Toolchain version not recorded. Reason: Toolchain version not recorded (`m95`).
- `tools.toolchains.rust` (16 missing): The Lab export does not carry this Standard field. Toolchain version not recorded. Reason: Toolchain version not recorded (`m95`).


## Captured run records

No records for this round are listed in the [run-record index](/downloads/results/run-records/index.json).


## Reproduction inventory

- **Type:** analysis-only
- **Script:** none published
- **Inputs:** `rounds/builder-low-1/MEASURED.md`, `rounds/builder-low-1/MISSING-DECLARED.json`, `rounds/builder-low-1/MISSING.md`, `rounds/builder-low-1/PLAN.json`, `rounds/builder-low-1/README.md`, `rounds/builder-low-1/bundles/bl1/EXCLUDED.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-1-go/r0/sol-low/a1/attempt.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-1-go/r0/sol-low/a1/evaluation.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-1-go/r0/sol-medium/a1/attempt.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-1-go/r0/sol-medium/a1/evaluation.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-1-go/r1/sol-low/a1/attempt.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-1-go/r1/sol-low/a1/evaluation.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-1-go/r1/sol-medium/a1/attempt.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-1-go/r1/sol-medium/a1/evaluation.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-4-elixir-2/r0/sol-low/a1/attempt.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-4-elixir-2/r0/sol-low/a1/evaluation.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-4-elixir-2/r0/sol-medium/a1/attempt.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-4-elixir-2/r0/sol-medium/a1/evaluation.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-4-elixir-2/r1/sol-low/a1/attempt.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-4-elixir-2/r1/sol-low/a1/evaluation.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-4-elixir-2/r1/sol-medium/a1/attempt.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-4-elixir-2/r1/sol-medium/a1/evaluation.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-4-go/r0/sol-low/a1/attempt.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-4-go/r0/sol-low/a1/evaluation.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-4-go/r0/sol-medium/a1/attempt.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-4-go/r0/sol-medium/a1/evaluation.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-4-go/r1/sol-low/a1/attempt.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-4-go/r1/sol-low/a1/evaluation.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-4-go/r1/sol-medium/a1/attempt.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-4-go/r1/sol-medium/a1/evaluation.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-6-elixir-2/r0/sol-low/a1/attempt.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-6-elixir-2/r0/sol-low/a1/evaluation.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-6-elixir-2/r0/sol-medium/a1/attempt.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-6-elixir-2/r0/sol-medium/a1/evaluation.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-6-elixir-2/r1/sol-low/a1/attempt.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-6-elixir-2/r1/sol-low/a1/evaluation.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-6-elixir-2/r1/sol-medium/a1/attempt.json`, `rounds/builder-low-1/bundles/bl1/attempts/r70-6-elixir-2/r1/sol-medium/a1/evaluation.json`, `rounds/builder-low-1/bundles/bl1/frozen/experiment.toml`, `rounds/builder-low-1/bundles/bl1/frozen/manifest.json`, `rounds/builder-low-1/bundles/bl1/frozen/resolved/sol-low.toml`, `rounds/builder-low-1/bundles/bl1/frozen/resolved/sol-medium.toml`, `rounds/builder-low-1/bundles/bl1/manifest.json`, `rounds/builder-low-1/bundles/bl1/report.md`, `rounds/builder-low-1/records.jsonl`
- **Full execution replay:** unavailable
- **Scope:** Published Standard records and the Lab public exports (frozen configuration, per-attempt outcomes, public test states, stage times and usage) are available; raw conversations, tool output and candidate code are not included.

See the [complete per-round reproduction inventory](/reproduce/) and the [repository reproduction guide](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/README.md).

## Related records

### Related experiments

- [builder-low-1](/rounds/builder-low-1/)
