Round cheap-profile-1
A lower-cost profile kept correctness across three development runs
Source-reported narrativeRound README and results text are source material. Numeric values that are not tied to a claim ledger record remain source-reported.
Round source README · rounds/cheap-profile-1/README.md
A descriptive screen across three runs. Runs 1 and 2 used four public development tasks; run 3 added four more. A rescue means review turned failing code into passing code. The round is approximate: Lab verified the code in the builder's workspace, not in a clean copy of the candidate as Kogen now does.
Question and scope
Does a low-effort builder with a medium-effort reviewer-fixer keep the default profile's correctness at lower total input and the sum of recorded stage times? The two settings used the same prose-plus-scenarios specifications and public development tasks.
Results from runs 1 and 2
| Profile | Scored attempts | Correct submissions | Wrong submissions | Wrong code at review | Reviewer edits on correct code | Total input | Sum of recorded stage times |
|---|---|---|---|---|---|---|---|
| Default | 8 | 8/8 | 0 | 0 | 6 | 2.22M | 2,621 s |
| Lower-cost profile | 8 | 8/8 | 0 | 0 | 8 | 1.32M | 1,521 s |
Both profiles were correct in 8 of 8 attempts. Total input fell about 40.5%. The sum of recorded stage times fell about 42%; the full attempt elapsed time fell about 39%. No wrong code reached review in either profile, so these runs say nothing about the reviewer's ability to rescue wrong code. All 14 attempts with reviewer edits had correct code before review.
Run 3: four more development tasks
The two profiles used one attempt on each of four additional public development tasks.
| Profile | Correct submissions | Wrong submissions | Wrong code at review | Reviewer edits on correct code | Builder input | Reviewer input | Total input | Sum of recorded stage times |
|---|---|---|---|---|---|---|---|---|
| Default | 4 of 4 | 0 | 0 | 0 of 4 (approved unchanged) | 0.24M | 0.18M | 0.41M | 1,143 s |
| Lower-cost profile | 4 of 4 | 0 | 0 | 2 of 4 | 0.14M | 0.19M | 0.33M | 1,031 s |
Both profiles were correct in 4 of 4 attempts. The lower-cost profile used about 20.9% less total input in this run. The lower-cost profile's sum of recorded stage times was about 10% lower. It made two edits to code that was already correct; the default reviewer approved all four correct trees unchanged. There were no builder stops or provider refusals.
Across all three runs, each profile was correct in 12 of 12 attempts, with no wrong submissions and no wrong code at review. Total input was 2.63M for the default and 1.65M for the lower-cost profile, a 37% reduction. The sum of recorded stage times was 3,764 s for the default and 2,552 s for the lower-cost profile, a 32% reduction. No builder stops or provider refusals were recorded in any run. Since no wrong code reached review, these runs provide no rescue evidence.
Limits
This is a descriptive screen on development tasks. Small counts do not establish how the settings will compare on fresh tasks.
Required reproduction metadata
- Frozen settings and resolved configurations are in the public bundles. They identify the role settings and experimental stops.
- Lab hidden-test outcomes are reported here; no official benchmark grade was assigned.
- Records contains one Standard 1.2 record per started attempt (24). Measured fields, declared gaps, and missing-field data describe record coverage.
- Pre-review grades contains 16 post-hoc grades for reviewer-edited attempts. They were computed without model calls on the attempts' own hosts.
- Public bundles contain frozen settings and per-attempt outcomes. Candidate code is not included.
Metadata
Round date: 2026-10-20 Label: DESCRIPTIVE Publication badge: DESCRIPTIVE; KEPT FOR AUDIT Recomputation status: OBSERVED SOURCE ONLY Why not VALID: DESCRIPTIVE_ONLY, PROTOCOL_DEVIATION, INCOMPLETE_CAPTURE — development-task screen, approximate execution, no official grades, and missing Standard capture fields declared below. STATUS: DESCRIPTIVE
Execution record
- Kogen commit: b45ba32bacf26e2b17cb0bb81015ba56d27a4b0b.
- Lab build: 3bf295ef.
- Harness commit: not recorded as a public Git commit; the execution build is identified above.
- Model and effort: The default used a Sol-medium builder and Sol-high reviewer-fixer; the lower-cost profile used a Sol-low builder and Sol-medium reviewer-fixer.
- Task IDs: r70-1-go, r70-4-go, r70-7-elixir, r70-7-go, r70-5-go, r70-5-elixir, rate-go, artifact-go.
- Command: the experiments were run and exported in Kogen Lab before publication.
- Attempts started: 24; all 24 were scored; recorded input was 4.28M.
- Raw records, pre-review grades, and public bundles are linked above.
The list below names what the records cannot carry; reason codes are explained in MISSING-DECLARED.json.
What's missing and why
circumstances.cap_end(24 missing): The Lab export does not carry this Standard field. Boundary telemetry not retained. Reason: Boundary telemetry not retained (m08).circumstances.cap_start(24 missing): The Lab export does not carry this Standard field. Boundary telemetry not retained. Reason: Boundary telemetry not retained (m08).circumstances.concurrent_cells_end(24 missing): The Lab export does not carry this Standard field. Host-wide concurrent cell count was not captured. Reason: Host-wide concurrent cell count was not captured (m66).circumstances.concurrent_cells_start(24 missing): The Lab export does not carry this Standard field. Host-wide concurrent cell count was not captured. Reason: Host-wide concurrent cell count was not captured (m66).circumstances.dispatcher_id(24 missing): The Lab export does not carry this Standard field. Exact dispatcher revision is not present in the public receipt extract. Reason: Exact dispatcher revision is not present in the public receipt extract (m63).circumstances.incidents(24 missing): The Lab export does not carry this Standard field. Per-cell incident receipt was not captured. Reason: Per-cell incident receipt was not captured (m78).circumstances.load1_end(24 missing): The Lab export does not carry this Standard field. Host boundary load was not captured in the public round data. Reason: Host boundary load was not captured in the public round data (m65).circumstances.load1_start(24 missing): The Lab export does not carry this Standard field. Host boundary load was not captured in the public round data. Reason: Host boundary load was not captured in the public round data (m65).circumstances.load_samples(24 missing): The Lab export does not carry this Standard field. Timestamped host-wide load samples were not captured. Reason: Timestamped host-wide load samples were not captured (m93).circumstances.queue(24 missing): The Lab export does not carry this Standard field. Queue receipt was not captured. Reason: Queue receipt was not captured (m91).cost.accounting(24 missing): The Lab export does not carry this Standard field. Per-cell cost and pricing inputs are not present in the public round data. Reason: Per-cell cost and pricing inputs are not present in the public round data (m70).cost.calculator_version(24 missing): The Lab export does not carry this Standard field. Cost calculator version not recorded. Reason: Cost calculator version not recorded (m61).cost.long_context_reconciled(24 missing): The Lab export does not carry this Standard field. Boolean receipt not recorded. Reason: Boolean receipt not recorded (m07).cost.price_table_version(24 missing): The Lab export does not carry this Standard field. Price table version not recorded. Reason: Price table version not recorded (m89).cost.usd(24 missing): The Lab export does not carry this Standard field. Per-cell cost and pricing inputs are not present in the public round data. Reason: Per-cell cost and pricing inputs are not present in the public round data (m70).effort.effective(24 missing): The Lab export does not carry this Standard field. Per-cell effective-effort receipt not published. Reason: Per-cell effective-effort receipt not published (m73).effort.requested(24 missing): The Lab export does not carry this Standard field. Not recorded in available public metadata. Reason: Not recorded in available public metadata (m39).effort.runner_requested(24 missing): The Lab export does not carry this Standard field. Not recorded in available public metadata. Reason: Not recorded in available public metadata (m39).environment.account_class(24 missing): The Lab export does not carry this Standard field. No owner-approved class-only account receipt is available for the cell. Reason: No owner-approved class-only account receipt is available for the cell (m100).environment.network.allowlist_hosts(24 missing): The Lab export does not carry this Standard field. Per-cell network allowlist not present in the public round data. Reason: Per-cell network allowlist not present in the public round data (m79).environment.network.profile(24 missing): The Lab export does not carry this Standard field. Per-cell network profile not present in the public round data. Reason: Per-cell network profile not present in the public round data (m80).environment.toolchains.erlang(24 missing): The Lab export does not carry this Standard field. Toolchain version not recorded. Reason: Toolchain version not recorded (m95).environment.toolchains.other_inventory(24 missing): The Lab export does not carry this Standard field. Full per-cell toolchain inventory not recorded. Reason: Full per-cell toolchain inventory not recorded (m64).environment.toolchains.ruby(24 missing): The Lab export does not carry this Standard field. Toolchain version not recorded. Reason: Toolchain version not recorded (m95).environment.toolchains.rust(24 missing): The Lab export does not carry this Standard field. Toolchain version not recorded. Reason: Toolchain version not recorded (m95).grade.grader(24 missing): The Lab export does not carry this Standard field. Per-cell grader software version is not in the public data. Reason: Per-cell grader software version is not in the public data (m77).grade.timestamp(24 missing): The Lab export does not carry this Standard field. No official grade in the public snapshot. Reason: No official grade in the public snapshot (m32).host.id(24 missing): The Lab export does not carry this Standard field. Not recorded in available public metadata. Reason: Not recorded in available public metadata (m39).itt.class(24 missing): The Lab export does not carry this Standard field. No evidence-backed ITT classification. Reason: No evidence-backed ITT classification (m28).itt.cohort(24 missing): The Lab export does not carry this Standard field. No evidence-backed ITT classification. Reason: No evidence-backed ITT classification (m28).outcome(24 missing): The Lab export does not carry this Standard field. Not available for ungraded delivery. Reason: Not available for ungraded delivery (m38).provenance.ledger_sha256(24 missing): The Lab export does not carry this Standard field. Not recorded in available public metadata. Reason: Not recorded in available public metadata (m39).sandbox.egress_allow(24 missing): The Lab export does not carry this Standard field. Per-cell egress allowlist is not present in the public round data. Reason: Per-cell egress allowlist is not present in the public round data (m75).sandbox.egress_profile(24 missing): The Lab export does not carry this Standard field. Per-cell egress profile is not present in the public round data. Reason: Per-cell egress profile is not present in the public round data (m76).sandbox.profile(24 missing): The Lab export does not carry this Standard field. Per-cell sandbox profile is not present in the public round data. Reason: Per-cell sandbox profile is not present in the public round data (m87).sandbox.profile_sha256(24 missing): The Lab export does not carry this Standard field. Per-cell sandbox profile fingerprint is not present in the public round data. Reason: Per-cell sandbox profile fingerprint is not present in the public round data (m86).setup.adapter_harness_sha(24 missing): The Lab export does not carry this Standard field. Not recorded in available public metadata. Reason: Not recorded in available public metadata (m39).setup.deps_source(24 missing): The Lab export does not carry this Standard field. Per-cell dependency source is not recorded. Reason: Per-cell dependency source is not recorded (m71).setup.sandbox_mode(24 missing): The Lab export does not carry this Standard field. Per-cell sandbox mode is not present in the public round data. Reason: Per-cell sandbox mode is not present in the public round data (m85).setup.sandbox_profile_sha256(24 missing): The Lab export does not carry this Standard field. Per-cell sandbox fingerprint is not present in the public round data. Reason: Per-cell sandbox fingerprint is not present in the public round data (m84).setup.task_base.hash(24 missing): The Lab export does not carry this Standard field. Original base commit/tree hash absent; fresh_base_commit is not a base hash. Reason: Original base commit/tree hash absent; fresh_base_commit is not a base hash (m42).setup.task_base.kind(24 missing): The Lab export does not carry this Standard field. Original base revision type not recorded per cell. Reason: Original base revision type not recorded per cell (m43).task.base_repo(24 missing): The Lab export does not carry this Standard field. Not recorded in available public metadata. Reason: Not recorded in available public metadata (m39).task.base_revision.hash(24 missing): The Lab export does not carry this Standard field. Original base commit/tree hash absent; fresh_base_commit is not a base hash. Reason: Original base commit/tree hash absent; fresh_base_commit is not a base hash (m42).task.base_revision.kind(24 missing): The Lab export does not carry this Standard field. Original base revision type not recorded per cell. Reason: Original base revision type not recorded per cell (m43).timestamps.phases.develop.end_utc(24 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.develop.start_utc(24 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.gate.end_utc(24 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.gate.start_utc(24 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.grade.end_utc(24 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.grade.start_utc(24 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.plan.end_utc(24 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.plan.start_utc(24 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.review.end_utc(24 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.review.start_utc(24 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.setup.end_utc(24 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.setup.start_utc(24 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.shape.end_utc(24 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timestamps.phases.shape.start_utc(24 missing): The Lab export does not carry this Standard field. Phase timestamp not present in the public per-cell data. Reason: Phase timestamp not present in the public per-cell data (m88).timing.phases_s.gate(24 missing): The Lab export does not carry this Standard field. Phase wall not emitted or not separable. Reason: Phase wall not emitted or not separable (m45).timing.phases_s.setup(24 missing): The Lab export does not carry this Standard field. Phase wall not emitted or not separable. Reason: Phase wall not emitted or not separable (m45).timing.timeout_cap_s(24 missing): The Lab export does not carry this Standard field. Timeout receipt unavailable. Reason: Timeout receipt unavailable (m53).tools.grader(24 missing): The Lab export does not carry this Standard field. Per-cell grader software version is not in the public data. Reason: Per-cell grader software version is not in the public data (m77).tools.runner(24 missing): The Lab export does not carry this Standard field. Per-cell runner version not recorded in the public round data. Reason: Per-cell runner version not recorded in the public round data (m83).tools.toolchains.erlang(24 missing): The Lab export does not carry this Standard field. Toolchain version not recorded. Reason: Toolchain version not recorded (m95).tools.toolchains.other_inventory(24 missing): The Lab export does not carry this Standard field. Full per-cell toolchain inventory not recorded. Reason: Full per-cell toolchain inventory not recorded (m64).tools.toolchains.ruby(24 missing): The Lab export does not carry this Standard field. Toolchain version not recorded. Reason: Toolchain version not recorded (m95).tools.toolchains.rust(24 missing): The Lab export does not carry this Standard field. Toolchain version not recorded. Reason: Toolchain version not recorded (m95).
Captured run records
Standard-schema records assigned to this round in the indexed snapshot.
No per-round file is listed. Check the run-record index for the complete partition list.
Reproduction inventory
Published analysis replay and full experiment replay are reported separately.
- Replay type
analysis-only- Script
- No per-round analysis script is published.
- Inputs
rounds/cheap-profile-1/MEASURED.mdrounds/cheap-profile-1/MISSING-DECLARED.jsonrounds/cheap-profile-1/MISSING.mdrounds/cheap-profile-1/PLAN.jsonrounds/cheap-profile-1/PRE-REVIEW-GRADES.jsonrounds/cheap-profile-1/README.mdrounds/cheap-profile-1/bundles/cp1/EXCLUDED.jsonrounds/cheap-profile-1/bundles/cp1/attempts/r70-1-go/r0/cheap/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp1/attempts/r70-1-go/r0/cheap/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp1/attempts/r70-1-go/r0/kogen-default/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp1/attempts/r70-1-go/r0/kogen-default/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp1/attempts/r70-4-go/r0/cheap/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp1/attempts/r70-4-go/r0/cheap/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp1/attempts/r70-4-go/r0/kogen-default/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp1/attempts/r70-4-go/r0/kogen-default/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp1/attempts/r70-7-elixir/r0/cheap/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp1/attempts/r70-7-elixir/r0/cheap/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp1/attempts/r70-7-elixir/r0/kogen-default/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp1/attempts/r70-7-elixir/r0/kogen-default/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp1/attempts/r70-7-go/r0/cheap/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp1/attempts/r70-7-go/r0/cheap/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp1/attempts/r70-7-go/r0/kogen-default/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp1/attempts/r70-7-go/r0/kogen-default/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp1/frozen/experiment.tomlrounds/cheap-profile-1/bundles/cp1/frozen/manifest.jsonrounds/cheap-profile-1/bundles/cp1/frozen/resolved/cheap.tomlrounds/cheap-profile-1/bundles/cp1/frozen/resolved/kogen-default.tomlrounds/cheap-profile-1/bundles/cp1/manifest.jsonrounds/cheap-profile-1/bundles/cp1/report.mdrounds/cheap-profile-1/bundles/cp1b/EXCLUDED.jsonrounds/cheap-profile-1/bundles/cp1b/attempts/r70-1-go/r0/cheap/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp1b/attempts/r70-1-go/r0/cheap/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp1b/attempts/r70-1-go/r0/kogen-default/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp1b/attempts/r70-1-go/r0/kogen-default/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp1b/attempts/r70-4-go/r0/cheap/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp1b/attempts/r70-4-go/r0/cheap/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp1b/attempts/r70-4-go/r0/kogen-default/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp1b/attempts/r70-4-go/r0/kogen-default/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp1b/attempts/r70-7-elixir/r0/cheap/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp1b/attempts/r70-7-elixir/r0/cheap/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp1b/attempts/r70-7-elixir/r0/kogen-default/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp1b/attempts/r70-7-elixir/r0/kogen-default/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp1b/attempts/r70-7-go/r0/cheap/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp1b/attempts/r70-7-go/r0/cheap/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp1b/attempts/r70-7-go/r0/kogen-default/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp1b/attempts/r70-7-go/r0/kogen-default/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp1b/frozen/experiment.tomlrounds/cheap-profile-1/bundles/cp1b/frozen/manifest.jsonrounds/cheap-profile-1/bundles/cp1b/frozen/resolved/cheap.tomlrounds/cheap-profile-1/bundles/cp1b/frozen/resolved/kogen-default.tomlrounds/cheap-profile-1/bundles/cp1b/manifest.jsonrounds/cheap-profile-1/bundles/cp1b/report.mdrounds/cheap-profile-1/bundles/cp2/EXCLUDED.jsonrounds/cheap-profile-1/bundles/cp2/attempts/artifact-go/r0/cheap/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp2/attempts/artifact-go/r0/cheap/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp2/attempts/artifact-go/r0/kogen-default/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp2/attempts/artifact-go/r0/kogen-default/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp2/attempts/r70-5-elixir/r0/cheap/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp2/attempts/r70-5-elixir/r0/cheap/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp2/attempts/r70-5-elixir/r0/kogen-default/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp2/attempts/r70-5-elixir/r0/kogen-default/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp2/attempts/r70-5-go/r0/cheap/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp2/attempts/r70-5-go/r0/cheap/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp2/attempts/r70-5-go/r0/kogen-default/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp2/attempts/r70-5-go/r0/kogen-default/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp2/attempts/rate-go/r0/cheap/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp2/attempts/rate-go/r0/cheap/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp2/attempts/rate-go/r0/kogen-default/a1/attempt.jsonrounds/cheap-profile-1/bundles/cp2/attempts/rate-go/r0/kogen-default/a1/evaluation.jsonrounds/cheap-profile-1/bundles/cp2/frozen/experiment.tomlrounds/cheap-profile-1/bundles/cp2/frozen/manifest.jsonrounds/cheap-profile-1/bundles/cp2/frozen/resolved/cheap.tomlrounds/cheap-profile-1/bundles/cp2/frozen/resolved/kogen-default.tomlrounds/cheap-profile-1/bundles/cp2/manifest.jsonrounds/cheap-profile-1/bundles/cp2/report.mdrounds/cheap-profile-1/records.jsonl
- Full execution replay
- Unavailable
- Scope
- Published bundles and records preserve observed Lab outcomes and configuration; model execution cannot be replayed from this repository snapshot.
View the complete per-round reproduction inventory · Repository reproduction guide