> Data revision: baf7c2f4307d3a989a8fb9363df8000fda3d9310
> Source files:
- [reproduce/api-patch-screen-2nd-host/README.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/api-patch-screen-2nd-host/README.md)
- [categories/head-to-head.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/categories/head-to-head.md)
- [site/content/finding-pages.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/site/content/finding-pages.json)

# Three speed patches for kogen-loop: none faster or cheaper beyond noise

- **Finding id:** `api-patch-screen-2nd-host`
- **Verdict:** No
- **Finished:** 30 October 2026
- **Comparison:** [kogen-loop speed, patches and connection routes](/comparisons/loop-speed-and-routes/)
- **Reproduction pack:** [reproduce/api-patch-screen-2nd-host/](https://github.com/KogenAI/kogen-bench/tree/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/api-patch-screen-2nd-host) (data, analysis script and rerun steps)
- **Category page:** [categories/head-to-head.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/categories/head-to-head.md)

## In plain terms

A short screening test of three changes to kogen-loop against its faster baseline build, run on a second computer with both sides on the same API key. All 16 runs passed, and no patch was faster or cheaper by more than run-to-run noise, so none was adopted. Total spend was $1.76 against a $3.00 limit.

Verdict key: **Yes**, The result supports the change or claim that was tested. **No**, The result does not support it, so nothing changes. **Unresolved**, The evidence is not strong enough either way. **Superseded**, A later test replaced this result.

## Full text of the reproduction pack

The text below is the pack's README as published in the repository (a system folder name may show without its leading slash, so that the site's privacy scan passes).

A short test of three loop patches against the `speed2` loop baseline on a second host, using the clean API route (both sides on one OpenAI API key, no subscription). All 16 runs passed and no patch was faster or cheaper beyond noise. Verdict: **no change** (nothing promoted), 30 October 2026. Total spend $1.76 of a $3.00 cap.

## What was measured

Four tasks, one run per tool per task, 16 counted runs, one run at a time on one Linux laptop, in a random order fixed in advance (seed `20261030`; the task order and the tool order within each task were shuffled from that seed). Model `gpt-6.1-terra`, effort high, shipped loop instructions, the common Responses endpoint through a gateway with zero upstream retries and the default service tier, an offline Go standard-library cache restored before every run. A run counts only if the pass/fail grade exists, every response reports usage, and every response's returned model is `gpt-6.1-terra`. For each patch, the first run also had to show a witness receipt that the patch's code path was active (for example, that the `finish` tool was omitted from the default terminal tools for `api-answer-end`); all 12 witnesses verified.

## Tools

| Tool | What it is | Model | Effort | Patch |
|---|---|---|---|---|
| `speed2` | loop build b8a4046 plus the speed2 variant (baseline for this test) | gpt-6.1-terra | high | [patches/speed2.diff](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/api-patch-screen-2nd-host/patches/speed2.diff) |
| `api-replay` | baseline plus native Responses encrypted-reasoning replay | gpt-6.1-terra | high | [patches/api-replay.diff](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/api-patch-screen-2nd-host/patches/api-replay.diff) |
| `api-batch` | baseline plus a batched-independent-I/O instruction | gpt-6.1-terra | high | [patches/api-batch.diff](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/api-patch-screen-2nd-host/patches/api-batch.diff) |
| `api-answer-end` | baseline plus finish tool omitted from the default terminal tools | gpt-6.1-terra | high | [patches/api-answer-end.diff](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/api-patch-screen-2nd-host/patches/api-answer-end.diff) |

Each `.diff.sha256` file holds the checksum of the patch as run; the binaries were attested by hash in the protocol file before the first run. Harness: the `lanes/` bare-agent release (`lanes_bare.py` sha256 `9744637d…`) with the round driver in [`lanes/apiscreen-patches-high-20261030/`](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/lanes/apiscreen-patches-high-20261030/); task ids in [tasks.txt](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/api-patch-screen-2nd-host/tasks.txt) are directories under `tasks/`.

## Rerun

The driver that ran the four patched tools and its frozen protocol are in [`lanes/apiscreen-patches-high-20261030/`](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/lanes/apiscreen-patches-high-20261030/) (`run_confirmation.py`, `confirmation_ext.py`, `protocol-frozen.json`, README with the setup). Build the four tools from kogen-loop `b8a4046` with [patches/speed2.diff](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/api-patch-screen-2nd-host/patches/speed2.diff) (the baseline) and, for each candidate, its own diff on top; then, with your own task kits (`tasks.txt`), an OpenAI API key and the variables the README names:

```sh
python3 -I lanes/apiscreen-patches-high-20261030/run_confirmation.py prepare
python3 -I lanes/apiscreen-patches-high-20261030/run_confirmation.py counted 0
```

`prepare` freezes the 16-cell order from seed `20261030`, and `counted N` runs cell N (0 to 15, in order) and grades it. The task kits (start trees, hidden tests and scenarios) stay private until the Kogen hill-climb ends; no date is set. A rerun also needs Linux with Bubblewrap, Go 1.27.2 and the host lease file the driver checks. Checked 31 October 2026 with a stand-in round directory and no model call: the files compile and `--help` prints the modes; `prepare` and the cells were not run. The original round cost about $1.5 at list price; wall times depend on the host and the provider on the day, so fresh values will differ.

## Reproduce the numbers

```sh
python3 -I reproduce/api-patch-screen-2nd-host/analyze.py | diff - reproduce/api-patch-screen-2nd-host/expected-output.txt
```

Verified 30 October 2026: this reproduces every published number exactly (arm medians, all twelve paired deltas, counted cost $1.52016145 and total $1.76347695).

## Files

- `runs.csv`: the 16 counted runs: task, tool, rep, outcome, passed, input tokens (uncached + cached), cached tokens, output tokens, requests, list-price cost in USD, agent wall in seconds, the returned model, whether all responses matched it, whether usage was complete, and whether the patch witness verified (`n/a` for the baseline). Reasoning tokens are not recorded separately in this source and are blank. Wall is agent-process time.
- `excluded.csv`: the charged attempts that are not in the counted rows (zero-call setup error, a witness-less first `api-batch` run, and two runs lost in a host reboot) so the cost denominator is complete.
- `patches/`: the four loop variants and their checksums.
- `analyze.py`, `expected-output.txt`.

## Expected output

| Tool | Passes | Median wall (s) | Median USD per pass |
|---|---:|---:|---:|
| speed2 | 4/4 | 142.442 | $0.066458 |
| api-replay | 4/4 | 166.319 | $0.082623 |
| api-batch | 4/4 | 143.939 | $0.065811 |
| api-answer-end | 4/4 | 173.418 | $0.080588 |

Task-paired deltas are candidate minus `speed2`; negative means faster or cheaper:

| Task | api-replay wall / USD | api-batch wall / USD | api-answer-end wall / USD |
|---|---|---|---|
| cron-go | +29.317 s / +$0.021549 | +1.335 s / −$0.007428 | +2.679 s / −$0.000971 |
| vm-go | +18.437 s / +$0.010782 | +1.658 s / +$0.006135 | +59.272 s / +$0.029232 |
| artifact-go | −5.411 s / −$0.001504 | −15.056 s / −$0.003174 | −2.384 s / −$0.001071 |
| readtime-go | −79.898 s / −$0.085844 | −48.850 s / −$0.038080 | −38.259 s / −$0.016653 |

## Cost

Counted runs $1.52016145; the charged-but-excluded attempts add $0.24331550, of which $0.17033925 is the two reboot-lost runs (not itemised in `runs.csv`; the figure is the sum of two spend-ledger rows), for a total of $1.76347695.

## Limits

One run per tool per task: four task clusters and four passing runs per tool cannot show a winner,. The deltas are dominated by `readtime-go` (large tails) and by one `vm-go` run. The reboot-lost charges are reported from the spend ledger, not from per-run rows. An earlier 16-run screen on this host (loop against `speed2`, 4 tasks, two repetitions) is described in the finding's source note but its per-run wall times were not available when this pack was built (only its token and cost rows are in the spend ledger), so it is not part of this pack. No Codex arm was run.
