> Data revision: baf7c2f4307d3a989a8fb9363df8000fda3d9310
> Source files:
- [reproduce/kontext-packet-only-localization-api-20261104/README.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/kontext-packet-only-localization-api-20261104/README.md)
- [categories/models-and-effort.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/categories/models-and-effort.md)
- [site/content/finding-pages.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/site/content/finding-pages.json)

# Packet-only localization over the API

- **Finding id:** `kontext-packet-only-localization-api-20261104`
- **Verdict:** Unresolved
- **Finished:** 4 November 2026
- **Comparison:** [Kogen Context component](/comparisons/kogen-context/)
- **Reproduction pack:** [reproduce/kontext-packet-only-localization-api-20261104/](https://github.com/KogenAI/kogen-bench/tree/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/kontext-packet-only-localization-api-20261104) (data, analysis script and rerun steps)
- **Category page:** [categories/models-and-effort.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/categories/models-and-effort.md)

## In plain terms

With the Kogen Context packet a no-tools model named the right first file in all 7 scored calls versus 4 of 8 without it. It is a small pilot of localization only, so the result is unresolved.

Verdict key: **Yes**, The result supports the change or claim that was tested. **No**, The result does not support it, so nothing changes. **Unresolved**, The evidence is not strong enough either way. **Superseded**, A later test replaced this result.

## Full text of the reproduction pack

The text below is the pack's README as published in the repository (a system folder name may show without its leading slash, so that the site's privacy scan passes).

**Verdict: unresolved.** One claude-sonnet-5-5 call per cell (no tools, no repository access) asked for up to 6 ranked files to edit, with the issue and bounded failure output (arm A) or the same plus the frozen kogen-context evidence packet (arm B). Four decision cases, 2 repetitions per arm. This measures where the packet helps localization only; it does not run a build and says nothing about verified outcomes.

| Arm | scored calls | mean recall of reference files | mean precision | first file correct | mean input tokens | list cost |
|---|---|---|---|---|---|---|
| A, no packet | 8 | 0.688 | 0.348 | 4/8 | 2169 | USD 0.0418 |
| B, packet | 7 | 0.857 | 0.39 | 7/7 | 3827 | USD 0.0675 |

Per case (recall per repetition, rank of the first correct file):

| case | arm | recall | first-hit rank |
|---|---|---|---|
| rg-ignore | A | 1.00, 0.00 | 3, None |
| rg-ignore | B | 1.00, 1.00 | 1, 1 |
| ripgrep | A | 0.50, 0.50 | 2, 2 |
| ripgrep | B | 0.50, parse error | 1 |
| gin | A | 0.75, 0.75 | 1, 1 |
| gin | B | 0.75, 0.75 | 1, 1 |
| axios-timeout | A | 1.00, 1.00 | 1, 1 |
| axios-timeout | B | 1.00, 1.00 | 1, 1 |

The packet moved the first-hit rank to 1 in every scored B call (A: 4 of 8), mostly by fixing rg-ignore repetition 2 and ripgrep, and left gin and axios-timeout unchanged. One B call (ripgrep, repetition 2) returned unparseable JSON and is excluded from B's scored calls (7 of 8). With 4 cases and 2 repetitions this is a descriptive pilot; reference files include documentation files that neither arm can be expected to name (ripgrep CHANGELOG, gin README). The failure text is the case's red diagnostic, bounded to 4,000 bytes.

## Source and runtime
Frozen binary kogen-context-build-evidence 0.3.0-failure-evidence, SHA-256 59ee239c2abf96b1737cc809063136fd3334df368ff0637a8b1e6aac7b69f747; packet flags `--deadline 1s --bytes 6500 --ranges 6`; cases and plan from kontext-matched-1. claude-sonnet-5-5, effort medium, Anthropic API. Total 16 calls, USD 0.12 (cap USD 5).

## Rerun
`python3 -I report.py data/results.jsonl` reproduces the table from the published rows. A fresh run needs the case kits and the frozen binary (see the kontext-matched-1 plan): `python3 -I localize_api.py KITDIR OUT.jsonl`.
