> Data revision: baf7c2f4307d3a989a8fb9363df8000fda3d9310
> Source files:
- [reproduce/lab-k-verify-isolated-s1-20261105/README.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/lab-k-verify-isolated-s1-20261105/README.md)
- [categories/harness-levers.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/categories/harness-levers.md)
- [site/content/finding-pages.json](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/site/content/finding-pages.json)

# Isolated, parallel-safe generated acceptance: no detector with a payoff passes

- **Finding id:** `lab-k-verify-isolated-s1-20261105`
- **Verdict:** No
- **Finished:** 5 November 2026
- **Comparison:** [Kogen verification component](/comparisons/kogen-verification/)
- **Reproduction pack:** [reproduce/lab-k-verify-isolated-s1-20261105/](https://github.com/KogenAI/kogen-bench/tree/baf7c2f4307d3a989a8fb9363df8000fda3d9310/reproduce/lab-k-verify-isolated-s1-20261105) (data, analysis script and rerun steps)
- **Category page:** [categories/harness-levers.md](https://github.com/KogenAI/kogen-bench/blob/baf7c2f4307d3a989a8fb9363df8000fda3d9310/categories/harness-levers.md)

## In plain terms

Tested three detectors on Kogen's own 604-scenario acceptance with 14 seeded hazards run and 40 scanned. A static lint refuses none of 509 real files but the real corpus holds no port or path hazard; a measured-wall check falsely refuses 10 of 28 sampled Intents at 10 s; two concurrent shuffled copies catch collisions when they overlap but add a median 14% wall. The first full suite run was stopped at 81 minutes and delivery Builds shared the host, so the numbers are a sample.

Verdict key: **Yes**, The result supports the change or claim that was tested. **No**, The result does not support it, so nothing changes. **Unresolved**, The evidence is not strong enough either way. **Superseded**, A later test replaced this result.

## Full text of the reproduction pack

The text below is the pack's README as published in the repository (a system folder name may show without its leading slash, so that the site's privacy scan passes).

**Finding.** Of three detectors for tests that collide, flake or wait when run together, none that would pay is proven. The static lint V1' (fixed listen port, fixed absolute-path write) refuses 0 of 509 real step and feature files and finds all 20 seeded port and path hazards, but the real corpus holds no port or path hazard, so it has no payoff. The measured-wall detector V4 catches seeded fixed waits of 11 to 13 s but falsely refuses 10 of 28 sampled real Intents at a 10 s budget (and 2 even at 30 s), because honest scenarios build Go drivers, run installed-CLI journeys and wait on real handlers. Two concurrent shuffled copies (V2) catch collisions whenever the two copies' hazard windows overlap, add a median 14.1% wall on single-Intent suites (above the 10% line), and its 10-repeat stability could not be shown in the window. The row is closed as low priority; the consumer-golden lesson of Build 49b36dac stays for the later Build.

## Question
Does a model-free detector refuse generated acceptance that is not isolated or parallel-safe (fixed port, fixed absolute path, shared mutable state, long fixed wait) without falsely refusing isolated, fast tests? Registered in Studio Lab lever lab-k-verify-tests-isolated-parallel-safe (revisions r3, r4, r5, each written before the measurement it governs). Gates: recall >= 95% on seeded hazards of the arm's designed classes, 0 false refusals on the real negative set, added wall <= 10% of readiness time, V2 stable in 10/10 repeats, smallest arm wins (V1' before V4 before V2). A REQUEST is written only for a proven arm whose designed classes have a confirmed real occurrence.

## Method (model-free, no seats, no Kogen or Studio commit)
- Trees: kogen [24136fca](https://github.com/KogenAI/kogen/commit/24136fca70dc16fe2f97907cecd924c4a403470b) (147 Intents, 604 scenarios; the EU mirror lagged one commit, so main was fetched from GitHub) and studio [4302e2c8](https://github.com/KogenAI/studio/commit/4302e2c86df46294192c339633a0d8de027be0d5) (static census only; no Postgres on the host). Release archive built in scratch with packaging/package.sh from the kogen tree.
- Suite: the readiness acceptance command of kogen's .kogen/project.yaml (`cucumber-js --strict --require features/steps/*.js --format message:<report> <features>`), serial, defined order, from a repo copy on a 2-core Linux host, user lanes. V4 reads the per-scenario wall (testCaseFinished minus testCaseStarted) from that run's own message report. V2 runs two repo copies with separate TMPDIR at the same time with `--order random:101` and `random:202` and refuses a scenario that passed in the defined-order run but fails in either copy. V1' is a regex lint over step and feature text.
- Seeded set: one hazard per mutant added as extra scenario(s) plus a lab step file; port (fixed listen, held 2 s), path (non-recursive mkdir of a fixed /tmp directory, held 2 s), shared state (a setter and two readers of a module-level token), sleep (11, 12, 13, 15, 18, 20, 25, 30, 40, 45 s). 40 mutants were generated and scanned by V1'; the dynamic arms ran 14 of them (4 port, 4 path, 3 shared, 3 sleep, classes interleaved), because the host was shared with delivery Builds and each run waited for a quiet host.
- Deviations (all registered before the run they affect, see the Lab record): the first full serial run (604 scenarios) was stopped after 81 min with 30 of 147 Intents done, because go-core-migration alone ran 4,032 s (16 scenarios, longest 1,657 s; it re-runs the suite inside a scenario); the dynamic real-corpus sample is therefore the 29 other finished Intents (284 scenarios, 785.7 s of scenario wall), 22 of them red on this host (stale drivers fail to compile on main); no second repeat, no full-suite V2 pair; mutant hosts were the 6 green Intents of that sample (hosts repeat); a delivery Build ran on the host during parts of the first run (sampler log in data/sampler.log: foreign go, node or test processes appeared in 15 of 237 samples during the first run, so its walls were mostly quiet; during the mutant runs delivery shells were present in most samples but every start waited for measured CPU below 35% on two 6 s looks, and a mutant's serial run and its pair ran back to back after one such check).

## Results
| arm (designed classes) | recall per class (mutants run / 10 per class) | false refusals on the real corpus | added wall | stability | gate verdict |
|---|---|---|---|---|---|
| V1' (port, path) | port 10/10, path 10/10 (all 40 mutants scanned; shared 0/10, sleep 0/10 by design; recall is the lint finding the text the mutation inserted) | 0 of 509 files (287 kogen + 222 studio step and feature files) | none (text scan) | n/a | passes its gates; no payoff (0 real port or path hazards in 258 Intents) |
| V4 (long fixed wait, B = 10 s) | sleep 3/3 run at B=10 and B=5; 0/3 at B=20 and 30 (waits run: 11, 12, 13 s); port, path, shared 0 | 10 of 28 sampled Intents falsely refused at B=10 (14 at 5, 3 at 20, 2 at 30); 3 of the 7 green ones | none (report parse) | one repeat only | FAILS 0 false refusals |
| V2 (port, path, shared) | port 4/4, path 2/4 (2/2 where the two copies' hazard windows overlapped), shared 3/3; sleep 0/3 | 0 other scenarios failed in 14 pairs; no full-suite pair run | median 14.1% (2.4 to 62.8%) on single-Intent suites; None% on the slice | 0 of 10 repeats | FAILS (added wall above 10%, recall n incomplete, stability not shown) |

- V4 on the sample (budget sweep, Intents flagged, of which false): 5 s: 15 (14); 10 s: 11 (10); 20 s: 3 (3); 30 s: 2 (2). The known 11 s wait Intent deterministic-check-gate (scenario wall 15.8 s) is flagged at 5 and 10 s and missed at 20 and 30 s; the 45 s wait Intent k-repin-adapt-f2e30a4 was not in the sample. Slowest honest scenarios: claude-code-full-lineup 82 s, k-accept-wait-handlers 31 s (a scenario about waits), k-acceptance-decision-check-answer 27 s, k-accept-drop-stale-drivers 17 s.
- V2 stability slice: 0 V2 repeats completed on the slice (the serial slice run finished, 189.2 s; the first pair was still waiting for a quiet host at the stop); stability not shown
- V2 caught 4/4 port collisions and 2/2 path collisions where the two copies' 2 s hazard windows overlapped; the 2 path misses had no overlap, so V2 recall depends on the suite's length and order, not on the hazard. Shared state was caught 3/3 (order reversal in one of the two shuffles).
- Limits: 14 of 40 dynamic mutants, a 29-Intent real sample, one V4 repeat, V2 stability not shown, Studio's suite not executed, no fresh 30-suite batch (needs a model), run times shared with a delivery host; the Stage 0 recorded causes (OAuth callback port, a too-short fixed deadline) were not re-created (positive control not run).

## What this decides
- No REQUEST: no arm with a payoff is proven. V1' is proven on its registered gates but its classes have no real occurrence; V4 and V2 fail theirs.
- If a lint is wanted anyway, V1' (listen on a fixed number, write to a fixed absolute path) is free and refused nothing real in 509 files. A wait rule is better taken from measured wall only above a budget set per suite, and this sample shows no single budget separating honest long scenarios from waits.
- Stage 0 numbers stand: 2 of 258 landed Intents (0.8%) carried a confirmed wait hazard; both are fixed.

## Rerun
```
cd reproduce/lab-k-verify-isolated-s1-20261105
python3 -I report.py data/runs data/mutants metrics.json
```
`data/runs/*.scen.json` hold one parsed Cucumber message report per run (scenario, wall, status); `data/mutants` the generated hazard step files and feature additions; `harness/` the driver (`orch.py`, `runsuite.sh`, `msg.py`, `v1prime.py`, `env.sh`) to repeat the runs on a host with the kogen tree at the commit above (build the archive with `sh packaging/package.sh`, set LABW). Raw message reports are not kept. In the pack the loopback address in the mutant step files and in error text is written as `localhost` and e-mail-like strings as `<email>` (publication scan); the runs used the numeric loopback address.
