F05

F05 — Context, Jev and cache

The authored family record is reproduced below. Structured links connect its hypotheses, rounds, and claim-ledger rows.

Source-reported narrativeNumeric statements in this family document remain source-reported unless an exact claim-ledger row is linked and its status allows that interpretation.

Family source document · hypotheses/f05-context-cache-jev.md

F05 — Context, Jev and cache

Coverage: H51–H61. Context diagnostics, offline routing, request telemetry, and live task outcomes use different analysis units. The Wave B claim ledger is linked at `results/claim-ledger.jsonl`. Unreconciled numeric observations are explicitly marked source-reported, not reproducible from public data.

Hypotheses

ID Falsifiable proposition Current state
H51 Cache reuse, lean prompts, or context management reduces Kogen loop tokens or cost more than other tested runtime changes while preserving task success. UNRESOLVED; historical cache summaries use incompatible definitions and epochs.
H52 A stable cache key/session affinity and stable prompt prefix raise the measured live hit rate to the declared gate on repeated requests. NOT ESTABLISHED; the public smoke did not meet the target.
H53 Cached, uncached, and input-token definitions can be reconciled across Kogen, Codex, and language arms to support a fair cost comparison. NOT ESTABLISHED; no public compatible accounting table.
H54 A Jev classifier selects useful transcript lines for compaction with lower latency and no material loss of task-relevant context. UNRESOLVED; available trace records report no actual compactions.
H55 Deterministic retrieval plus Jev and Luna readers produces more useful source-grounded packets for a stronger worker at lower total cost than the baseline. UNRESOLVED; source screens are not joined to public Build outcomes.
H56 A bounded packet of code relationships rescues failures beyond what file retrieval alone achieves on matched cases. NOT ESTABLISHED; no public paired rescue test.
H57 Prepared task-specific context improves Jev routing/classification accuracy over a bare-context baseline and improves downstream Build decisions. NOT ESTABLISHED; offline diagnostics do not measure Build efficacy.
H58 At least one Jev role among audit, routing, classification, review, compaction, or context preparation improves a predeclared outcome at acceptable cost. UNRESOLVED; role inventory is incomplete and no general role winner is measured.
H59 Passing a planner’s context to the builder reduces builder tokens without reducing matched-task success. UNRESOLVED; the public R54 receipt is interim and does not establish token savings.
H60 Runtime code inspection yields more useful task evidence than static retrieval on a fixed checkpoint and improves task outcomes. NOT ESTABLISHED; Tidewave data is a partial diagnostic, not a live comparative result.
H61 Language comparisons hold cache accounting, prompt, and human steering constant well enough to attribute outcome differences to language. NOT ESTABLISHED; no reconciled public control table.

Experiment register

experiment_id Arm / task set Model / effort Revision; host / epoch Planned / started / finished / graded / ITT n Outcome, status, correction or supersession Claim ID Public receipt
x-context Context-packet and routing summaries; exact arm/task manifest is not recovered. Model/effort alignment not recovered. Not recovered. Lifecycle fields are null in the ledger; no exact public task/cell join. INTERIM; inconclusive source summary only, not a live context benefit. CL-WA-x-context-reported-x-context; CL-WA-x-context-reported-x-context-admission-guard (INTERIM). round, reported data, claim ledger
x-compaction Transcript trace census, peak-input observations, and elision review are separate analyses. kh, Codex, and Pi trace arms are reported; no matched task success arm. Exact trace files and comparable runtime epoch are not in the public tree. Source-reported trace census only; no joined Build cells. INTERIM; the ledger’s reported compacted count is zero, so no compaction efficacy comparison occurred. CL-WA-x-compaction-reported-x-compaction-traces; CL-WA-x-compaction-reported-x-compaction-peak-input; CL-WA-x-compaction-reported-x-compaction-elision-review; CL-WA-x-compaction-reported-e06-context-pilot-link (INTERIM). round, reported data, claim ledger
e06-context One context pilot; no adjacent matched comparison. See the receipt; no matched builder arm. Source and exact model-request receipt are incomplete. A single reported pilot; live Build comparison count is zero. INTERIM; context/compaction diagnostic only. CL-WA-e06-context-reported-e06-context-pilot (INTERIM). round, reported data, claim ledger
e09-tidewave Offline locator gates and a separate partial runtime-tool screen; keep protocols separate. Model dispatches were not part of the locator gate; runtime-tool request settings are on the receipt. Exact data/source rows are not included in the public tree. The runtime screen stopped before its planned population; no matched end-to-end success cohort. INTERIM; no locator or Tidewave efficacy claim. CL-WA-e09-tidewave-reported-e09-locator-gates; CL-WA-e09-tidewave-reported-e09-tidewave-runtime (INTERIM). round, reported data, claim ledger
r54 Pack-plan handoff versus plain pack on five listed tasks; exported task-by-host-arm rows are on the receipt. Exact model/effort assignment is not recoverable from the public arm rows. Public host IDs are listed; full environment epoch is unresolved. Round page describes 15 per arm, but delivery and official outcome populations are not joined to a valid shared ITT cohort. INTERIM. The round page reports 11/15 versus 12/15 and higher builder tokens for handoff; this is source-reported, not reproducible from public data as a reconciled comparison. No H51–H61-linked Wave B claim ID. r54, raw records, official outcomes
jev-route-ctx Offline route agreement on task labels; not a live Build arm. Route models/settings are on the receipt. Evaluation source rows and versioned label file are not public. Offline routing decisions only; no Build ITT. INTERIM; reference agreement is not route accuracy or task success. CL-WA-jev-route-ctx-reported-observation (INTERIM; no H IDs assigned in Wave B). round, claim ledger
jev-triage-ctx Prepared-context triage labels versus a prior bare-context diagnostic; the planned full pass did not run. Model/effort details are on the receipt; no builder arm. Label and source records are not public. Offline sample only; no live Build cohort. INTERIM; progression gate failed; do not infer better localization or completion. CL-WA-jev-triage-ctx-reported-observation (INTERIM; no H IDs assigned in Wave B). round, claim ledger
t98-cache-smoke One cache telemetry smoke; exact task and cell IDs are not public. Sol medium is source-reported; request-level model/effort rows are missing. Kogen pin is on the receipt; exact harness commit is missing. Receipt lists one planned/started/finished smoke cell, zero graded cells, and zero ITT cells. Request counts are a separate population. INTERIM; the reported weighted cache-hit rate was 67.0% across 18 usage-bearing requests out of 20 requests, below the 95% gate. This is source-reported, not reproducible from public data and is not a Build-efficacy result. CL-WA-t98-cache-smoke-reported-observation (INTERIM, linked by Wave B to H115, not H52). T98 smoke, claim ledger
t100-cache-lane Prospective cache-change lane. Not run. Not recovered. Planned count is not established; started, finished, graded, and ITT are zero in the claim ledger. NOT-RUN; it does not test a cache-change effect. CL-WA-t100-cache-lane-reported-observation (NOT-RUN, linked by Wave B to H116, not H52). T100 lane, claim ledger
historical cache summaries Kogen/Codex cache rates and a separate local aggregate; cohorts and token semantics differ. Not aligned. Epochs and request ledgers unavailable. Denominators are not public. INTERIM; historical rates and the reported percentage change are source-reported, not reproducible from public data and are not compatible with T98. CL-H115-historical-cache-hit-comparison; CL-H116-historical-cache-change (INTERIM). claim ledger
R70 language cohorts Language/task comparison; not a matched cache, prompt, and steering audit. Codex Luna max in separate language cohorts; see the language receipts. Different cohorts and source captures remain separated. Do not pool original, rerun, extension, or task-eight counts. Existing language evidence does not resolve H61’s cache/prompt/steering attribution. No H51–H61-linked claim ID. R70, R70 rerun, R70 extension

Design implications

Keep request telemetry, offline context quality, route agreement, and scored Build outcomes in separate tables. Cache comparisons require a shared definition, fixed request denominator, model and effort, prefix/session policy, host/provider epoch, and exact request records. Context-packet comparisons require a matched baseline and report both packet cost and downstream task success.

What this evidence does not show

The T98 smoke does not establish that a cache fix improves success, time, or cost. Historical cache-rate summaries are not denominator- or epoch-matched to T98. R54 does not demonstrate that handoff saves tokens or preserves success. Offline Jev agreement and partial Tidewave diagnostics do not establish live routing or context benefits. The available pages do not isolate language from cache, prompt, or human-steering differences.

Smallest open tests

Publish raw per-request cache records and repeat the same request sequence before and after a fixed-affinity change, with a predeclared gate and identical accounting. For context, test a bounded packet against a no-packet control on matched failures, then grade final Builds and include packet-generation cost. For H61, rerun language arms with identical prompt, cache key, steering policy, model, and token accounting.