Stable identity register
Hypotheses
Every H ID is generated from the source hypothesis register. Propositions and evidence statuses are reproduced without inferring results from nearby prose.
All entries are present in this page without JavaScript. Use your browser's find command, or open a family page for its source summary.
| ID | Falsifiable proposition | Evidence status |
|---|---|---|
| H01 | On matched tasks and model settings, the full Kogen Shape/Build/Review route has a higher pass rate than direct Codex. | UNRESOLVED; available same-model summaries are not a joined public comparison. |
| H02 | Shaping improves final task completion enough to offset its added cost and time. | UNRESOLVED; no public causal estimate isolates shaping. |
| H03 | For a defined population of approved Intents that pass specified probes, Build completion can approach the declared reliability target. | NOT ESTABLISHED; no validated guarantee cohort is available. |
| H04 | A bounded Shaping time and human-input budget can minimize total cost while preserving completion and resolving material unknowns. | UNRESOLVED; no public duration-by-human-input comparison. |
| H05 | Running probes that settle stated assumptions before Build reduces failed or stopped Builds against a matched no-probe condition. | UNRESOLVED; no isolated public contrast. |
| H06 | A task-granularity rule for Intents improves completion per unit of integration and coordination cost over alternative sizes. | UNRESOLVED; no public granularity comparison. |
| H07 | More specific Intent detail raises builder success, with a larger effect for weaker builders than for stronger builders. | UNRESOLVED; related records are exploratory or have unreconciled outcome joins. |
| H08 | A compact context pack and a strong-model plan allow a cheap builder to outperform its matched direct baseline without exceeding a direct strong-model cost target. | UNRESOLVED; no joined public route comparison. |
| H09 | One strong-model plan can guide multiple Luna-low builders and a low-cost integrator with equal or higher completion at lower total cost than a matched alternative. | NOT ESTABLISHED; no complete public comparison identified. |
| H10 | Giving a weak builder access to a stronger advisor after a defined failure state improves completion per total cost over retrying or restarting with a strong model. | NOT ESTABLISHED; adjacent escalation records do not test this advisor contrast. |
| H11 | Escalation triggered by observed failure matches or exceeds text-based pre-routing on completion while reducing total cost. | UNRESOLVED; cascade records do not yield a reconciled public verdict. |
| H12 | Checks, review, and actionable retries increase final completion over a reject-only gate under matched tasks and attempt budgets. | NOT ESTABLISHED; no prospective repair-benefit estimate. |
| H13 | Two complementary shapers resolve failures missed by either single shaper at an acceptable added cost. | NOT ESTABLISHED; no matched public comparison identified. |
| H14 | Sol planning with Luna-max building improves the success-cost frontier over an all-Luna route on matched tasks. | UNRESOLVED; available R74 record is WITHDRAWN. |
| H15 | More complete Shaping plus executable specifications improves success or cost over a matched direct Codex or Grok Build. | NOT ESTABLISHED; no controlled outcome comparison. |
| ID | Falsifiable proposition | Evidence status |
|---|---|---|
| H16 | Under matched concurrent project workloads, worktree, copy-on-write, or another isolation design reduces cross-project interference and failure without unacceptable setup or Build-time cost. | NOT ESTABLISHED; no matched design comparison. |
| H17 | Rebase-based landing produces fewer unwanted base-history changes or recovery errors than automatic merge while remaining within a declared time and conflict budget. | NOT ESTABLISHED; no comparative reliability or cost study. |
| H18 | A headless command-driven Shaping interface improves external-driver completion, latency, or error recovery over a Kogen TUI under the same tasks. | NOT ESTABLISHED; no measured usability or runtime comparison. |
| ID | Falsifiable proposition | Evidence status |
|---|---|---|
| H19 | On matched ChatGPT models, task IDs, prompts, tools, and effort, Pi and Codex differ in success, time, cost, or cache use by a reproducible amount. | UNRESOLVED; harness-pick records do not reconstruct the scored cohort. |
| H20 | Under matched prompts and tool access, at least one of Pi, OpenCode, Grok Build, Claude Code, or Codex has a better predeclared success-cost-time profile than the alternatives. | NOT ESTABLISHED; no complete comparable matrix. |
| H21 | A tuned custom Kogen harness matches or exceeds the selected external harness on the same task/model population and cost rule. | UNRESOLVED; associated receipts are interim or confounded. |
| H22 | End-to-end Kogen improves task completion enough to offset any slower loop overhead relative to a standalone harness. | NOT ESTABLISHED; no joined end-to-end comparison. |
| H23 | Elixir kh and Rust khr differ measurably in startup, tool-call, file-write, integration, or total Build time under equivalent work. | NOT ESTABLISHED; Rust khr comparison is not established in the public receipt. |
| H24 | Removing TUI code and using a command-only loop reduces runtime or latency while preserving task success. | NOT ESTABLISHED; no isolated test identified. |
| H25 | A lean coding-specific system prompt lowers tokens or improves cache use without reducing task success. | NOT ESTABLISHED; no public one-factor prompt result. |
| H26 | Replacing Claude Code’s prompt and constraining its tools changes its task outcomes in a way that separates harness effects from prompt/tool effects. | NOT ESTABLISHED; no paired public comparison. |
| H27 | A Claude Code -p wrapper preserves the relevant subscription route while improving Kogen control without unacceptable quality or latency loss. | NOT ESTABLISHED; no completed matched comparison. |
| H28 | Claude Code’s task quality offsets its Kogen loop maintenance cost compared with a native loop. | UNRESOLVED; available candidate records do not establish a scored cohort. |
| H29 | Parallel subagents in Shape, Develop, or Review improve completion or wall time enough to offset their added calls and tokens versus one editing agent. | UNRESOLVED; exploratory source summaries are not joined to exact public outcomes. |
| H30 | Omitting helper agents does not materially reduce task completion and lowers cost or complexity under a declared margin. | NOT ESTABLISHED; no public non-inferiority comparison. |
| H31 | Mid-turn steering, external tool access, and recovery affordances explain a measurable portion of Pi/Codex quality differences. | NOT ESTABLISHED; no isolated affordance ablation. |
| H32 | A mechanical verification gate after the final edit reduces unverified completion without unacceptable cost or false blocking. | UNRESOLVED; the public kh-climb receipt is partial and lacks exact cell joins. |
| H33 | Loop verbosity, output format, or prompt changes account for a measurable part of harness outcome differences. | UNRESOLVED; exploratory surface records are partial. |
| H34 | Shell-only editing tools outperform apply-patch-style tools on a matched task set under a predeclared completion and cost rule. | UNRESOLVED; historical pooled comparison is not fully reconciled. |
| H35 | A strong plan plus shell builder matches direct Sol while reducing total cost on matched tasks. | UNRESOLVED; public rows do not establish the source-reported comparison. |
| H36 | Kogen’s own loop differs from Codex embedded as its builder after the builder model, tasks, and stage policy are held constant. | NOT ESTABLISHED; no clean loop-isolation comparison. |
| H37 | A native Grok provider in Kogen gives a lower-cost or higher-success route than the matched alternative, including Grok Build as a separate comparator. | NOT ESTABLISHED; the formal Grok frontier has no run result. |
| H38 | An alternative provider or subscription has a better cost-success profile than current subsidized options under matched tasks and limits. | NOT ESTABLISHED; no comparative subscription matrix. |
| ID | Falsifiable proposition | Evidence status |
|---|---|---|
| H39 | A role-specific model/effort allocation improves the declared completion-cost-time frontier over a fixed allocation on matched tasks. | UNRESOLVED; no complete role-by-role public comparison. |
| H40 | The newer Luna model has a better cost-success profile than the earlier Luna baseline on the same tasks and harness. | NOT ESTABLISHED; model preference alone is not a measured comparison. |
| H41 | Increasing Luna effort improves completion more on harder tasks and yields a measurable cost tradeoff. | UNRESOLVED; no full difficulty-stratified effort curve. |
| H42 | The compared Sol, Luna, and Astra settings have distinguishable success, time, or cost profiles when builder role and tasks are held fixed. | UNRESOLVED; associated screens are exploratory. |
| H43 | The Sol low-effort setting outperforms Luna max in shaping, review, or synthesis roles enough to justify added cost. | NOT ESTABLISHED; no isolated role comparison. |
| H44 | Luna low plus strong-model assistance exceeds matched direct Luna-low and direct-Sol options on a declared cost/time-success frontier. | NOT ESTABLISHED; no exact public comparison. |
| H45 | Sol medium has a higher pass rate than Codex on a predeclared hard-task population. | UNRESOLVED; the public R66 record does not preserve the historical high-effort anchor cohort. |
| H46 | Astra is not cost-effective for routine tasks but improves outcomes enough on a capped high-value task set to justify a targeted route. | NOT ESTABLISHED; a small screen is insufficient for a policy. |
| H47 | Escalating after an observed failure beats fixed direct models or text-based routing under equal task and cost rules. | UNRESOLVED; cascade summaries are not reconciled. |
| H48 | Routing from task text has worse accuracy or cost than a matched policy using observed outcomes or context. | NOT ESTABLISHED; offline routing agreement is not live Build success. |
| H49 | Luna max for most implementation/operator work with Sol reserved for planning/rules is on the declared cost-success frontier. | NOT ESTABLISHED; operating preference is not a measured optimum. |
| H50 | A combined plan-level, role, reviewer, and contract route beats direct Sol on a preregistered completion-cost frontier. | NOT ESTABLISHED; available panels are not one matched frontier test. |
| ID | Falsifiable proposition | Evidence status |
|---|---|---|
| H51 | Cache reuse, lean prompts, or context management reduces Kogen loop tokens or cost more than other tested runtime changes while preserving task success. | UNRESOLVED; historical cache summaries use incompatible definitions and epochs. |
| H52 | A stable cache key/session affinity and stable prompt prefix raise the measured live hit rate to the declared gate on repeated requests. | NOT ESTABLISHED; the public smoke did not meet the target. |
| H53 | Cached, uncached, and input-token definitions can be reconciled across Kogen, Codex, and language arms to support a fair cost comparison. | NOT ESTABLISHED; no public compatible accounting table. |
| H54 | A Jev classifier selects useful transcript lines for compaction with lower latency and no material loss of task-relevant context. | UNRESOLVED; available trace records report no actual compactions. |
| H55 | Deterministic retrieval plus Jev and Luna readers produces more useful source-grounded packets for a stronger worker at lower total cost than the baseline. | UNRESOLVED; source screens are not joined to public Build outcomes. |
| H56 | A bounded packet of code relationships rescues failures beyond what file retrieval alone achieves on matched cases. | NOT ESTABLISHED; no public paired rescue test. |
| H57 | Prepared task-specific context improves Jev routing/classification accuracy over a bare-context baseline and improves downstream Build decisions. | NOT ESTABLISHED; offline diagnostics do not measure Build efficacy. |
| H58 | At least one Jev role among audit, routing, classification, review, compaction, or context preparation improves a predeclared outcome at acceptable cost. | UNRESOLVED; role inventory is incomplete and no general role winner is measured. |
| H59 | Passing a planner’s context to the builder reduces builder tokens without reducing matched-task success. | UNRESOLVED; the public R54 receipt is interim and does not establish token savings. |
| H60 | Runtime code inspection yields more useful task evidence than static retrieval on a fixed checkpoint and improves task outcomes. | NOT ESTABLISHED; Tidewave data is a partial diagnostic, not a live comparative result. |
| H61 | Language comparisons hold cache accounting, prompt, and human steering constant well enough to attribute outcome differences to language. | NOT ESTABLISHED; no reconciled public control table. |
| ID | Falsifiable proposition | Evidence status |
|---|---|---|
| H62 | A selective, faster check set can preserve detection of real failures while reducing check time or cost. | NOT ESTABLISHED. No public matched before/after result measures both failure detection and work or time. |
| H63 | A catalog of safe parallel tests can reduce verification wall time without changing results. | NOT ESTABLISHED. No public controlled catalog or concurrency comparison is identified. |
| H64 | Targeted tests of Build mechanics can preserve defect detection at lower cost than repeated full shaping checks. | NOT ESTABLISHED. No matched mechanics-test versus full-check result is published. |
| H65 | Clear handoff, process custody, failure classes, and visible stage status can prevent repeated expensive Build attempts. | NOT ESTABLISHED. Public records include blocked and smoke-only workflows, but no isolated live recovery comparison. |
| H66 | Strong deterministic checks can enable a low-effort model to complete tasks that weaker checks miss. | NOT ESTABLISHED. No check-ablation comparison is joined to official outcomes. |
| H67 | A scoped reviewer veto improves acceptance quality enough to justify its review and repair cost. | INTERIM, OFFLINE ONLY. Reviewer diagnostics are incomplete and are not a prospective live veto test. |
| H68 | Executable review catches small-diff false accepts that other review methods miss. | INTERIM, OFFLINE ONLY. The public summaries do not include the patch-level decision ledger. |
| H69 | Contract audits detect missing requested behavior and can safely gate completion or trigger repair. | INTERIM, OFFLINE ONLY. The published audit did not run a repair or Build rerun. |
| H70 | Explicit behavior checks outperform source-text or file-existence checks without losing relevant coverage. | NOT ESTABLISHED. Offline audits identify test-quality concerns, but do not compare those check designs prospectively. |
| H71 | The proposed static and domain-specific checks are accurate and fast enough for the default gate. | NOT ESTABLISHED. The study census lists check research, but no public default-gate portfolio comparison is available. |
| H72 | A post-change fix loop can repair a static-check failure rather than reject functionally passing work. | NOT ESTABLISHED. No matched repair-loop outcome is identified in the public records. |
| H73 | Verification should score observable behavior even when an implementation reports its own checks as successful. | PARTIAL, OFFLINE ONLY. Saved-output and candidate audits are counterfactual evidence, not a prospective grading-policy test. |
| H74 | Reusable duplicate, error, and type checks provide value beyond the Kogen project. | NOT ESTABLISHED. No cross-project transfer or maintenance comparison is published. |
| ID | Falsifiable proposition | Evidence status |
|---|---|---|
| H75 | A cheap starting task and one-lever-at-a-time climb identifies useful task and recipe changes without mistaking saturation for general success. | INTERIM. Several desktop Mac hill-climb iterations are reported, but cell-level outcomes and the final iteration are not fully reconciled in the public export. |
| H76 | Luna or Grok is the better low-cost, fast starting builder for the intended task set. | NOT ESTABLISHED. No matched comparison for this starting-model choice is linked to a public claim. |
| H77 | A benchmark limited to Rails tasks can misstate performance on Elixir/Phoenix or Rust work. | NOT ESTABLISHED. Cross-stack task and implementation records exist, but no matched task-set study isolates stack or recency effects. |
| H78 | Public real-codebase tasks pinned to source revisions are more reproducible than hidden replay tasks. | NOT ESTABLISHED. Public task provenance is useful for reproduction, but no task-quality comparison tests the proposition. |
| H79 | SQLite tasks allow more reliable parallel execution than PostgreSQL-heavy tasks while a smaller PostgreSQL set preserves realism. | NOT ESTABLISHED. No controlled database-task concurrency comparison is published. |
| H80 | Independent review and wrong-solution and valid-alternative controls expose benchmark or grader errors before costly runs. | PARTIAL METHOD EVIDENCE. Public records preserve invalid grades and control lanes, but do not quantify the operating characteristics of this review policy. |
| H81 | Wall-time comparisons require equivalent stable hosts and controlled load rather than a single remote development link. | METHOD REQUIREMENT, NOT A PERFORMANCE RESULT. Public records do not establish a controlled host-effect estimate. |
| H82 | Pairing equivalent workers is valid when host, load, concurrency, and provider epoch are recorded and balanced. | NOT ESTABLISHED. No completed public balance audit validates this pairing rule. |
| H83 | Preliminary grading and reuse of frozen baseline results can reduce new cells without changing a decision. | NOT ESTABLISHED. No public operating-characteristic or cost comparison validates the savings and error rate. |
| H84 | A provider-load epoch can explain cross-day timing changes even when the harness is unchanged. | NOT ESTABLISHED. The public snapshot does not isolate provider epoch from host, load, or revision. |
| H85 | Sequential allocation, saturation pairs, single-repetition screens, and futility stops reduce study cost while preserving decisions. | NOT ESTABLISHED. No public simulation or prospective validation measures decision error and savings together. |
| H86 | Results with pinned versions, pass rate, wall time, cost, invalid cases, and full setup are reproducible and interpretable. | PARTIAL REPORTING EVIDENCE. Public pages and ledgers supply some of these fields, but unresolved joins and missing protocol pins remain. |
| H87 | Design and decision rules recorded before execution, with exploratory results labelled, reduce retrospective reinterpretation. | PARTIAL PROCESS EVIDENCE. Public pages distinguish preregistered, descriptive, withdrawn, and incomplete cohorts; a complete registration-history audit is not available here. |
| H88 | Intention-to-treat scoring counts internal stops as failures and excludes only demonstrated environment or adapter faults. | NOT ESTABLISHED AS A COMPLETE POLICY. Several unresolved ITT dispositions remain in the public exports. |
| H89 | Recording task, arm, revision, host, load, timeout, attempts, setup, and token categories is sufficient for later reproduction. | PARTIAL REPORTING EVIDENCE. The public run ledger exposes missing fields, so completeness is not established. |
| H90 | Kogen can solve strategically chosen tasks that direct Codex does not solve. | NOT ESTABLISHED. The public frontier lane has no released outcome cohort. |
| ID | Falsifiable proposition | Evidence status |
|---|---|---|
| H91 | A matched comparison can identify the best implementation language for the Kogen agent loop across task success, reliability, cost, and implementation time. | DESCRIPTIVE ONLY. Public cohorts are small and incomplete, and some original tasks have implementation confounds. |
| H92 | Rust has a better runtime and concurrency tradeoff than Elixir OTP when logic, prompts, and repair opportunities are equivalent. | NOT ESTABLISHED. The public comparison does not hold all of those factors constant. |
| H93 | Separating Mix applications into an umbrella makes tests and the workflow faster. | NOT ESTABLISHED. No before/after architecture comparison is identified. |
| H94 | Language-agnostic executable acceptance specifications preserve the same semantics across implementations. | PARITY METHOD, NOT EFFICACY. Conformance can test parity but does not show improved task completion. |
| H95 | Quint specifications written during planning improve Kogen or direct-model outcomes. | NOT ESTABLISHED. No controlled live comparison is identified. |
| H96 | Reviewing the translation from natural-language requirements into Quint catches omitted behavior. | NOT ESTABLISHED. No public translation-audit outcome is linked. |
| H97 | A fair language comparison holds tasks, prompts, model, effort, steering, grader, and repair loop constant. | METHOD REQUIREMENT. Existing language cohorts do not establish full parity. |
| H98 | Gleam should remain eligible until a defined matched-comparison rule supports excluding it. | NOT ESTABLISHED. No public inclusion-rule test or Gleam cohort is available. |
| H99 | Compile time, run wall time, token use, and task grade should all inform the language choice. | PARTIAL MEASUREMENT. Public timing and resource files exist, but their measurements are descriptive and do not by themselves rank full implementations. |
| ID | Falsifiable proposition | Evidence status |
|---|---|---|
| H100 | A parity-checked Elixir/Phoenix implementation can approach or beat the selected Rust implementation on a matched Campfire feature. | NOT ESTABLISHED. Both public Campfire lanes were withdrawn before formal cells. |
| H101 | A small representative feature can be compared fairly across stacks using SQLite, matched optimization, and disabled caching. | NOT ESTABLISHED. No qualifying matched feature cohort ran. |
| H102 | A stronger Exqlite or Ecto implementation changes the relative runtime result. | NOT ESTABLISHED. No public matched adapter or implementation comparison is available. |
| H103 | Throughput measurements reject implementations that drop messages or otherwise fail correctness. | NOT ESTABLISHED. No public correctness-control results are available for the Campfire comparison. |
| H104 | The chosen feature is representative, reviewable, and suitable for a public web-stack benchmark. | NOT ESTABLISHED. No completed public review of the feature-selection rule is linked. |
| H105 | Rails agent tasks can be ported to Elixir or Rust so agent performance can be compared on equivalent tasks. | NOT ESTABLISHED. The public Elixir-port round does not establish a matched cross-stack agent comparison. |
| ID | Falsifiable proposition | Evidence status |
|---|---|---|
| H106 | A compact structured format can reduce tokens while remaining easier for agents to parse than JSON, YAML, or TOML and more useful than plain text. | NOT ESTABLISHED. No complete, matched parse-and-repair comparison is published. |
| H107 | Translating tool output into a compact format, with a fallback when needed, improves agent comprehension. | NOT ESTABLISHED. No public round-trip and live task comparison is linked. |
| H108 | Compact stack traces and error reports help an agent find and fix the correct code. | NOT ESTABLISHED. No controlled trace-format repair outcome is available. |
| H109 | Structured logs improve diagnosis without removing useful detail. | NOT ESTABLISHED. No matched logging-format study is published. |
| H110 | Agent-facing output rules should include only formats and checks with measured improvements. | INTERIM. A partial output-feedback cohort exists, but does not establish a format preference or completion benefit. |
| ID | Falsifiable proposition | Evidence status |
|---|---|---|
| H111 | Staged and direct model comparison. On a shared task set, a staged Kogen route may help a weaker builder solve failures while direct Sol high may have a higher pass rate, lower cost per pass, or longer wall time. Falsify with exact task-matched outcomes, cost, and wall receipts for every arm. | UNRESOLVED; historical summary is source-reported and not reproducible from public data. |
| H112 | Route cost and quality frontier. Across a shared task set, direct Sol, route-v3, and direct Luna max may occupy different pass-rate and cost-per-pass points. Falsify with one reconciled cohort, compatible cost accounting, and task-level outcomes. | UNRESOLVED; historical cost frontier is source-reported and not reproducible from public data. |
| H113 | R72 same-model comparison. On R72 P1, the Kogen Luna-max ladder may outperform direct Codex Luna max. Falsify with a complete cell join and a prespecified paired analysis. | INTERIM; R72 is descriptive and does not establish superiority. |
| H114 | R70 language choice. The R70 implementation-stack comparison and selected RvE sample may identify a better language for Kogen. Evaluate with a parity-controlled comparison whose task, prompt, harness, and repair loop are matched; a selected sample alone cannot determine a broad language winner. | CONFOUNDED / DESCRIPTIVE; language cohorts remain separate. |
| H115 | Cache-hit explanation. A Kogen cache-hit rate below Codex's may explain much of the token-spend difference. Falsify with request-level records from matched epochs and compatible cache/token definitions, then relate cache changes to cost. | INTERIM; historical cache summaries are unreconciled; T98 is telemetry only. |
| H116 | Cache-change effect. A cache change may increase cache use. Falsify with a defined metric and baseline, matched request traffic, and before/after records. | NOT-RUN; the T100 lane has no result. |
| ID | Falsifiable proposition | Evidence status |
|---|---|---|
| H117 | Planner context pack. A compact context pack may produce a better or cheaper plan than a file-list input. Compare the two inputs with all other planner and builder settings fixed. | Not established as a distinct run. |
| H118 | Plan guard. A plan sanity/constraint gate may prevent plans that would otherwise cause Build failures. Compare downstream outcomes and gate cost with and without the guard. | Not established as a distinct run. |
| H119 | Plan examples. Relevant examples may improve transfer from plan to implementation. Hold the plan and builder settings fixed while changing example exposure. | Not established as a distinct run. |
| H120 | Plan reproduction evidence. Evidence that reproduces a failure may improve the plan's diagnosis and implementation. Compare with an otherwise identical plan without that evidence. | Not established as a distinct run. |
| H121 | Plan interfaces. Explicit cross-component interfaces may reduce integration failures. Compare matched tasks with interfaces present or absent. | Not established as a distinct run. |
| H122 | Planner turns. Varying the number of planner turns may change completion enough to justify its time and token cost. Compare fixed turn budgets. | Not established as a distinct run. |
| H123 | Plan debate. Competing plans may improve plan choice and downstream task success. Compare debate against a single-plan control. | Not established as a distinct run. |
| H124 | Plan specificity. Criteria, approach, and step-level plans may affect builders differently. Compare each level within builder and task strata. | A planning-arm cohort appears in R51; its summary is withheld pending cell-level reconciliation. |
| H125 | Review scope. A narrower review surface may reduce cost while preserving useful findings. Compare findings, false blocks, missed defects, and review cost. | Not established as a distinct run. |
| H126 | Executable review. Running review checks may detect defects that text-only review misses. Measure true and false review findings and prospective repair outcomes. | No live, prospective executable-review benefit is established. |
| H127 | Contract gate. Checking requirements before acceptance or escalation may catch incomplete work. Compare detection and false-block rates, then measure Build outcomes after any repair. | No prospective Build benefit is established. |
| H128 | Best-of-N. Generating and selecting among candidates may increase successful completion enough to justify the additional attempts. Compare selected outcomes with the same direct baseline and report pass@N separately. | No deployable best-of-N benefit is established. |
| H129 | Hybrid pipeline. Combining staged modes may improve the pass/cost frontier over its constituent recipes. Compare the complete bundle and every constituent arm. | Not established as a distinct run. |
| H130 | Split work. Dividing implementation among workers may increase successful throughput. Compare a fixed split plan with a solo arm, including coordination cost and wall time. | No split-specific outcome comparison is established. |
| H131 | Routing policies. Dynamic, forced, or versioned routing may assign useful models more effectively than fixed routes. Compare on the same tasks and model/effort choices. | The historical route-v3 summary is source-reported and not reproducible from public data. |
| H132 | Route cascade. Triggering escalation only after failure may preserve quality at lower cost than always escalating. Compare policy-level success and cost against fixed direct models. | No validated cascade policy ranking is established. |
| H133 | Cascade gate. Contract, review, or combined gates may differ in their ability to trigger useful escalation. Compare gate sensitivity, false triggers, success, and cost. | No final gate ranking is established. |
| H134 | Context handoff. Passing prepared context to later stages may reduce repeated work or cost without reducing completion. Compare a handoff arm with a matched no-handoff arm. | R54 reports a handoff vs plain-pack comparison; its round status remains INTERIM. |
| H135 | Recipe attribution. Staged, direct, direct-shell, direct-escalate, plan-shell, and escalate-shell recipes may differ in outcome and cost. Each needs an explicit arm manifest and direct within-cohort comparison. | Multiple labels occur across rounds, but the public recipe-to-cell index is incomplete. |
| H136 | Ladder composition. Luna-only, Sol-effort, and diverse ladders may differ in success and cost. Compare each ladder under a fixed task set and trigger rule. | Several exploratory ladder rounds ran; no general best ladder is established. |
| H137 | Harness-version effect. Changes in Codex CLI or Claude Code versions may change earlier harness comparisons. Compare versions on identical tasks, requests, and host conditions. | No controlled version comparison is established. |
| H138 | Matched model/effort baselines. Kogen configurations with Luna max or Sol low/medium/high builders, with and without Sol planning, may differ from direct baselines. Match the effective model and effort at each tested role. | No complete current-version matched set is established. |
| H139 | Raw versus shaped Intent. Supplying a raw external-project Intent instead of a provided shaped Intent may change Kogen's apparent advantage. Compare both inputs against the same task and model baselines. | No exact raw-vs-provided arm join is established. |
| ID | Falsifiable proposition | Evidence status |
|---|---|---|
| H140 | Cross-harness route. Claude as the primary builder with Codex adversarial review may outperform an agent-only route on completion quality or cost. Compare the full route with a matched single-harness control. | NO EMPIRICAL EVIDENCE; no controlled comparison is identified. |
| H141 | Jev audit during Shaping. A controller-guaranteed Jev audit during Shaping may catch feasibility or context errors before approval. Compare audit findings, false alarms, and downstream Build outcomes with the audit disabled. | NO EMPIRICAL EVIDENCE; no controlled comparison is identified. |
| H142 | Vertical integration. Building and integrating the same requested feature may reduce wasted or brittle partial implementations compared with plumbing work that is not tied to an end-to-end Intent. Compare completion and rework on matched tasks. | NO EMPIRICAL EVIDENCE; no controlled comparison is identified. |
| H143 | Stashed work reuse. Reusing valid existing work may reduce Build time without increasing correctness failures. Compare against a clean-start control with the same acceptance suite. | NO EMPIRICAL EVIDENCE; no controlled comparison is identified. |
| H144 | Shaping preflight. A bounded, low-cost preflight audit may verify material risks before longer Shaping work. Compare audit cost, detected risks, false blocks, and Build outcomes against a matched no-preflight control. | NO EMPIRICAL EVIDENCE; no controlled comparison is identified. |
| H145 | Build reliability. Reliability changes may prevent repeated costly cycles and classify environment failures separately from semantic failures. Compare retries, successful completion, and total cost under injected and naturally occurring failure classes. | INTERIM; no joined reliability or cost comparison is available. |
| H146 | Parallel Builds on a second Mac. A second Mac may raise throughput while preserving Build ownership and verification. Compare serial and parallel operation on the same eligible workload, including coordination, queue, and grading time. | Source-reported topology diagnostic only; the proposed second-host workflow is untested. |
| H147 | Build status and alerts. Showing current stage/check state and offering a completion alert may reduce unattended waiting and missed failures. Measure detection time, missed events, and operator wait; this is not a model pass-rate hypothesis. | NO EMPIRICAL EVIDENCE; no alert/control comparison is identified. |
| ID | Falsifiable proposition | Evidence status |
|---|---|---|
| H148 | New context data structure. A new structure for code and domain knowledge may provide more bounded, explainable context than a conventional graph, retrieval-augmented generation, or vector store. Compare retrieval and downstream task outcomes using the same queries and packet budget. | OFFLINE, SOURCE-REPORTED; retrieval diagnostics do not measure Build success. |
| H149 | Change-centered persistence. Persisting changes, queries, or transformations as primary objects may handle code and requirement evolution better than storing atomic facts and edges. Compare update effort, stale-answer rate, and retrieval quality after controlled edits. | OFFLINE, SOURCE-REPORTED; freshness diagnostics do not measure Build success. |
| H150 | Bounded uncertain context. A context store may produce bounded task packets while exposing uncertainty, stale observations, and exceptions after edits. Compare packet size, calibration, and task outcomes under controlled repository changes. | OFFLINE, SOURCE-REPORTED; no matched live Build result is available. |
| H151 | Targeted Luna-medium screen. Luna medium on simpler tasks where Codex failed may reveal a lower-cost harness improvement sooner than difficult Sol tasks. Compare on frozen prior-failure tasks, with a same-task direct baseline and complete cost/effort receipts. | INTERIM; the frontier result is incomplete and is not the proposed Luna-medium screen. |
| H152 | TrackLine task fairness. TrackLine must be repaired and audited as a fair task before its results inform harness decisions. Establish task correctness, valid alternatives, reproducible setup, and grader sensitivity before scored use. | NO EMPIRICAL EVIDENCE; no completed public fairness audit is identified. |
| H153 | Harness and prompt mining. Mining harness implementations and prompts may reveal mechanisms behind performance differences. Convert candidate mechanisms into a complete feature map and test them with isolated ablations rather than treating the inventory itself as causal evidence. | INTERIM; mixed-version inventory only, with no causal result. |