--- name: Hypothesis Generation Loop slug: hypothesis-generation-loop category: AI Engineering description: Hypothesis Generation Loop generates candidate research hypotheses and vets them against retrieved literature. Use it when you need a ranked pool of novel, testable hypotheses rather than a single research question or a proposal review. github: "https://github.com/gaasher/Agent-Loop-Skills/tree/main/loops/hypothesis-gen" language: Python stars: 166 forks: 19 install: "npx degit https://github.com/gaasher/Agent-Loop-Skills/tree/main/loops/hypothesis-gen ~/.claude/skills/hypothesis-gen" installs_to: ~/.claude/skills/hypothesis-gen source_path: loops/hypothesis-gen/SKILL.md collection_size: 25 category_size: 3670 collection_url: "https://dirskills.com/collections/gaasher/Agent-Loop-Skills" added: 2026-09-08T05:34:41.887Z last_synced: 2026-09-08T05:34:41.887Z canonical_url: "https://dirskills.com/skills/hypothesis-generation-loop" --- # Hypothesis Generation Loop Hypothesis Generation Loop generates candidate research hypotheses and vets them against retrieved literature. Use it when you need a ranked pool of novel, testable hypotheses rather than a single research question or a proposal review. **Install:** ```bash npx degit https://github.com/gaasher/Agent-Loop-Skills/tree/main/loops/hypothesis-gen ~/.claude/skills/hypothesis-gen ``` ## README # Hypothesis Generation Loop A **multi-agent, literature-grounded** generation loop. The artifact is a growing **pool of hypotheses**; the feedback signal is the count of **strong, distinct hypotheses** that clear the bar — where "strong" is decided against real retrieved literature, not assertion. Each round: **generate → ground → judge → keep → mutate toward the gaps**, until the pool stops growing (saturation). The discipline: a hypothesis enters the pool only if the literature says it is **not already established** (novelty), prior work makes it **plausible** (grounding), and a **feasible test** exists. Generating is not confirming — the output is a ranked set of strong *candidates to test*, each stated with how to test it. The cast (all in `roles/`): - `roles/Generator.md` — proposes a batch of candidate hypotheses aimed at the open gaps. - `roles/LiteratureScout.md` — grounds each candidate in real literature (novelty · support · gap); emits `litscout.json` (validates `schemas/litscout.schema.json`). - `roles/Judge.md` — scores each against the **fixed rubric** and decides keep/kill/dedupe; emits `verdict.json` (validates `schemas/verdict.schema.json`). **Spawn-or-degrade.** On Claude Code, spawn Generator / LiteratureScout / Judge as real `Agent` subagents each round; otherwise adopt each role inline in this context. You are the orchestrator. ## When to use Use when the user wants candidate hypotheses *generated and vetted* for a question or domain. Default: run the full generate→ground→judge loop below until saturation. Escape hatch: if the user only wants a single batch (no looping), run one round and report the kept hypotheses. Not for sharpening or decomposing a question, and not for grading an existing written proposal. ## Setup **Resolve bindings interactively.** If `loop.run.yaml` exists in the working dir, load it, confirm the values in one line, and skip to the loop. Otherwise: on Claude Code (the `AskUserQuestion` tool is available) infer a likely value for each binding and present it as the recommended option; on other hosts ask each as a quoted plain-text prompt. Then write `loop.run.yaml` (format: `examples/run.example.yaml`) and confirm every value plus the live/degraded literature tier before creating any other files. | binding | meaning | default | how to infer | |---|---|---|---| | `` | the research question / domain, plus any scope (field, population, constraints) | — | ask the user | | `` | candidate hypotheses the Generator proposes per round | 6 | — | | `` | rubric score (0-100) a hypothesis must clear to enter the pool | 65 | — | | `` | LiteratureScout grounding depth (`low`/`medium`/`high`, see below) | `medium` | — | | `` | where rounds, ledger, and lit cache live | `./sandbox` | — | | `` | max rounds | 6 | — | | `` | stop after this many rounds with no new kept hypothesis | 2 | — | | `` | final ranked hypothesis set | `/hypotheses.md` | — | **Grounding depth dial** (`` caps per round — candidates examined deeply · queries each · papers read full-text): | preset | candidates deep | queries each | fulltext reads | |---|---|---|---| | **low** | 2 | 1 | 0 (snippet/abstract only) | | **medium** *(recommended)* | all | 2 | 1 | | **high** | all | 3 | 3 | **Literature toolchain.** Paper search goes through the sibling **`literature-search` skill** — resolve `` (it installs as a sibling, e.g. `~/.claude/skills/literature-search/`), ` = python3`, and ` = /tools/lit_search.py` (note the `tools/` segment); append `--cache-dir /literature/.cache` after a subcommand to reuse the cache. Confirm ` --help` works at setup; if the skill is absent, tell the user and either install it (copy the repo's `loops/literature-search` folder into `~/.claude/skills/`) or degrade all retrieval to WebSearch/WebFetch (tag that evidence `source:"web"`). The keyless S2 + arXiv core needs no setup; a free `S2_API_KEY` makes `snippet`/`cite` reliable. **API key (optional, never block).** The `literature-search` skill owns the key convention: run ` keys --init`, then have the user fill the printed `keys.env` themselves and never paste secrets into chat. Re-run ` keys` to record the tier in `loop.run.yaml` (`literature_tiers`, presence only). A missing key just degrades to the keyless pool → WebSearch. **Initialise the sandbox** once bindings are confirmed: ``` / ├── loop.run.yaml ← resolved bindings + literature_tiers ├── ledger.tsv ← header only (see Ledger) └── literature/.cache/ ← lit_search on-disk cache ``` Start with an empty `pool` and `gaps` seeded from ``; create no round files until the loop runs. ## The loop `pool` = the kept hypotheses (starts empty). `gaps` = open questions the LiteratureScout has surfaced (starts empty; seed from ``). `dry` = consecutive rounds with no new keep (starts 0). `` starts at 1. Copy this checklist and tick items off: - [ ] **Generate** — run `roles/Generator.md` (spawn-or-degrade) with ``, the current `pool`, `gaps`, and ``; it writes `round/candidates.json` (`` specific, testable, plausibly-novel candidates aimed at the gaps, none duplicating the pool). - [ ] **Ground** — run `roles/LiteratureScout.md` (spawn-or-degrade) on `candidates.json` with `` and the `` caps; it writes `round/litscout.json` (validates `schemas/litscout.schema.json`) — per candidate: novelty + closest prior work, support, gap, testability, each citing **real** evidence. - [ ] **Judge** — as `roles/Judge.md`, apply the fixed rubric + evidence gate to `litscout.json`, checking each candidate against the `pool` for duplicates; write `round/verdict.json` (validates `schemas/verdict.schema.json`): scores, `total`, `keep`, `duplicate_of`. - [ ] **Update** — add every `keep:true` non-duplicate to `pool` (with scores + grounding + how-to-test); add this round's `gap` points to `gaps`. If ≥1 new keep, `dry = 0`; else `dry += 1`. - [ ] **Log** one ledger row (see Ledger); `N = N + 1`. - [ ] **Stop check** — `dry == ` (saturation) or `N > ` → stop (see Stops). **Re-ground every round** — novelty is judged from a *fresh* literature check each round, never carried over, so "the literature already covers this" reliably kills a crowded idea. Every cited snippet comes from a real retrieval that round; on `{"error","fallback"}` fall back to WebSearch/WebFetch — never invent a paper. `schemas/litscout.schema.json` gates the LiteratureScout output — a generic instance: ```json {"round": 2, "evaluations": [{"hid": "r2h1", "novelty_assessment": "novel", "closest_prior_work": [{"claim": "X tested for facts, not skills", "cites": ["E1"]}], "support": [{"claim": "spacing aids motor consolidation", "cites": ["E2"]}], "gap": [{"claim": "long-term procedural retention untested at scale", "cites": ["E1"]}], "testability_note": "RCT: spaced vs massed schedule, 1-month retention."}], "evidence": [{"key": "E1", "title": "...", "source": "s2", "id": "a1b2", "snippet": "...verbatim..."}]} ``` `schemas/verdict.schema.json` gates the Judge output — a generic instance: ```json {"round": 2, "verdicts": [{"hid": "r2h1", "scores": {"novelty": 4, "grounding": 4, "testability": 5, "specificity": 4, "significance": 4}, "total": 82.0, "keep": true, "gate_failures": [], "duplicate_of": null, "rationale": "Closest work shows X untested for skills -> novel; supported; clean RCT named."}]} ``` ## Ledger `/ledger.tsv`, tab-separated, never commas in free text. Header: ``` round generated kept_new pool_size top_kept ``` Example: ``` round generated kept_new pool_size top_kept 1 6 3 3 spaced practice aids procedural (not just declarative) retention [82] 2 6 2 5 sleep-timed review beats time-of-day-matched review [78] 3 6 0 5 - ``` Per-round `candidates.json` / `litscout.json` / `verdict.json` live in `round/`. Report the **best** state of the pool when stopping, not just the last round. Leave `ledger.tsv`, `round*/`, and `literature/` untracked. ## Constraints - **Never fabricate citations or snippets** — every evidence entry is a real ``/WebFetch retrieval from that round, verbatim; the evidence gate exists to catch fabrication. - **Novelty is decided by the literature, not assertion** — a hypothesis the search shows is already established is killed, however appealing; "I think it's novel" with no closest-work search caps novelty. - **Generate ≠ confirm** — kept hypotheses are strong *candidates to test*, each stated with its test; never report them as established findings. - **The rubric is fixed** and the keep-bar is stable across rounds, so saturation is a meaningful stop. - **Reward distinct hypotheses, not volume** — duplicates and rewordings are cut. - **No installs** — the sibling `literature-search` skill is stdlib-only; never print or commit API keys (`keys.env` stays gitignored at the project root). The sandbox is self-contained — no `../` escapes. - Do not pause the loop to ask whether to continue; run until saturation or budget. ## Stops The loop stops on the first of: - **Saturation** — `dry == ` consecutive rounds add no new kept hypothesis. - **Budget** — `` rounds reached. End with the **ranked hypothesis set** (`` path) — each hypothesis with its statement, novelty assessment + closest prior work (cited), supporting evidence (cited), the gap it fills, and how to test it — plus the pool-size trajectory from `ledger.tsv` and the **strongest unexplored gaps**, so the user sees both the vetted hypotheses and where a deeper run would look next.