--- name: Research Question Loop slug: research-question-loop category: AI Engineering description: Research Question Loop sharpens a vague topic into a few strong research questions. It drafts candidates, scores them with a Specific/Answerable/Novel/Feasible/Significant rubric, and revises weak areas until enough clear the bar. github: "https://github.com/gaasher/Agent-Loop-Skills/tree/main/loops/research-question" language: Python stars: 166 forks: 19 install: "npx degit https://github.com/gaasher/Agent-Loop-Skills/tree/main/loops/research-question ~/.claude/skills/research-question" installs_to: ~/.claude/skills/research-question source_path: loops/research-question/SKILL.md collection_size: 25 category_size: 3670 collection_url: "https://dirskills.com/collections/gaasher/Agent-Loop-Skills" added: 2026-09-08T05:34:45.700Z last_synced: 2026-09-08T05:34:45.700Z canonical_url: "https://dirskills.com/skills/research-question-loop" --- # Research Question Loop Research Question Loop sharpens a vague topic into a few strong research questions. It drafts candidates, scores them with a Specific/Answerable/Novel/Feasible/Significant rubric, and revises weak areas until enough clear the bar. **Install:** ```bash npx degit https://github.com/gaasher/Agent-Loop-Skills/tree/main/loops/research-question ~/.claude/skills/research-question ``` ## README # Research Question Loop A **sharpen → score → revise** loop for the *framing* stage of research. The artifact is a small set of research questions; the feedback signal is how many **clear the bar** — each scored 0-5 on five fixed axes (Specific, Answerable, Novel, Feasible, Significant). You start from a vague topic, draft candidates, score each against the rubric (with a light novelty check against the literature), and rewrite the weakest axis of the promising ones until enough are strong. A good research question is the hard part of research: too broad and it cannot be answered; too narrow and it does not matter; already settled and there is no point. The goal is **a few excellent questions**, not many mediocre ones — this loop drives toward the narrow band that is answerable, novel, and worth answering. ## Scope & limitations This loop produces and refines **questions**, grounded by a *light* novelty check (a few searches), not a full survey — for an exhaustive map use the literature-survey loop, and to turn a question into testable predictions use the hypothesis-generation loop. The novelty check needs web or literature access; without it (`novelty_check: none`), novelty is the loop's best judgment and must be labeled unverified. ## When to use Use when the user has a topic, area, or rough curiosity and wants it turned into concrete questions worth pursuing. Default: run the full draft→score→revise loop below. Escape hatch: if the user only wants candidates rated (no rewriting), score the set once and report the rubric breakdown. Not for grading a finished proposal, and not for generating hypotheses or experimental designs. ## Setup Resolve bindings interactively. If `loop.run.yaml` exists in the working dir, load it, confirm the values in one line, and skip to the loop. Otherwise: on Claude Code (the `AskUserQuestion` tool is available) infer a likely value for each binding and present it as the recommended option; on other hosts ask each as a quoted plain-text prompt. Then write `loop.run.yaml` (format: `examples/run.example.yaml`) and confirm the values before creating any other files. | binding | meaning | default | how to infer | |---|---|---|---| | `` | the area of interest (field, population, scope, what the user already cares about) | — | ask the user | | `` | how many strong questions to deliver | 3 | — | | `` | rubric score (0-100) a question must clear to count as strong | 75 | a solid question without demanding perfection | | `` | how to check whether a question is already answered: `lit` \| `web` \| `none` | `lit` if the sibling skill is installed, else `web` | probe for the literature-search skill (below) | | `` | output question set | `/questions.md` | — | | `` | where the ledger and report live | `./sandbox` | — | | `` | max iterations | 8 | — | **Novelty toolchain (only for `novelty_check: lit`).** Paper search goes through the sibling **`literature-search` skill** (` = /tools/lit_search.py`, with ` = python3` and `` its installed location, e.g. `~/.claude/skills/literature-search/`); the relevant moves are ` search ""` (is a direct answer already published?) and ` snippet ""` (pinpoint the answering passage). Confirm ` --help` works at setup; if the skill is absent, tell the user and either install it (copy the repo's `loops/literature-search` folder into `~/.claude/skills/`) or degrade to `web` (host WebSearch/WebFetch) or `none`. Record the resolved choice in `` so re-runs are non-interactive. ## The loop The **rubric** (a fresh Grader scores each question 0-5 per axis — see grading below): | Axis | 5 | 3 | 1 | |---|---|---|---| | **Specific** | one clear construct/relationship, well-scoped | direction clear, scope loose | broad/ambiguous topic, not a question | | **Answerable** | a concrete study/analysis could resolve it; the answer-shape is clear | resolvable in principle, approach unclear | not empirically/analytically decidable | | **Novel** | open per the novelty check; closest work cited | partly addressed; a real twist remains | already answered (check found a direct answer) | | **Feasible** | data/methods/access plausibly exist | feasible with effort | needs unavailable data or impossible measurement | | **Significant** | answering it changes understanding or practice | a useful increment | marginal even if answered | **Grading — spawn a fresh Grader per iteration (spawn-or-degrade).** Each iteration, spawn a freshly instantiated **Grader** subagent — separate from whoever drafted or revised the questions, so the score is independent and not self-graded — and give it each candidate plus its novelty evidence. It returns the five raw 0-5 per-axis points (no weights). On Claude Code spawn it as a real `Agent`; otherwise adopt the Grader role inline in a clean pass. The orchestrator sums to a **raw score out of 25**, then converts to the 0-100 score used everywhere: `total = 100 × raw / 25` (e.g. raw 20/25 → `total` 80). A question is **strong** when `total ≥ ` **and** no axis scored 1 (a single fatal axis sinks it regardless of `total`). Copy this checklist and tick items off: - [ ] Iteration 0 — restate `` and what is interesting about it; draft 3-5 candidate questions spanning different angles (mechanism, comparison, condition/boundary, application). Record nothing as strong yet. - [ ] Gather novelty evidence per candidate via `` (` search`/`snippet`, or WebSearch, or skip). - [ ] Score — spawn a **fresh Grader** with each candidate + its evidence; it returns raw 0-5 per axis; convert `raw/25 → total/100`. - [ ] Diagnose each promising question's **lowest axis** — the one thing keeping it from strong. - [ ] Revise that one axis (one focused move per question); drop a fatally-flawed question revision cannot save; add a fresh candidate if short. - [ ] Append a ledger row; stop when `` clear the bar, or at ``. **Iteration 0 — frame & draft.** Restate `` and what is interesting about it; draft 3-5 candidate questions spanning different angles. Record nothing as strong yet. **Then, until stop (`` strong, or ``):** 1. **Gather novelty evidence.** Run the `` for each candidate's core: ` search`/`snippet` (or WebSearch). If a direct answer exists, note the closest answered work; if only related work exists, note the open part. Each `` call prints JSON; on failure it prints `{"error","fallback"}` and exits non-zero — then fall back to WebSearch/WebFetch. 2. **Score.** Spawn a **fresh Grader** (spawn-or-degrade) and hand it each candidate plus its evidence; it returns the raw per-axis points. Convert `raw/25 → total/100`. 3. **Diagnose & revise.** Find each promising question's **lowest axis** and make one focused move on it: narrow an over-broad question to a specific population/condition; operationalize an unanswerable one into a measurable comparison; pivot an already-answered one toward the part the check showed is still open; raise significance by tying it to a decision or a contested claim. Drop questions with a fatal axis that revision cannot save; add a fresh candidate if you are short. 4. **Log** one ledger row and continue until `` are strong. **On stop**, write ``: each strong question with its rubric scores, the novelty note (closest answered work / the open part), why it is answerable (the study-shape that would resolve it), and why it matters — plus any runners-up and the axis that held them back. ## Ledger `/ledger.tsv`, tab-separated, never commas in the text. Header: ``` iter question total weakest_axis revision ``` Example: ``` iter question total weakest_axis revision 0 how does sleep affect learning 35 specific drafted; far too broad 1 does sleep timing affect retention 62 answerable operationalized: spaced-review vs sleep-matched review, 1-week retention 2 does post-learning sleep within 3h beat delayed sleep for procedural retention 86 - strong (novel per check: tested for declarative not procedural) ``` Report the **best** outcome — the strong questions and their scores — not necessarily the last iteration's set. ## Constraints - **A few strong questions beat many weak ones** — do not pad `` with questions that do not clear the bar; report them as runners-up with the blocking axis instead. - **Novelty is checked, not assumed** — when `` is `lit`/`web`, actually search, cite the closest answered work, and never claim novelty the check contradicts. When `none`, label novelty unverified. - **Grade with a fresh Grader** scoring raw 0-5 per axis (no weights); the orchestrator converts `raw/25 → 100` and never lets the drafter/reviser grade its own questions, so the score stays honest. - **One focused revision per question per iteration**, targeting its weakest axis, so improvement is attributable and questions converge rather than thrash. - **Keep the rubric and threshold fixed** so "strong" means the same thing throughout. - The sandbox is self-contained — no `../` escapes. Do not pause the loop to ask whether to continue; run until `` clear the bar or `` is hit.