--- name: Research Proposal Loop slug: research-proposal-loop category: AI Engineering description: Research Proposal Loop iteratively grades and revises an existing research proposal using literature-grounded feedback. Use it when a proposal needs stronger soundness, contribution, and evidence before it passes a target score. github: "https://github.com/gaasher/Agent-Loop-Skills/tree/main/loops/research-proposal" language: Python stars: 166 forks: 19 install: "npx degit https://github.com/gaasher/Agent-Loop-Skills/tree/main/loops/research-proposal ~/.claude/skills/research-proposal" installs_to: ~/.claude/skills/research-proposal source_path: loops/research-proposal/SKILL.md collection_size: 25 category_size: 3670 collection_url: "https://dirskills.com/collections/gaasher/Agent-Loop-Skills" added: 2026-09-08T05:34:45.407Z last_synced: 2026-09-08T05:34:45.407Z canonical_url: "https://dirskills.com/skills/research-proposal-loop" --- # Research Proposal Loop Research Proposal Loop iteratively grades and revises an existing research proposal using literature-grounded feedback. Use it when a proposal needs stronger soundness, contribution, and evidence before it passes a target score. **Install:** ```bash npx degit https://github.com/gaasher/Agent-Loop-Skills/tree/main/loops/research-proposal ~/.claude/skills/research-proposal ``` ## README # Research Proposal Loop The artifact is a **research proposal**; the feedback signal is **ScholarEval** (literature-grounded Soundness + Contribution) turned into a **0-100 grade** by a **Judge** against the fixed `rubric.md`. Each iteration **evaluates → grades → revises** until the grade clears `` or the budget runs out. **North star.** Clearing the threshold is the stopping condition, not the goal. The goal is the strongest, most novel, genuinely publishable version the proposal can *honestly* become — every revision should ask "does this make the work more significant and more novel?", not just "does this patch a flaw?". This is **grounded ambition**: the lift comes from better-justified methods, a sharper-but-defensible novelty claim, and stronger baselines, all backed by real retrieved evidence. Overclaiming *lowers* the grade (evidence gate + Contribution axis); it never raises it. The cast (all in this folder): - `roles/ScholarEval.md` — the two-module literature-grounded evaluator; emits `scholareval.json`. - `roles/Judge.md` — grades the feedback → 0-100 + ranked fixes; emits `verdict.json` (decides pass). - `roles/Reviser.md` — rewrites the proposal to address the fixes (guards the research intent). - `rubrics/rubric.md` — the **fixed** grading rubric (the Judge never edits it). - `schemas/scholareval.schema.json`, `schemas/verdict.schema.json` — the two validated outputs. **Spawn-or-degrade.** On Claude Code, spawn ScholarEval / Reviser as real `Agent` subagents; otherwise adopt each role inline. You are the orchestrator and the Judge. ## When to use Use when a written proposal exists and the user wants it pushed past a quality bar with literature-grounded critique. Default: run the full evaluate→grade→revise loop below. Escape hatch: if the user only wants the critique (no rewriting), run one ScholarEval + Judge pass and stop. Not for writing a proposal from a blank page, and not for a standalone literature survey. ## Setup **Resolve bindings interactively.** If `loop.run.yaml` exists in the working dir, load it and skip to the loop. Otherwise: on Claude Code (the `AskUserQuestion` tool is available) infer a likely value for each binding and present it as the recommended option; on other hosts ask each as a quoted plain-text prompt. Then write `loop.run.yaml` (format: `examples/run.example.yaml`) and confirm every value, ``, and the live/degraded literature tier before creating any other files. | binding | meaning | default | how to infer | |---|---|---|---| | `` | file with problem + methodology + planned experiments | — | scan for a likely `.md`/`.txt`/`.pdf`; if only prose is pasted, save it to `/iter1/proposal.md` | | `` | core research question + headline contribution, 2-3 sentences — **frozen**; the Reviser may never change it | — | read the proposal, extract it, have the user confirm | | `` | grade (0-100) the proposal must reach to pass | 75 | a solid, well-grounded proposal without demanding perfection | | `` | max iterations | 6 | — | | `` | stop after this many consecutive no-improvement iterations | 2 | — | | `` | how much literature ScholarEval pulls per iteration (`low`/`medium`/`high`, see below) | `medium` | — | | `grade_weights` | `soundness · contribution · evidence_quality` (must sum to 1; frozen for the run) | `0.45 · 0.35 · 0.20` | `rubric.md`; recommend default | | `` | where snapshots, ledger, and lit cache live | `./sandbox` | — | **Evaluation depth dial** (`` caps per iteration — methods · dimensions examined · queries each · papers read full-text): | preset | methods · dims | queries each | fulltext reads | effort | |---|---|---|---|---| | **low** | 2 · 2 | 1 | 0 (snippet/abstract only) | low | | **medium** *(recommended)* | 4 · 3 | 2 | 2 | medium | | **high** | 6 · 5 | 3 | 5 | high | **Literature toolchain.** Paper search goes through the sibling **`literature-search` skill** — resolve `` (it installs as a sibling, e.g. `~/.claude/skills/literature-search/`), ` = python3`, and ` = /tools/lit_search.py`; append `--cache-dir /literature/.cache` after a subcommand to reuse the cache. Confirm ` --help` works at setup; if the skill is absent, tell the user and either install it (copy the repo's `loops/literature-search` folder into `~/.claude/skills/`) or degrade all retrieval to WebSearch/WebFetch (no ranked snippets or citation-graph expansion). The keyless **S2 + arXiv** core (`search`/`snippet`/`cite`/`fulltext`) needs no setup; a free `S2_API_KEY` makes `snippet`+`cite` (ScholarEval's two defining moves) reliable. **API key (optional, never block).** The `literature-search` skill owns the key convention: run ` keys --init`, then have the user fill the printed `keys.env` themselves and never paste secrets into chat. Re-run ` keys` to confirm presence and record the tier in `loop.run.yaml` (`literature_tiers`, presence only). A missing key just degrades to the keyless pool → WebSearch. **Initialise the sandbox** once bindings are confirmed: ``` / ├── loop.run.yaml ← resolved bindings + + grade_weights + literature_tiers ├── ledger.tsv ← header only (see Ledger) ├── literature/.cache/ ← lit_search on-disk cache └── iter1/proposal.md ← the input proposal (the baseline) ``` ## The loop `` starts at 1; iteration 1 **evaluates the unmodified proposal** (the baseline grade — no revision before it). Re-evaluate fresh every iteration: the grade comes only from a *new* ScholarEval pass on the *revised* proposal, never carried over. Copy this checklist and tick items off: - [ ] **Evaluate** — run `roles/ScholarEval.md` (spawn-or-degrade) on `iter/proposal.md` with the `` caps and ``; it writes `iter/scholareval.json` (validates against `schemas/scholareval.schema.json`). - [ ] **Grade** — as the Judge (`roles/Judge.md`) apply `rubrics/rubric.md` to `scholareval.json`: evidence gate → 0-5 per axis → weighted `grade` → hard gates → `pass` + ranked `fixes`; write `iter/verdict.json` (validates against `schemas/verdict.schema.json`). - [ ] **Log** — append one `ledger.tsv` row (see Ledger). - [ ] **Stop check** — `verdict.pass == true`, or `N == `, or grade flat for `` iterations → stop (see Stops). - [ ] **Revise** — run `roles/Reviser.md` (spawn-or-degrade) with `verdict.json`, `scholareval.json`, ``, ``, and a small revision search budget (≈`` searches + a couple of reads); it applies one focused fix batch and writes `iter/proposal.md` + `iter/revision_notes.md`. - [ ] **`N = N + 1`** and repeat. Every cited snippet must come from a real retrieval that iteration — never fabricated. When a lit tool returns `{"error","fallback"}`, fall back to WebSearch/WebFetch; never invent a paper. ## Ledger `/ledger.tsv`, tab-separated, never commas in free text: ``` iter grade pass soundness contribution evidence_quality top_fix revision_summary 1 58.0 no 3 2 4 add MM-GBSA re-scoring baseline (no revision) 2 71.0 no 4 3 4 add head-to-head vs [C2] pipeline added re-scoring + scoped affinity claim 3 82.0 yes 4 4 4 - reframed contribution around integration + new benchmark ``` The matching `scholareval.json` and `verdict.json` for each iteration live in `iter/`. Report the **best**-grade iteration when stopping on budget/plateau, not necessarily the last. Leave `ledger.tsv`, `iter*/`, and `literature/` untracked. ## Constraints - **Never fabricate citations or snippets** — every `evidence` entry comes from a real ``/WebFetch result retrieved that iteration, verbatim; the evidence gate exists to catch fabrication. - **The rubric is fixed** — the Judge never edits `rubrics/rubric.md`, so the passing threshold stays meaningful across iterations. - **Protect ``** — the Reviser may strengthen but never replace the core research question or headline contribution, and never gut a central method just to lift the grade. - **Grade through evidence, not prose** — eloquence earns nothing; grounded support and surviving novelty earn the score. - **One focused revision batch per iteration**, so each grade move is attributable. - **No installs** — the sibling `literature-search` skill is stdlib-only; never print or commit API keys (`keys.env` stays gitignored at the project root). The sandbox is self-contained — no `../` escapes. ## Stops The loop stops on the first of: - **Pass** — `verdict.pass == true`. Report the final proposal (`iter/proposal.md`), its grade, and the grade trajectory. - **Budget** — `N == `. Report the best-grade iteration as the deliverable. - **Plateau** — grade hasn't improved for `` consecutive iterations. Report the best iteration and the standing priority-1 fixes. Always end with the deliverable proposal path, its grade and pass/fail, the grade trajectory from `ledger.tsv`, and — if it did not pass — the standing blockers between this proposal and the bar.