--- name: Auto Paper Improvement Loop slug: auto-paper-improvement-loop category: Writing description: Auto Paper Improvement Loop iteratively improves a generated paper via external LLM review, fix implementation, and recompilation for two rounds. Use it after paper generation to catch structural and presentation issues. github: "https://github.com/wanshuiyin/Auto-claude-code-research-in-sleep/tree/main/skills/auto-paper-improvement-loop" language: Python stars: 14656 forks: 1291 install: "npx degit https://github.com/wanshuiyin/Auto-claude-code-research-in-sleep/tree/main/skills/auto-paper-improvement-loop ~/.claude/skills/auto-paper-improvement-loop" installs_to: ~/.claude/skills/auto-paper-improvement-loop source_path: skills/auto-paper-improvement-loop/SKILL.md collection_size: 25 category_size: 1012 collection_url: "https://dirskills.com/collections/wanshuiyin/Auto-claude-code-research-in-sleep" added: 2026-08-14T07:13:00.923Z last_synced: 2026-08-14T07:13:00.923Z canonical_url: "https://dirskills.com/skills/auto-paper-improvement-loop" --- # Auto Paper Improvement Loop Auto Paper Improvement Loop iteratively improves a generated paper via external LLM review, fix implementation, and recompilation for two rounds. Use it after paper generation to catch structural and presentation issues. **Install:** ```bash npx degit https://github.com/wanshuiyin/Auto-claude-code-research-in-sleep/tree/main/skills/auto-paper-improvement-loop ~/.claude/skills/auto-paper-improvement-loop ``` ## README # Auto Paper Improvement Loop: Review β†’ Fix β†’ Recompile > πŸ”’ **Do not wrap this skill in `/loop`, `/schedule`, or `CronCreate`.** It > already loops internally (review β†’ fix β†’ recompile) with its own round > structure and a deliberate fresh-reviewer bias guard each round (no > `codex-reply`). Re-asking it to "improve the paper" on a > wall-clock timer produces no new signal β€” quality changes when the *review* > changes, not when the clock ticks β€” and a timed re-run that also accepts its > own output to decide when to stop crosses into self-acquittal > (`acceptance-gate.md`). Schedule the *external wait that precedes it*, not the > improvement loop. See > [`shared-references/external-cadence.md`](../shared-references/external-cadence.md). Autonomously improve the paper at: **$ARGUMENTS** ## Context This skill is designed to run **after** Workflow 3 (`/paper-plan` β†’ `/paper-figure` β†’ `/paper-write` β†’ `/paper-compile`). It takes a compiled paper and iteratively improves it through external LLM review. Unlike `/auto-review-loop` (which iterates on **research** β€” running experiments, collecting data, rewriting narrative), this skill iterates on **paper writing quality** β€” fixing theoretical inconsistencies, softening overclaims, adding missing content, and improving presentation. ## Constants - **MAX_ROUNDS = 2** β€” Two rounds of reviewβ†’fixβ†’recompile. Empirically, Round 1 catches structural issues (4β†’6/10), Round 2 catches remaining presentation issues (6β†’7/10). Diminishing returns beyond 2 rounds for writing-only improvements. - **REVIEWER_MODEL = `gpt-5.6-sol`** β€” Model used via Codex MCP for paper review. - **REVIEWER_BIAS_GUARD = true** β€” When `true`, every review round uses a fresh `mcp__codex__codex` thread with no prior review context. Never use `mcp__codex__codex-reply` for review rounds. Set to `false` only for deliberate debugging of the legacy behavior. **Empirical evidence:** running the same paper with `codex-reply` + "since last round we did X" prompts inflated scores from real 3/10 β†’ fake 8/10 across multiple rounds; switching to fresh threads recovered the true 3/10 assessment. - **REVIEW_LOG = `PAPER_IMPROVEMENT_LOG.md`** β€” Cumulative log of all rounds, stored in paper directory. - **HUMAN_CHECKPOINT = false** β€” When `true`, pause after each round's review and present score + weaknesses to the user. The user can approve fixes, provide custom modification instructions, skip specific fixes, or stop early. When `false` (default), runs fully autonomously. - **EDIT_WHITELIST = `null`** β€” Optional path to a YAML/JSON whitelist file constraining which paths and operations the fix-implementation step may touch. When `null` (default), all edits proceed unconstrained. When set via `β€” edit-whitelist ` (also accepts `β€” edit_whitelist `), the loop loads the file at startup and consults it before each edit; rejected edits are logged to `PAPER_IMPROVEMENT_LOG.md` rather than silently dropped. See "Optional: Edit Whitelist" below. > πŸ’‘ Override: `/auto-paper-improvement-loop "paper/" β€” human checkpoint: true` ## Optional: Style reference (`β€” style-ref: `, opt-in) Lets the user steer **structural fixes only** during improvement (section reordering hints, paragraph length nudges, figure density adjustments) toward a reference paper. **Default OFF β€” when the user does not pass `β€” style-ref`, do nothing differently from before.** Only when `β€” style-ref: ` appears in `$ARGUMENTS`, run the helper FIRST, before the loop starts: ```bash # Resolve $STYLE_HELPER via the canonical strict-safe chain (see # shared-references/integration-contract.md Β§2). Policy A β€” gate: # unresolved helper means --style-ref cannot be satisfied, so abort. cd "$(git rev-parse --show-toplevel 2>/dev/null || pwd)" || exit 1 if [ -z "${ARIS_REPO:-}" ] && [ -f .aris/installed-skills.txt ]; then ARIS_REPO=$(awk -F'\t' '$1=="repo_root"{print $2; exit}' .aris/installed-skills.txt 2>/dev/null) || true fi if [ -z "${ARIS_REPO:-}" ] && [ -f "$HOME/.aris/repo" ]; then ARIS_REPO=$(cat "$HOME/.aris/repo" 2>/dev/null) || true fi STYLE_HELPER=".aris/tools/extract_paper_style.py" [ -f "$STYLE_HELPER" ] || STYLE_HELPER="tools/extract_paper_style.py" [ -f "$STYLE_HELPER" ] || { [ -n "${ARIS_REPO:-}" ] && STYLE_HELPER="$ARIS_REPO/tools/extract_paper_style.py"; } [ -f "$STYLE_HELPER" ] || { echo "ERROR: extract_paper_style.py not resolved at .aris/tools/, tools/, \$ARIS_REPO/tools/, or via ~/.aris/repo." >&2 echo " Fix: rerun bash tools/install_aris.sh or smart_update.sh (refreshes ~/.aris/repo), export ARIS_REPO, or copy the helper to tools/." >&2 echo " --style-ref cannot be satisfied; aborting." >&2 exit 1 } STYLE_STATUS=0 CACHE=$(python3 "$STYLE_HELPER" --source "") || STYLE_STATUS=$? case "$STYLE_STATUS" in 0) ;; # use $CACHE/style_profile.md as structural guidance for the FIX phase only 2) echo "warning: style-ref skipped (missing optional dep)" >&2 ;; 3) echo "error: --style-ref source failed; aborting loop" >&2 ; exit 1 ;; *) echo "error: helper failed unexpectedly; aborting loop" >&2 ; exit 1 ;; esac ``` Sources accepted: local TeX dir / file, local PDF, arXiv id, http(s) URL. Overleaf URLs/IDs are rejected β€” clone via `/overleaf-sync setup ` first and pass the local clone path. **Strict rules** (full contract in `tools/extract_paper_style.py` docstring): - Use `style_profile.md` only during the **fix-implementation** phase, to nudge structural choices when applying reviewer feedback. Reviewer feedback always takes precedence; style ref is tie-breaker for *how* to apply a fix, not *whether* to apply it. - **Never copy prose, claims, examples, or terminology** from anything reachable through the cache when implementing fixes. - **Never pass `β€” style-ref` (or the cache contents) to the GPT-5.6-Sol reviewer sub-agent.** The Reviewer Independence Protocol below requires reviewers see only the artifact and the user's prompt β€” leaking the style ref would contaminate the review with author-side context. **This is the most critical invariant in this skill.** ## Optional: Edit Whitelist (`β€” edit-whitelist `, opt-in) Lets the caller hard-constrain which files and operations the **fix-implementation** step (Step 3 / Step 6) is allowed to touch. **Default OFF β€” when the user does not pass `β€” edit-whitelist` (or the alias `β€” edit_whitelist`), the loop applies all reviewer-driven edits without restriction, exactly as before.** This is the parameter that upstream pipelines (e.g. `/resubmit-pipeline` Phase 2) use to enforce text-only resubmit microedits: no `.bib` mutations, no `.sty` / `.bst` mutations, no edits to prior-submission directories, no new `\cite{...}`, no new theorem environments, no new numerical claims. ### Schema The whitelist file is YAML or JSON. All four sections are optional: ```yaml allowed_paths: - sec/*.tex - main.tex - figures/*.tex forbidden_paths: - "**/*.bib" - "**/*.sty" - "**/*.bst" - "../OldSubmission/**" forbidden_operations: - new_cite # blocks \cite{...}, \citep{...}, \citet{...}, \citeauthor{...} additions - new_bibitem # blocks \bibitem{...} additions - new_theorem_env # blocks \begin{theorem|lemma|proposition|corollary} additions - numerical_claim # blocks adding new numbers / percentages / metrics forbidden_deletions: # operations that block REMOVALS, not additions - delete_existing_cite # blocks removal of \cite{...} from the body (use citation-audit --soft-only instead) - delete_theorem_env # blocks removal of an existing \begin{theorem|...} block requires_user_approval_for: # operations that don't auto-reject but pause for explicit user OK - rewrite_abstract # paraphrasing the entire abstract triggers a checkpoint - rewrite_intro_first_para - delete_section max_edits_per_round: 30 # hard cap on number of accepted edits per round (rejections are not counted; if cap is hit, remaining proposed edits are deferred to the next round with a warning) rationale: "Resubmit mode: text-only microedits, paper structure frozen by user constraint." ``` ### Resolution rules - **`allowed_paths` empty AND `forbidden_paths` empty** β†’ whitelist is a no-op (advisory: the file is loaded and `rationale` echoed to the log, but no path filtering is applied). - **`allowed_paths` empty, `forbidden_paths` non-empty** β†’ all paths NOT matched by `forbidden_paths` are mutable. - **`allowed_paths` non-empty, `forbidden_paths` empty** β†’ only paths matching `allowed_paths` are mutable. - **Both non-empty** β†’ an edit is allowed iff the target matches `allowed_paths` AND does NOT match `forbidden_paths`. `forbidden_paths` always wins on overlap. - **`forbidden_operations` missing or empty** β†’ no operation-level guard; only path-level filtering applies. ### Glob semantics Use bash `extglob` / Python `fnmatch.fnmatch` semantics. `**` matches any depth (zero or more directory segments). Patterns are matched against the path **relative to the paper directory** (e.g. `paper/sec/intro.tex` matches `sec/*.tex` when paper-directory is `paper/`). ### Forbidden-operation detectors For each candidate edit's diff (the new lines being added β€” deletions are exempt), the loop runs these regex checks and rejects if any forbidden operation matches: | Operation | Detector (added lines only) | |-----------|------------------------------| | `new_cite` | `\\cite[a-zA-Z]*\{[^}]+\}` (catches `\cite`, `\citep`, `\citet`, `\citeauthor`, `\citeyear`, `\citealp`, etc.) | | `new_bibitem` | `\\bibitem\{[^}]+\}` | | `new_theorem_env` | `\\begin\{(theorem|lemma|proposition|corollary)\*?\}` | | `numerical_claim` | New token matching `\b\d+(\.\d+)?%?\b` that did NOT appear in the deleted/replaced lines (i.e. genuinely new numbers, not edits to existing ones) | ### Behavior at loop start (before Round 1 fix-implementation) 1. If `β€” edit-whitelist ` is present in `$ARGUMENTS`, set `EDIT_WHITELIST = `. 2. Load the file (`yaml.safe_load`; if it fails, fall back to `json.loads`). On load failure, abort the loop with a clear error β€” do NOT silently proceed unconstrained. 3. Echo `rationale` (if present) into `PAPER_IMPROVEMENT_LOG.md` under a new "Edit Whitelist" preamble section so the audit trail records why edits were constrained. ### Behavior during fix-implementation (Steps 3 and 6) Before applying each proposed edit: 1. Resolve target file path relative to the paper directory. 2. Path check: if `allowed_paths` is non-empty, target must match at least one pattern. Then if `forbidden_paths` is non-empty, target must NOT match any pattern. If either fails β†’ reject as `path` violation. 3. Operation check: build the unified diff (or just the set of newly-added lines) for the proposed edit. For each entry in `forbidden_operations`, run its detector on the added lines. If any detector matches β†’ reject as `operation` violation. 4. If all checks pass, apply the edit normally. 5. If rejected, append an entry to `PAPER_IMPROVEMENT_LOG.md` under a `## Rejected by edit_whitelist (Round N)` heading with this schema: ``` - file: reason: path | operation pattern: reviewer_concern: ``` 6. Continue with the remaining edits in the round. Do NOT abort the whole round on a single rejection. ### End-of-round surfacing At the end of each round (after the recompile, before moving to the next round), if any edits were rejected during that round's fix step: - Print a one-line summary to the round's checkpoint output: `Edit whitelist rejected N edits this round (M path, K operation). See PAPER_IMPROVEMENT_LOG.md "Rejected by edit_whitelist (Round N)".` - If `HUMAN_CHECKPOINT = true`, include the rejection list in the checkpoint shown to the user before they approve next-round fixes. ### Example invocations ```bash # Resubmit-pipeline Phase 2 caller (text-only mode): /auto-paper-improvement-loop "paper/" β€” edit-whitelist .resubmit/edit_whitelist.yaml # Aliased form is accepted: /auto-paper-improvement-loop "paper/" β€” edit_whitelist .resubmit/edit_whitelist.yaml # Combined with other flags: /auto-paper-improvement-loop "paper/" β€” human checkpoint: true β€” edit-whitelist constraints.yaml ``` ### Rationale Without a whitelist, the loop's reviewer-driven fix step is free to add citations, introduce new theorem environments, or tweak numerical claims β€” all of which are reasonable for first-submission polish but **forbidden** in resubmit / camera-ready / rebuttal-only modes where the paper structure is frozen by external constraint. Routing those constraints through a first-class parameter (rather than relying on the LLM to "remember" not to do them) makes the constraint enforceable, auditable via `PAPER_IMPROVEMENT_LOG.md`, and visible to the user at each round's checkpoint. ## Inputs 1. **Compiled paper** β€” `paper/main.pdf` + LaTeX source files 2. **All section `.tex` files** β€” concatenated for review prompt ## State Persistence (Compact Recovery) If the context window fills up mid-loop, Claude Code auto-compacts. To recover, this skill writes `PAPER_IMPROVEMENT_STATE.json` after each round: ```json { "current_round": 1, "threadId": "019ce736-...", "last_score": 6, "status": "in_progress", "timestamp": "2026-03-13T21:00:00" } ``` **On startup**: if `PAPER_IMPROVEMENT_STATE.json` exists with `"status": "in_progress"` AND `timestamp` is within 24 hours, read it + `PAPER_IMPROVEMENT_LOG.md` to recover context, then resume from the next round. Otherwise (file absent, `"status": "completed"`, or older than 24 hours), start fresh. **After each round**: overwrite the state file. **On completion**: set `"status": "completed"`. ## Reviewer Independence Protocol The reviewer must be context-naive on every round. Prior-round summaries, fix lists, and executor explanations are not evidence; they are a source of confirmation bias. If the reviewer is told what changed, scores tend to drift upward even when the manuscript itself has not materially improved. Rules: - Every round starts with `mcp__codex__codex`, not `mcp__codex__codex-reply`. - Never pass a prior threadId into the next review prompt. - Never include "since last round", "we fixed", "after applying", or any fix summary in the reviewer prompt. - The only acceptable evidence of improvement is the current `.tex` source and compiled PDF. - If a fix cannot be observed in the files, the reviewer should not be told it happened. - If recovery metadata is needed, store the returned threadId for crash recovery only; do not use it to preserve review context. Set `REVIEWER_BIAS_GUARD = false` only if you explicitly want the legacy, context-carrying behavior for debugging. ## Workflow ### Step 0: Preserve Original ```bash cp paper/main.pdf paper/main_round0_original.pdf ``` ### Step 1: Collect Paper Text Concatenate all section files into a single text block for the review prompt: ```bash # Collect all sections in order for f in paper/sections/*.tex; do echo "% === $(basename $f) ===" cat "$f" done > /tmp/paper_full_text.txt ``` ### Step 2: Round 1 Review Send the full paper text AND compiled PDF to GPT-5.6-Sol xhigh: ``` mcp__codex__codex: model: gpt-5.6-sol config: {"model_reasoning_effort": "xhigh"} prompt: | You are reviewing a [VENUE] paper. Please provide a detailed, structured review. ## Paper Files: - LaTeX source: [list all section .tex files] - Compiled PDF: paper/main.pdf - Figures: [list figure files] Read BOTH the LaTeX source (for content/logic) AND the compiled PDF (for visual presentation). ## Review Instructions Please act as a senior ML reviewer ([VENUE] level). Provide: 1. **Overall Score** (1-10, where 6 = weak accept, 7 = accept) 2. **Summary** (2-3 sentences) 3. **Strengths** (bullet list, ranked) 4. **Weaknesses** (bullet list, ranked: CRITICAL > MAJOR > MINOR) 5. **For each CRITICAL/MAJOR weakness**: A specific, actionable fix 6. **Missing References** (if any) 7. **Visual Review** (from the PDF): - Figure quality: readable? labels legible? colors distinguishable in grayscale? - Figure-caption alignment: does each caption match its figure? - Layout: orphaned headers, awkward page breaks, figures far from references? - Table formatting: aligned columns, consistent decimals, bold for best results? - Visual consistency: same color scheme across all figures? 8. **Verdict**: Ready for submission? Yes / Almost / No Focus on: theoretical rigor, claims vs evidence alignment, writing clarity, self-containedness, notation consistency, AND visual presentation quality. ``` Save the threadId for Round 2. ### Step 2b: Human Checkpoint (if enabled) **Skip if `HUMAN_CHECKPOINT = false`.** Present the review results and wait for user input: ``` πŸ“‹ Round 1 review complete. Score: X/10 β€” [verdict] Key weaknesses (by severity): 1. [CRITICAL] ... 2. [MAJOR] ... 3. [MINOR] ... Reply "go" to implement all fixes, give custom instructions, "skip 2" to skip specific fixes, or "stop" to end. ``` Parse user response same as `/auto-review-loop`: approve / custom instructions / skip / stop. ### Step 3: Implement Round 1 Fixes Parse the review and implement fixes by severity: **Priority order:** 1. CRITICAL fixes (assumption mismatches, internal contradictions) 2. MAJOR fixes (overclaims, missing content, notation issues) 3. MINOR fixes (if time permits) **Edit-whitelist gate (if set):** If `EDIT_WHITELIST` is set, before applying each proposed edit, check the target path against `allowed_paths` / `forbidden_paths` and the new-lines diff against `forbidden_operations` per the "Optional: Edit Whitelist" section. Rejections are logged to `PAPER_IMPROVEMENT_LOG.md` under `## Rejected by edit_whitelist (Round 1)` with file, reason (`path` or `operation`), the offending pattern, and the original reviewer concern. The loop continues with remaining edits β€” a rejection never aborts the round. Surface a rejection summary at the end of the round. **Common fix patterns:** | Issue | Fix Pattern | |-------|-------------| | Assumption-model mismatch | Rewrite assumption to match the model, add formal proposition bridging the gap | | Overclaims | Soften language: "validate" β†’ "demonstrate practical relevance", "comparable" β†’ "qualitatively competitive" | | Missing metrics | Add quantitative table with honest parameter counts and caveats | | Theorem not self-contained | Add "Interpretation" paragraph listing all dependencies | | Notation confusion | Rename conflicting symbols globally, add Notation paragraph | | Missing references | Add to `references.bib`, cite in appropriate locations | | Theory-practice gap | Explicitly frame theory as idealized; add synthetic validation subsection | | Proof gap (theory papers) | Run `/proof-checker` if PROOF_AUDIT.md doesn't exist yet; fix FATAL/CRITICAL issues | | Writing clutter / passive voice | Apply sciwrite 5-pass audit: clutter extraction β†’ active voice β†’ sentence architecture β†’ keyword consistency β†’ numerical integrity. See `paper-write` Step 5 | | Number mismatch (paper vs results) | Run `/paper-claim-audit` if PAPER_CLAIM_AUDIT.md doesn't exist; fix any `number_mismatch` or `aggregation_mismatch` claims | | Keyword inconsistency | The "Banana Rule": if Methods says "obese group", Results must not say "heavier group". Extract key terms, verify consistency across all sections | ### Step 4: Recompile Round 1 ```bash cd paper && latexmk -C && latexmk -pdf -interaction=nonstopmode -halt-on-error main.tex cp main.pdf main_round1.pdf ``` Verify: 0 undefined references, 0 undefined citations. ### Step 4.5: Restatement Regression Test After every recompilation, rer