--- name: Optimize Loop slug: optimize-loop category: AI Engineering description: Optimize Loop runs an evaluator-optimizer cycle that keeps a correctness gate passing while reducing a measured cost. Use it to refactor code or tune SQL when results must stay unchanged but complexity or latency should drop. github: "https://github.com/gaasher/Agent-Loop-Skills/tree/main/loops/optimize-loop" language: Python stars: 166 forks: 19 install: "npx degit https://github.com/gaasher/Agent-Loop-Skills/tree/main/loops/optimize-loop ~/.claude/skills/optimize-loop" installs_to: ~/.claude/skills/optimize-loop source_path: loops/optimize-loop/SKILL.md collection_size: 25 category_size: 3670 collection_url: "https://dirskills.com/collections/gaasher/Agent-Loop-Skills" added: 2026-09-08T05:34:43.624Z last_synced: 2026-09-08T05:34:43.624Z canonical_url: "https://dirskills.com/skills/optimize-loop" --- # Optimize Loop Optimize Loop runs an evaluator-optimizer cycle that keeps a correctness gate passing while reducing a measured cost. Use it to refactor code or tune SQL when results must stay unchanged but complexity or latency should drop. **Install:** ```bash npx degit https://github.com/gaasher/Agent-Loop-Skills/tree/main/loops/optimize-loop ~/.claude/skills/optimize-loop ``` ## README # Optimize Loop An **evaluator-optimizer** loop with a **pluggable correctness gate + minimized metric**. The artifact is some editable thing (a code module or a SQL query); the feedback signal is two-part: a **bound gate that must pass** (behaviour/results unchanged) and a **bound metric that must drop** (the cost you minimize). You apply one change, check the gate, measure the metric, and keep the change only if the gate passes AND the metric improves — otherwise you revert. Repeat until the metric stops improving or the budget runs out. Once the loop starts, do not pause for permission. Two ready bindings ship in `tools/` (both vendored, stdlib-only): - **code mode** — gate: `` (the test suite) exits 0; metric: `tools/metrics.py` prints `complexity` (primary), `max_nesting`, `loc` (lexicographic tie-breakers). Lower is better. - **sql mode** — gate: the result-set `hash` from `tools/bench.py` matches the baseline; metric: the same tool's `median_ms`. Lower is better. The gate is non-negotiable in both modes: a change that fails it is a regression, not an improvement. Never edit the ground truth (the tests / `tools/metrics.py` in code mode, the database / `tools/bench.py` in sql mode) — editing what measures you to move the number defeats the loop. ## When to use Use when there is a clear correctness bound to *hold* and a number to *minimize*: refactoring code that has a passing test suite (cut complexity), or tuning a SQL query that has a fixed result-set (cut latency). The default is the matching shipped tool; the escape hatch is to bind any `` that exits 0 on pass and any `` that prints a single number to minimize (e.g. a linter's issue count, or a non-SQLite engine's timing + result fingerprint). Not for adding features or fixing bugs — those *intend* to change behaviour, which this loop is built to forbid. ## Setup **Resolve bindings interactively.** If `loop.run.yaml` exists in the working dir, load it, confirm the values back in one line, and skip to the loop. Otherwise pick `` first (it selects the gate + metric), then on Claude Code (the `AskUserQuestion` tool is available) infer a likely value for each binding and present it as the recommended option; on other hosts ask each as a quoted plain-text prompt. Then write `loop.run.yaml` and confirm the values before creating any other files. `` and `` are the pluggable core: bind them per `` from the table. In sql mode one bench command supplies both — its `hash` is the gate, its `median_ms` is the metric. | binding | meaning | default | how to infer | |---|---|---|---| | `` | `code` (refactor under test) or `sql` (query, fixed results) | — | the artifact's kind | | `` | the file(s) the loop may change | — | code: source files (not tests/configs/the tool); sql: the query file (+ optional indexes file) | | `` | the **bound gate** that must PASS, else revert | — | code: the test command (exits 0 on pass); sql: implicit — candidate `hash` must equal the baseline `hash` from the bench command | | `` | the **bound metric** printing a number to minimize | — | code: `python3 /tools/metrics.py ` (→ `complexity`, then `max_nesting`, `loc`); sql: `python3 /tools/bench.py --db --query --setup --repeat 5` (→ `median_ms`, `hash`) | | `` | where snapshots + the ledger live | `./sandbox` | — | | `` | max iterations (hard cap) | 8 | — | | `` | stop after N consecutive no-improvement iterations | 3 | — | `` is this skill's installed folder; substitute the real path when writing `loop.run.yaml`. For non-default engines/languages, bind any ``/`` meeting the contract above. Two worked configs: `examples/refactor.run.yaml` (code) and `examples/sql.run.yaml` (sql). ## The loop (until plateau or ``) Copy this checklist and tick items off: - [ ] Iteration 0 — baseline: run `` (code) — if not green, stop (the loop needs a passing gate to protect behaviour). Run ``; record the metric as the current best, and in sql mode record the baseline `hash` as the correctness reference. Log the baseline row. - [ ] Snapshot every file in `` to `/iter/` so the iteration reverts. - [ ] Apply **one** focused change (see the per-mode ideas below) — one idea per iteration so each metric delta is attributable. - [ ] Check the gate: code — run ``; sql — read the candidate's `hash` from ``. - [ ] If the gate fails (tests non-zero / `hash` ≠ baseline / the tool errored), **discard**: restore from the snapshot, log the reason, continue. - [ ] Measure the metric and compare to the best: code — the triple `(complexity, max_nesting, loc)` **lexicographically** (complexity first; only on a tie consult `max_nesting`, then `loc`); sql — `median_ms`, keeping only on a margin clear of timing noise (default ≥ 3% relative). - [ ] **Keep** if the metric strictly improves the best (update the best, leave the files in place), else **discard** (restore from the snapshot). - [ ] Append a ledger row; stop on plateau (``) or ``. **Change ideas — code mode:** flatten nested `if/else` into guard clauses, replace a hand-rolled loop with a stdlib call (`sum`, `min`, `max`, `statistics.*`), collapse duplicated branches, remove dead code. Preserve public behaviour — names, signatures, return shapes, raised exceptions; the test suite is the contract. **Change ideas — sql mode:** add an index to `` covering filtered/joined/grouped columns; rewrite the query (correlated subquery → `JOIN` + `GROUP BY`, hoist a repeated computation, replace `SELECT *` with needed columns, push a filter earlier, drop a redundant `DISTINCT`/sort). The `hash` is over the multiset of rows, so it does **not** catch a changed row **order** — if `ORDER BY` is part of the contract, eyeball that the rewrite preserves it. **Lexicographic keep (code mode), current best `(18, 3, 64)`:** `(15, 3, 45)` keep (lower complexity); `(18, 2, 70)` keep (tie complexity, lower nesting); `(18, 3, 61)` keep (tie, fewer lines); `(18, 3, 64)` discard (no progress); `(19, 1, 20)` discard (higher complexity outweighs simpler nesting/loc). **Plateau counting:** increment the no-improvement counter on *every* iteration that does not set a new best — discarded for a failed gate, a broken change, or an insufficient metric gain — and reset it to 0 on each keep. `` fruitless iterations in a row ends the run; `` is the hard cap. On stop, restore the working files to the **best** iteration (if the latest was a discard) and report: baseline vs best metric (and, in sql mode, the speedup factor), the trajectory, and the winning change set. If you run low on ideas before the budget, look harder rather than stopping early. ## Ledger `/ledger.tsv`, tab-separated, never commas in the description. `status` ∈ {`keep`, `discard`, `baseline`}. Use the columns for the active ``. Code mode header `iter complexity max_nesting loc status description`: ``` iter complexity max_nesting loc status description 0 23 7 86 baseline unmodified module 1 19 5 78 keep flatten summarize guard clauses 2 19 5 80 discard extract helper (no complexity gain) 3 13 3 40 keep use statistics + min/max/median ``` SQL mode header `iter median_ms rows hash_ok status description` (`hash_ok` ∈ {`yes`,`no`,`-`}): ``` iter median_ms rows hash_ok status description 0 1121.06 10 - baseline correlated subquery no index 1 6.82 10 yes keep rewrite correlated subquery as JOIN + GROUP BY 2 1.18 10 yes keep add index orders(customer_id, amount) 4 0.40 10 no discard drop ORDER BY — changed result set ``` Report the **best** iteration, not necessarily the last. ## Constraints - **Only edit files in ``** — the gate's ground truth is read-only: the tests and `tools/metrics.py` (code), the database and `tools/bench.py` (sql). Editing what measures you to move the number invalidates the run. - **The correctness gate is non-negotiable.** A change that fails it (tests red, or `hash` ≠ baseline) is a regression, not an optimization — revert it regardless of the metric. A green gate after a behaviour change means the gate is too weak, not that the change is safe; prefer holding behaviour identical over trusting a thin gate. - **One change per iteration**, so each metric delta is attributable. - **Measure every candidate the same way** (code: the same ``; sql: the same `--repeat`); compare the metric, not a single noisy run. - Keep changes within the existing dependency set; do not add imports the project lacks (stdlib is fine). - The sandbox is self-contained — no `../` escapes beyond the bound ``. - Do not pause the loop to ask whether to continue; run until plateau or budget.