--- name: Blue Team slug: blue-team category: Quality description: Blue Team patches a target against a concrete set of failing cases, one root-cause class at a time, while checking that previously passing cases still pass. Use it for red-team failure catalogues or CI test failures when you want the fix loop to stop only when regressions are closed. github: "https://github.com/gaasher/Agent-Loop-Skills/tree/main/loops/blue-team" language: Python stars: 166 forks: 19 install: "npx degit https://github.com/gaasher/Agent-Loop-Skills/tree/main/loops/blue-team ~/.claude/skills/blue-team" installs_to: ~/.claude/skills/blue-team source_path: loops/blue-team/SKILL.md collection_size: 25 category_size: 1897 collection_url: "https://dirskills.com/collections/gaasher/Agent-Loop-Skills" added: 2026-09-08T05:34:40.394Z last_synced: 2026-09-08T05:34:40.394Z canonical_url: "https://dirskills.com/skills/blue-team" --- # Blue Team Blue Team patches a target against a concrete set of failing cases, one root-cause class at a time, while checking that previously passing cases still pass. Use it for red-team failure catalogues or CI test failures when you want the fix loop to stop only when regressions are closed. **Install:** ```bash npx degit https://github.com/gaasher/Agent-Loop-Skills/tree/main/loops/blue-team ~/.claude/skills/blue-team ``` ## README # Blue Team A **defensive fixer** loop — the inverse of `red-team`. The artifact is the **target, now writable**; the feedback signal is two-part, like `optimize-loop`: a **gate that must hold** (nothing that passed before regresses) and a **metric that must drop** (the count of open failure classes, toward zero). You point it at a set of **concrete failed cases** and fix them one root-cause class at a time. Each iteration you patch one class, then run `tools/verify.py`, and keep the patch only if it closes the class with no regression, else revert. You loop until every class is closed (**dry**) or the budget runs out, then hand the patch set off as a pull request. This is the *fix* half of a find→fix setup (see [Pairing](#pairing)). The failed cases come from a real source; `tools/ingest.py` normalizes any of them into one catalogue: - **`oracle` mode** — a `red-team` `failures.jsonl`: each case is an input where the target's verdict disagrees with a ground-truth **oracle**. A case is closed when target and oracle now agree; a regression is a benign `` input that newly disagrees (most often a new over-block). - **`tests` mode** — a **CI/CD test-failure report** (`pytest --junitxml` / JUnit XML, or a list of failing node ids): each case is a failing test. A case is closed when its test now passes; a regression is any *other* test that was passing and now fails. ## When to use Use to fix a concrete set of failing cases in code or a guardrail/classifier/filter/prompt/API the user owns — a red-team catalogue, or the failing tests from a CI run — driving the open-class count to zero without breaking what worked. A `class` is the root-cause group the loop closes as a unit (a red-team technique, or a CI failure area / test class). Default: pick the mode that matches the source (`oracle` for red-team, `tests` for CI/CD). Escape hatch: in `oracle` mode with no separate functional test suite, the `` alone is the regression guard; in `tests` mode the suite's own previously-passing tests are the guard. Not for finding new failures (run `red-team`), and not for editing the ground truth (the oracle, the tests, or the holdout). ## Setup Resolve bindings interactively. If `loop.run.yaml` exists, load it, confirm the values in one line, and skip to the loop. Otherwise: on Claude Code (the `AskUserQuestion` tool is available) infer a likely value per binding and recommend it; on other hosts ask each as a quoted prompt. Then write `loop.run.yaml` and confirm before creating any other files. Two worked configs: `examples/run.example.yaml` (oracle mode) and `examples/tests.run.yaml` (tests mode). | binding | meaning | default | how to infer | |---|---|---|---| | `` | where the failed cases come from: `oracle` (red-team) or `tests` (CI/CD) | — | red-team `failures.jsonl` → `oracle`; failing pytest/JUnit → `tests` | | `` | the file(s) the loop may edit to fix the target | — | the source/guardrail/classifier behind the failures | | `` | the failed cases to close, JSONL; build it with `tools/ingest.py` (see below) | `/catalogue.jsonl` | red-team's ``, or a JUnit report | | `` | *(oracle mode)* ground-truth verdict (frozen), same stdin→verdict contract as red-team | — | a reference checker / policy impl | | `` | *(oracle mode)* benign inputs that must keep passing (regression guard) | `/holdout.jsonl` | known-good inputs the oracle agrees on | | `` | *(tests mode)* runs the suite and writes a JUnit XML; regressions read from it | — | `pytest --junitxml=` (or any runner that emits JUnit) | | `` | *(tests mode)* path to the JUnit XML `` writes | `/junit.xml` | — | | `` | `branches` (one commit per kept fix → feeds the PR) or `snapshots` (folder per iter) | `branches` | dirty / non-git tree → snapshots | | `` | branch the fixes land on and the PR opens from | `blue-team/` | today's date as `` | | `` | where snapshots + the ledger live | `./sandbox` | — | | `` | max iterations | 8 | — | | `` | give up on one class after N failed attempts → mark it a residual | 3 | — | `` is this skill's installed folder; substitute the real path when writing `loop.run.yaml`. **Build the catalogue first** with `tools/ingest.py`, which normalizes any source into `{id, ..., class}`: ``` python3 /tools/ingest.py --from red-team --in --out # oracle mode python3 /tools/ingest.py --from junit --in --out # tests mode ``` **The signal** each iteration is `tools/verify.py`, in the mode matching ``: ``` # oracle mode — runs , e.g. "python3 ./guardrail.py" python3 /tools/verify.py --target "" --oracle "" \ --catalogue --holdout # tests mode — writes the JUnit report verify.py then reads python3 /tools/verify.py --test-cmd "" --junit --catalogue ``` Either way it prints one JSON object: `{mode, open_classes, closed_classes, open_count, closed_count, regressions, regression_count, still_failing}`. ## The loop Copy this checklist and tick items off: - [ ] Iteration 0 — baseline: run `tools/verify.py` in the `` mode; record the **open classes** (should match the catalogue) as the current state and confirm `regression_count` is 0 — if it is not, the catalogue or holdout is dirty, so fix that before fixing the target. Log the baseline row. Save a pristine copy of `` to `/iter0/` (snapshots mode) or note the branch base (branches mode) — this is the **baseline** the final handoff diffs against, and also the snapshot iteration 1 reverts to. - [ ] In `branches` mode, open the run on a fresh branch: `git checkout -b `. - [ ] For iteration N (≥1): snapshot the **current, pre-patch** `` to `iter/` (or note the git HEAD) *before* editing, so a discard can restore exactly this state. - [ ] Pick **one** open class. Read its `still_failing` examples + the suggested fix from the catalogue, and patch `` at the **root cause** — one fix should close *all* payloads of that class (e.g. normalize case once, not per-keyword). One class per iteration so each delta is attributable. - [ ] Check the signal: run `tools/verify.py`. **Discard** — restore the snapshot / `git reset --hard` — if `regression_count > 0` (the gate) or the targeted class is still open. `verify.py`'s regression check *is* the gate: in oracle mode a regression is a newly-broken `` case (e.g. a fix that closes a bypass by over-blocking benign inputs); in tests mode it is any previously-passing test the patch broke. - [ ] **Keep** if there are no regressions and `open_count` strictly dropped. In `branches` mode commit it: `git commit -am "close : "`. Append a ledger row. - [ ] If a class resists `` attempts, mark it an **open residual** and move on rather than thrashing. Stop when `open_count` = 0 (dry), at ``, or when every remaining class is a residual. `` counts **attempts** (each keep *or* discard is one iteration), not classes closed — a discard still consumes the budget. On stop, restore the working files to the **best** iteration (most classes closed, zero regressions) and report: classes closed vs residual, the failures resolved (oracle mode: the bypass/over-block split), and regressions avoided. Then open the pull request (see [Handoff](#handoff-the-pull-request)). **Fix toolkit.** In `tests` mode the patches are ordinary bug-fixes, grouped by failure area and applied one area per iteration. In `oracle` mode (hardening a guardrail/filter), reach for these root-cause patterns, mirroring red-team's attack toolkit: - **Normalize before matching** — case-fold, de-leet (homoglyph/leet → letters), strip spacing and punctuation, NFKC-normalize unicode. One normalization step closes case / leetspeak / spacing classes. - **Broaden the policy** — add missing synonyms/expansions to the blocked set (the `missing-synonym` class), keyed to the oracle's categories, not ad-hoc strings. - **Tighten over-broad rules** — scope a match to whole words / the right context so benign inputs stop tripping it (the `overblock` class), the most common source of regressions. Mind the **interaction order** (both modes): make the narrowing/over-broad fix *before* a sweeping one. A fix that strips separators (closing `spacing`) can re-collapse a benign input into an over-broad substring and silently reopen an `overblock` class — and likewise a broad code change can reopen a test a narrower fix had to protect. Fix the narrow/over-broad case first, then generalize. ## Ledger `/ledger.tsv`, tab-separated, never commas in the description. `regr` = `regression_count` this iteration (the gate: 0 is clean); `status` ∈ {`keep`,`discard`,`baseline`,`residual`}. Header `iter class_targeted regr open_classes status description`: ``` iter class_targeted regr open_classes status description 0 - 0 5 baseline catalogue: 5 open classes 1 case-bypass 0 4 keep case-fold the input before matching 2 leetspeak 1 4 discard de-leet regex also over-blocked a holdout input (regression) 3 leetspeak 0 3 keep de-leet via translate table, holdout clean 4 missing-synonym 0 2 keep add passphrase/credentials/api-key to the policy set 5 overblock 0 1 keep require whole-word "secret key", not bare "secret" 6 spacing 0 0 keep strip non-alphanumerics before matching — dry ``` Report the **best** iteration (open_classes lowest with `regr` 0), not necessarily the last. ## Constraints - **Only edit ``.** The ground truth — the oracle + `` (oracle mode) or the test suite (tests mode) — and `tools/verify.py` are frozen; editing what measures the fix manufactures a pass (same rule as red-team and optimize-loop). If the oracle or a test is itself wrong, that is a finding to report, not something to patch here. - **Fix the root cause, not the payload.** One fix should close every item of a class; patching a single example string while siblings still fail means the class is not closed. This mirrors red-team's class accounting, so the two loops agree on what "closed" means. - **The regression gate is non-negotiable.** A patch that breaks something that passed before — a new over-block/bypass (oracle mode) or a previously-passing test (tests mode) — is a regression, not progress; revert it regardless of how many classes it closes. Prefer a narrower fix over a sweeping one. - **One class per iteration**, so each delta is attributable and a bad fix is cheap to revert. - Keep changes within the project's existing dependency set (stdlib is fine); stay inside the repo and ``; do not pause the loop to ask whether to continue — run until dry, budget, or residuals. ## Handoff: the pull request The deliverable is a human-reviewable **pull request** — the communication interface to whoever owns the target (a human keeps final approval, as with Copilot Autofix). On stop, with the working tree at the best iteration: - In `branches` mode each kept fix is already a commit on `` whose message names the class + the one-line fix. Push and open the PR with `gh pr create`, body = the ledger (catalogue → fixes, classes closed, residuals listed, regressions avoided) and one reproducible example per closed class. - **Confirm once before opening the PR** — pushing a branch and creating a PR is outward-facing; never auto-push silently. - **Degrade, don't fail:** if there is no remote or `gh` is unavailable/unauthed, still produce the handoff — leave the commits on ``, write `git format-patch` output and a `PR_BODY.md` into ``, and tell the user the single command to open the PR themselves. In `snapshots` mode (no git), emit a unified `diff` of `` against the `iter0/` baseline copy, plus `PR_BODY.md`, instead. ## Pairing This skill is the **fixer** — the back half of a find→fix loop. It assumes a catalogue already exists; by itself it closes classes but does not search for new ones. 1. **Find** — `red-team` runs against the **frozen** target → `failures.jsonl` (distinct classes). 2. **Fix** *(this loop)* — patch the target to close those classes under the gate, **between** red-team runs, never inside one. 3. **Re-verify** — a **fresh** `red-team` run against the patched target confirms each class is closed and surfaces any new class the fix introduced (the regression gate already guards the over-block direction within this loop). Keep the two agents independent — the attacker that wrote the catalogue should not grade its own patch. The `purple-team` loop orchestrates the full find → fix → re-verify cycle until a fresh find stays dry; this skill deliberately covers only the fix phase.