---
name: Agent Skill Testing
slug: agent-skill-testing
category: Quality
description: Test and measure the activation, outcomes, and routing of agent skills in isolation. Ideal for verifying skill behavior and avoiding false activations.
github: "https://github.com/MisterBrookT/skill2/tree/main/skills/skill2-test"
language: Python
stars: 10
forks: 0
install: "git clone https://github.com/MisterBrookT/skill2"
added: 2026-07-17T07:10:03.914Z
last_synced: 2026-07-17T07:10:03.914Z
canonical_url: "https://dirskills.com/skills/agent-skill-testing"
---

# Agent Skill Testing

Test and measure the activation, outcomes, and routing of agent skills in isolation. Ideal for verifying skill behavior and avoiding false activations.

**Install:** `git clone https://github.com/MisterBrookT/skill2`

## README

# Test Agent Skills

Measure behavior without inheriting current chat, global memory, or unrelated Skills.

## Ownership

- Test owns live activation, outcome, routing, and baseline evidence.
- Audit owns static structure and trigger-overlap review.
- Create owns Skill changes; Test never rewrites a failing Skill automatically.

## Layers

| Layer | Installed Skills | Measures |
| --- | --- | --- |
| Target-only | Target Skill only | Activation, outcome, baseline uplift |
| Pack | Candidate sibling Skills | Routing and false activation |

Use target-only for core outcomes. Use pack for adjacent and unrelated routing cases.

## Cases

- Core positive: target should activate and produce required artifact or answer.
- Paraphrase: same intent, different wording.
- Adjacent: sibling should own request.
- Unrelated: no package Skill should activate.
- Assertions: inspect files, output, commands, or explicit events.

Default one trial. Add repetitions only for known nondeterminism or regression confidence.

## Isolation

- Fresh session, temporary home, temporary workspace.
- Install only layer-required Skills.
- Minimal authentication/configuration; no user memory or chat history.
- Guard host-home reads and writes.
- Adapter owns harness-specific install and event parsing.
- Save raw events, final output, workspace, version, and Skill hash locally.

## Verdicts

- `pass`: activation and outcome assertions pass.
- `fail`: deterministic assertion fails.
- `inconclusive`: runner or evidence cannot support claim.
- Baseline also passes: no demonstrated deterministic uplift.
- Exact isolated `SKILL.md` read is medium-confidence activation; explicit activation event is high confidence.

Fake runner tests validate adapter plumbing, not provider behavior. Never claim unrun results.

```bash
uv run --script <skill-dir>/scripts/run -- test skills/<name> --agent <agent> --cases cases/<name>.yaml --baseline
uv run --script <skill-dir>/scripts/run -- test skills/<name> --agent <agent> --cases cases/<name>.yaml --pack
```