🧪
QualityTypeScript

AI System Quality Engineer

by PramodDutta

AI System Quality Engineer is a Quality skill for Claude Code, published by PramodDutta in qaskills.

216 stars23 forkson PramodDutta/qaskillsAdded 2026/09/04+1% in starsRepository updated 2026/08/30
View source on GitHubDownload repo ZIP
agent-browseragent-skillsai-testingclaude-codecursorcypressplaywrightqaqaskilssdetseleniumtest-automationtestingvibium
Install in seconds
Install AI System Quality Engineer
Copy AI System Quality Engineer into your Claude Code skills folder. Run the command in your terminal, or review the source on GitHub before installing.
terminal
npx degit https://github.com/PramodDutta/qaskills/tree/main/seed-skills/ai-system-quality-engineer ~/.claude/skills/ai-system-quality-engineer

Requires Node.js. Downloads this skill only — not the rest of the repository — into your Claude Code skills folder.

Without Node.js

git clone https://github.com/PramodDutta/qaskills.git

Clones the whole repository, then copy the skill’s own directory into your skills folder yourself.

In this catalog

Source file
seed-skills/ai-system-quality-engineer/SKILL.md in PramodDutta/qaskills
Installs to
~/.claude/skills/ai-system-quality-engineer
Collection
One of 25 skills cataloged from this repository
Category
Quality — 1557 skills

What AI System Quality Engineer does

AI System Quality Engineer tests LLM, RAG, MCP, and agent systems with golden datasets, deterministic checks, model-graded metrics, and adversarial probes. Use it to score quality and gate CI when behavior regresses.

AI System Quality Engineer is cataloged under Quality on DirSkills. AI System Quality Engineer comes from a repository tagged agent-browser, agent-skills, ai-testing, claude-code and cursor.

Documentation

README

AI System Quality Engineer

You are the quality owner for a system whose outputs are non-deterministic: an LLM app, a RAG pipeline, an MCP tool server, or a multi-step agent. Your job is to make its behavior measurable and to fail CI when quality regresses, without pretending a stochastic system is deterministic.

You do not reinvent the evaluators. You orchestrate the primitives the catalog already ships and compose them into one gated pipeline. Install what a given system needs:

This is the opening of the README. Read the full README on GitHub.

Frequently asked about AI System Quality Engineer

  • What else does PramodDutta publish alongside AI System Quality Engineer?

    AI System Quality Engineer is one of 25 skills that DirSkills catalogs from PramodDutta/qaskills, the repository it ships in. Its siblings there include A/B Test Validation, AFL++ Fuzzing Testing and AI Agent Evaluation. Each one is a separate skill with its own page in this directory, installs the same way AI System Quality Engineer does, and is maintained by PramodDutta in that same repository. The rest of the collection is listed on the PramodDutta/qaskills page.

  • How does AI System Quality Engineer compare to other Quality skills?

    AI System Quality Engineer ranks #1410 by stars among the 1557 Quality skills in this catalog. The most-starred ones next to it are Benchmark, Benchmark Optimization Loop and API Design Patterns. DirSkills ranks by the star count of the repository each skill ships in, so that order reflects how popular those repositories are rather than any review of AI System Quality Engineer against them. Open each page to compare what they document and how they install.

More from PramodDutta/qaskills

AI System Quality Engineer is one of 25 skills cataloged on DirSkills from PramodDutta/qaskills.

See all 25 skills →
🧪
2h ago

A/B Test Validation

A/B Test Validation checks experiment implementations for traffic splitting, statistical significance, metric tracking, and cleanup. Use it when setting up, reviewing, or debugging A/B testing in a codebase or CI pipeline.
Quality
21623
🪲
2h ago

AFL++ Fuzzing Testing

AFL++ Fuzzing Testing helps set up, review, debug, and integrate AFL++ mutation-based fuzz tests for binary programs. Use it to find crashes, hangs, and security vulnerabilities and to wire fuzzing into CI/CD.
Quality
21623
🧪
2h ago

AI Agent Evaluation

AI Agent Evaluation provides patterns for testing AI agents with multi-turn conversations, LLM-as-judge scoring, benchmarks, regression checks, and safety cases. Use it when building reproducible eval pipelines for agent quality and reliability.
AI Engineering
21623
🛡️
2h ago

AI Release Guardian

AI Release Guardian analyzes a git diff, maps release risks, selects relevant tests, and checks coverage gaps on changed lines. It produces a go/no-go report with cited evidence and does not merge or deploy.
Quality
21623
🧪
2h ago

AI Test Generation Patterns

AI Test Generation Patterns provides prompts and review checklists for generating unit, integration, and E2E tests with coding agents. Use it to target coverage gaps, strengthen assertions, and validate AI-generated test code.
Quality
21623
🧪
2h ago

AI Test Orchestration

AI Test Orchestration selects and prioritizes tests based on code changes, historical failures, and risk signals. Use it to manage flaky tests, optimize parallel execution, and predict likely failures in CI.
Quality
21623

More Quality skills

AI System Quality Engineer is one of 1557 Quality skills cataloged on DirSkills, ranked #1410 by stars among them.

Browse all Quality skills →
⏱️
3w ago

Benchmark

Benchmark measures performance baselines, detects regressions before and after PRs, and compares stack alternatives for pages, APIs, and builds.
Quality
239.8K36.4K
⏱️
3w ago

Benchmark Optimization Loop

Benchmark Optimization Loop turns vague performance goals into a bounded, measured optimization process. Use it when asked to make something faster, benchmark latency/throughput/cost, or choose the best implementation by repeated measured tests.
Quality
239.8K36.4K
🔌
3w ago

API Design Patterns

API Design Patterns provides conventions and best practices for designing consistent REST APIs. Use it when creating endpoints, reviewing contracts, handling pagination, error responses, status codes, filtering, or versioning.
Quality
239.8K36.4K