RAG Eval

by agentscope-ai

RAG Eval is an AI Engineering skill for Claude Code, published by agentscope-ai in OpenJudge.

797 stars64 forkson agentscope-ai/OpenJudgeAdded 2026/08/23+1% in starsRepository updated 2026/08/03

View source on GitHub Download repo ZIP

agentagent-skillsai-agentalignmentevaluationgraderllmrewardreward-modelrlhfskill-mdskills

Install in seconds

Install RAG Eval

Copy RAG Eval into your Claude Code skills folder. Run the command in your terminal, or review the source on GitHub before installing.

terminal

npx degit https://github.com/agentscope-ai/OpenJudge/tree/main/skills/eval_pipeline/05-rag-eval ~/.claude/skills/05-rag-eval

Requires Node.js. Downloads this skill only — not the rest of the repository — into your Claude Code skills folder.

Without Node.js

git clone https://github.com/agentscope-ai/OpenJudge.git

Clones the whole repository, then copy the skill’s own directory into your skills folder yourself.

In this catalog

Source file: skills/eval_pipeline/05-rag-eval/SKILL.md in agentscope-ai/OpenJudge
Installs to: ~/.claude/skills/05-rag-eval
Collection: One of 18 skills cataloged from this repository
Category: AI Engineering — 2451 skills

What RAG Eval does

RAG Eval diagnoses RAG systems by separating retrieval problems from generation problems. Use it to check faithfulness, retrieval quality, hallucinations, and chunking changes with a diagnostic matrix.

RAG Eval is cataloged under AI Engineering on DirSkills. RAG Eval comes from a repository tagged agent, agent-skills, ai-agent, alignment and evaluation.

Documentation

README

Excerpt

RAG Eval

Evaluate RAG systems by diagnosing retrieval and generation separately. A single "RAG accuracy" number hides whether the problem is finding the right documents or using them correctly. This skill separates them so you know what to fix.

When to Activate

User has a RAG pipeline (retriever + generator) with traces
User wants to know if their RAG system hallucinates
User is optimizing chunking strategy and needs before/after comparison
User wants to build a RAG evaluation dataset

This is the opening of the README. Read the full README on GitHub.

Frequently asked about RAG Eval

What else does agentscope-ai publish alongside RAG Eval?
RAG Eval is one of 18 skills that DirSkills catalogs from agentscope-ai/OpenJudge, the repository it ships in. Its siblings there include Align Human, Auto Arena and BibTeX Verification. Each one is a separate skill with its own page in this directory, installs the same way RAG Eval does, and is maintained by agentscope-ai in that same repository. The rest of the collection is listed on the agentscope-ai/OpenJudge page.
How does RAG Eval compare to other AI Engineering skills?
RAG Eval ranks #1630 by stars among the 2451 AI Engineering skills in this catalog. The most-starred ones next to it are Architecture Decision Records, AI-First Engineering and Agentic OS. DirSkills ranks by the star count of the repository each skill ships in, so that order reflects how popular those repositories are rather than any review of RAG Eval against them. Open each page to compare what they document and how they install.