LLM Evaluation

by nimadorostkar

LLM Evaluation is an AI Engineering skill for Claude Code, published by nimadorostkar in Claude-Skills-collection.

18 stars2 forkson nimadorostkar/Claude-Skills-collectionAdded 2026/07/16+13% in starsRepository updated 2026/07/14

View source on GitHub Download repo ZIP

aiclaudeclaude-skillsskills

Install in seconds

Install LLM Evaluation

Copy LLM Evaluation into your Claude Code skills folder. Run the command in your terminal, or review the source on GitHub before installing.

terminal

npx degit https://github.com/nimadorostkar/Claude-Skills-collection/tree/main/skills/ai/llm-evaluation ~/.claude/skills/llm-evaluation

Requires Node.js. Downloads this skill only — not the rest of the repository — into your Claude Code skills folder.

Without Node.js

git clone https://github.com/nimadorostkar/Claude-Skills-collection.git

Clones the whole repository, then copy the skill’s own directory into your skills folder yourself.

In this catalog

Source file: skills/ai/llm-evaluation/SKILL.md in nimadorostkar/Claude-Skills-collection
Installs to: ~/.claude/skills/llm-evaluation
Collection: One of 50 skills cataloged from this repository
Category: AI Engineering — 3670 skills

What LLM Evaluation does

Provides a systematic approach for measuring LLM feature quality, including building evaluation sets, selecting metrics, using LLM-as-judge, regression testing, and production evaluation. Use before iterating on prompts or models to avoid guesswork.

LLM Evaluation is cataloged under AI Engineering on DirSkills. LLM Evaluation comes from a repository tagged ai, claude, claude-skills and skills.

Documentation

README

Excerpt

LLM Evaluation

Purpose

Know whether an LLM feature is getting better or worse. Without evaluation, every prompt change is a guess, and the confidence that a change helped is indistinguishable from the confidence that it did not.

When to Use

Before iterating on any prompt or model in production.
Comparing models, prompts, or retrieval strategies.
Setting up regression testing for an LLM feature.
Deciding whether a quality complaint is real or anecdotal.

Capabilities

Evaluation-set construction from real usage.
Metric selection: exact match, similarity, rubric-based, task-specific.
LLM-as-judge, with the controls that make it trustworthy.
Regression testing in CI.
Online evaluation and production monitoring.

Inputs

This is the opening of the README. Read the full README on GitHub.

Frequently asked about LLM Evaluation

What else does nimadorostkar publish alongside LLM Evaluation?
LLM Evaluation is one of 50 skills that DirSkills catalogs from nimadorostkar/Claude-Skills-collection, the repository it ships in. Its siblings there include API Design, Agent Design and Agent Instructions. Each one is a separate skill with its own page in this directory, installs the same way LLM Evaluation does, and is maintained by nimadorostkar in that same repository. The rest of the collection is listed on the nimadorostkar/Claude-Skills-collection page.
How does LLM Evaluation compare to other AI Engineering skills?
LLM Evaluation ranks #3594 by stars among the 3670 AI Engineering skills in this catalog. The most-starred ones next to it are Architecture Decision Records, AI-First Engineering and Agentic OS. DirSkills ranks by the star count of the repository each skill ships in, so that order reflects how popular those repositories are rather than any review of LLM Evaluation against them. Open each page to compare what they document and how they install.