CLI Eval

CLI Eval is an AI Engineering skill for Claude Code, published by diegosouzapw in OmniRoute.

46.8K stars6.3K forkson diegosouzapw/OmniRouteAdded 2026/08/13Repository updated 2026/08/13

a2aai-agentsai-gatewayanthropicclaudeclaude-codeclinecodexcopilotcursordeepseekfree-aigeminikimillm-gatewaymcpopenaiopenai-proxyqwentoken-saver

Install in seconds

Install CLI Eval

Copy CLI Eval into your Claude Code skills folder. Run the command in your terminal, or review the source on GitHub before installing.

terminal

npx degit https://github.com/diegosouzapw/OmniRoute/tree/release/v3.8.50/skills/cli-eval ~/.claude/skills/cli-eval

Requires Node.js. Downloads this skill only — not the rest of the repository — into your Claude Code skills folder.

Without Node.js

git clone https://github.com/diegosouzapw/OmniRoute.git

Clones the whole repository, then copy the skill’s own directory into your skills folder yourself.

In this catalog

Source file: skills/cli-eval/SKILL.md in diegosouzapw/OmniRoute
Installs to: ~/.claude/skills/cli-eval
Collection: One of 25 skills cataloged from this repository
Category: AI Engineering — 2451 skills

What CLI Eval does

CLI Eval creates and runs LLM evaluation suites from the command line, with live benchmark progress, scorecards, model comparisons, and CI integration.

CLI Eval is cataloged under AI Engineering on DirSkills. CLI Eval comes from a repository tagged a2a, ai-agents, ai-gateway, anthropic and claude.

Documentation

README

Excerpt

Overview

Create and run evaluation suites, watch live benchmark progress, view scorecards, compare model performance, and integrate eval runs with CI workflows from the CLI.

Quick install

npm install -g omniroute   # or: npx omniroute
omniroute --version

Subcommands

`eval`

Example:

omniroute eval

`eval suites`

Example:

omniroute eval suites

`eval list`

Example:

omniroute eval list

`eval get <suiteId>`

Example:

omniroute eval get <suiteId>

`eval create`

Flags:

--file <path>

Example:

omniroute eval create

`eval run <suiteId>`

Flags:

This is the opening of the README. Read the full README on GitHub.

Frequently asked about CLI Eval

What else does diegosouzapw publish alongside CLI Eval?
CLI Eval is one of 25 skills that DirSkills catalogs from diegosouzapw/OmniRoute, the repository it ships in. Its siblings there include CLI Backup Sync, CLI Batches and CLI Chat. Each one is a separate skill with its own page in this directory, installs the same way CLI Eval does, and is maintained by diegosouzapw in that same repository. The rest of the collection is listed on the diegosouzapw/OmniRoute page.
How does CLI Eval compare to other AI Engineering skills?
CLI Eval ranks #106 by stars among the 2451 AI Engineering skills in this catalog. The most-starred ones next to it are Architecture Decision Records, AI-First Engineering and Agentic OS. DirSkills ranks by the star count of the repository each skill ships in, so that order reflects how popular those repositories are rather than any review of CLI Eval against them. Open each page to compare what they document and how they install.