Reference Hallucination Arena

by agentscope-ai

Reference Hallucination Arena is a Quality skill for Claude Code, published by agentscope-ai in OpenJudge.

797 stars64 forkson agentscope-ai/OpenJudgeAdded 2026/08/23+1% in starsRepository updated 2026/08/03

View source on GitHub Download repo ZIP

agentagent-skillsai-agentalignmentevaluationgraderllmrewardreward-modelrlhfskill-mdskills

Install in seconds

Install Reference Hallucination Arena

Copy Reference Hallucination Arena into your Claude Code skills folder. Run the command in your terminal, or review the source on GitHub before installing.

terminal

npx degit https://github.com/agentscope-ai/OpenJudge/tree/main/skills/ref-hallucination-arena ~/.claude/skills/ref-hallucination-arena

Requires Node.js. Downloads this skill only — not the rest of the repository — into your Claude Code skills folder.

Without Node.js

git clone https://github.com/agentscope-ai/OpenJudge.git

Clones the whole repository, then copy the skill’s own directory into your skills folder yourself.

In this catalog

Source file: skills/ref-hallucination-arena/SKILL.md in agentscope-ai/OpenJudge
Installs to: ~/.claude/skills/ref-hallucination-arena
Collection: One of 18 skills cataloged from this repository
Category: Quality — 1354 skills

What Reference Hallucination Arena does

Reference Hallucination Arena evaluates LLMs on recommending real academic papers by checking cited references against Crossref, PubMed, arXiv, and DBLP. Use it to benchmark citation accuracy, hallucination rate, and year/field constraints, including tool-augmented search mode.

Reference Hallucination Arena is cataloged under Quality on DirSkills. Reference Hallucination Arena comes from a repository tagged agent, agent-skills, ai-agent, alignment and evaluation.

Documentation

README

Excerpt

Reference Hallucination Arena Skill

Evaluate how accurately LLMs recommend real academic references using the OpenJudge RefArenaPipeline:

Load queries — from JSON/JSONL dataset
Collect responses — BibTeX-formatted references from target models
Extract references — parse BibTeX entries from model output
Verify references — cross-check against Crossref / PubMed / arXiv / DBLP
Score & rank — compute verification rate, per-field accuracy, discipline breakdown
Generate report — Markdown report + visualization charts

Prerequisites

# Install OpenJudge
pip install py-openjudge

# Extra dependency for ref_hallucination_arena (chart generation)
pip install matplotlib

This is the opening of the README. Read the full README on GitHub.

Frequently asked about Reference Hallucination Arena

What else does agentscope-ai publish alongside Reference Hallucination Arena?
Reference Hallucination Arena is one of 18 skills that DirSkills catalogs from agentscope-ai/OpenJudge, the repository it ships in. Its siblings there include Align Human, Auto Arena and BibTeX Verification. Each one is a separate skill with its own page in this directory, installs the same way Reference Hallucination Arena does, and is maintained by agentscope-ai in that same repository. The rest of the collection is listed on the agentscope-ai/OpenJudge page.
How does Reference Hallucination Arena compare to other Quality skills?
Reference Hallucination Arena ranks #977 by stars among the 1354 Quality skills in this catalog. The most-starred ones next to it are Benchmark, Benchmark Optimization Loop and API Design Patterns. DirSkills ranks by the star count of the repository each skill ships in, so that order reflects how popular those repositories are rather than any review of Reference Hallucination Arena against them. Open each page to compare what they document and how they install.