RL Reward Construction

by agentscope-ai

RL Reward Construction is an AI Engineering skill for Claude Code, published by agentscope-ai in OpenJudge.

797 stars64 forkson agentscope-ai/OpenJudgeAdded 2026/08/23+1% in starsRepository updated 2026/08/03

View source on GitHub Download repo ZIP

agentagent-skillsai-agentalignmentevaluationgraderllmrewardreward-modelrlhfskill-mdskills

Install in seconds

Install RL Reward Construction

Copy RL Reward Construction into your Claude Code skills folder. Run the command in your terminal, or review the source on GitHub before installing.

terminal

npx degit https://github.com/agentscope-ai/OpenJudge/tree/main/skills/rl-reward ~/.claude/skills/rl-reward

Requires Node.js. Downloads this skill only — not the rest of the repository — into your Claude Code skills folder.

Without Node.js

git clone https://github.com/agentscope-ai/OpenJudge.git

Clones the whole repository, then copy the skill’s own directory into your skills folder yourself.

In this catalog

Source file: skills/rl-reward/SKILL.md in agentscope-ai/OpenJudge
Installs to: ~/.claude/skills/rl-reward
Collection: One of 18 skills cataloged from this repository
Category: AI Engineering — 2451 skills

What RL Reward Construction does

RL Reward Construction builds reward signals with OpenJudge for RLHF and RLAIF. Use it to choose pointwise, pairwise, tournament, or listwise strategies for GRPO, DPO, Best-of-N, and reward normalization.

RL Reward Construction is cataloged under AI Engineering on DirSkills. RL Reward Construction comes from a repository tagged agent, agent-skills, ai-agent, alignment and evaluation.

Documentation

README

Excerpt

RL Reward Construction with OpenJudge

Build reward signals for reinforcement learning from human feedback (RLHF) and reinforcement learning from AI feedback (RLAIF) using the openjudge library.

When to Use This Skill

Building scalar rewards for GRPO / REINFORCE rollout scoring
Generating (chosen, rejected) preference pairs for DPO / IPO
Best-of-N candidate selection
Multi-dimensional reward shaping (correctness + safety + format)
Replacing or bootstrapping a reward model with LLM-as-judge

This is the opening of the README. Read the full README on GitHub.

Frequently asked about RL Reward Construction

What else does agentscope-ai publish alongside RL Reward Construction?
RL Reward Construction is one of 18 skills that DirSkills catalogs from agentscope-ai/OpenJudge, the repository it ships in. Its siblings there include Align Human, Auto Arena and BibTeX Verification. Each one is a separate skill with its own page in this directory, installs the same way RL Reward Construction does, and is maintained by agentscope-ai in that same repository. The rest of the collection is listed on the agentscope-ai/OpenJudge page.
How does RL Reward Construction compare to other AI Engineering skills?
RL Reward Construction ranks #1632 by stars among the 2451 AI Engineering skills in this catalog. The most-starred ones next to it are Architecture Decision Records, AI-First Engineering and Agentic OS. DirSkills ranks by the star count of the repository each skill ships in, so that order reflects how popular those repositories are rather than any review of RL Reward Construction against them. Open each page to compare what they document and how they install.