Dataset Evaluation

by awslabs

Dataset Evaluation is a Data skill for Claude Code, published by awslabs in agent-plugins.

867 stars145 forkson awslabs/agent-pluginsAdded 2026/08/22Repository updated 2026/08/21

View source on GitHub Download repo ZIP

agent-pluginsagent-skillsagentsawscoding-agent-skillscoding-agents

Install in seconds

Install Dataset Evaluation

Copy Dataset Evaluation into your Claude Code skills folder. Run the command in your terminal, or review the source on GitHub before installing.

terminal

npx degit https://github.com/awslabs/agent-plugins/tree/main/plugins/sagemaker-ai/skills/dataset-evaluation ~/.claude/skills/dataset-evaluation

Requires Node.js. Downloads this skill only — not the rest of the repository — into your Claude Code skills folder.

Without Node.js

git clone https://github.com/awslabs/agent-plugins.git

Clones the whole repository, then copy the skill’s own directory into your skills folder yourself.

In this catalog

Source file: plugins/sagemaker-ai/skills/dataset-evaluation/SKILL.md in awslabs/agent-plugins
Installs to: ~/.claude/skills/dataset-evaluation
Collection: One of 25 skills cataloged from this repository
Category: Data — 668 skills

What Dataset Evaluation does

Dataset Evaluation checks your dataset’s file type, schema, and format for SageMaker fine-tuning or evaluation. Use it before training or when you need to know whether your data is valid for a selected model and strategy.

Dataset Evaluation is cataloged under Data on DirSkills. Dataset Evaluation comes from a repository tagged agent-plugins, agent-skills, agents, aws and coding-agent-skills.

Documentation

README

Excerpt

Workflow Instruction

Follow the workflow shown below. Locate the dataset, check the file type, and resolve any issues with missing files or wrong file types. Determine the fine-tuning model and fine-tuning strategy. Run the appropriate validation based on the model family. Summarize the results: is the dataset ready for fine-tuning?

Prerequisites

The SDK environment has been verified (SDK version, region, execution role). If not done, activate the sdk-getting-started skill first.

This is the opening of the README. Read the full README on GitHub.

Frequently asked about Dataset Evaluation

What else does awslabs publish alongside Dataset Evaluation?
Dataset Evaluation is one of 25 skills that DirSkills catalogs from awslabs/agent-plugins, the repository it ships in. Its siblings there include AWS Architecture Diagram, AWS Lambda and AWS Lambda Durable Functions. Each one is a separate skill with its own page in this directory, installs the same way Dataset Evaluation does, and is maintained by awslabs in that same repository. The rest of the collection is listed on the awslabs/agent-plugins page.
How does Dataset Evaluation compare to other Data skills?
Dataset Evaluation ranks #498 by stars among the 668 Data skills in this catalog. The most-starred ones next to it are Benchmark Methodology, Jupyter Notebook and Solana. DirSkills ranks by the star count of the repository each skill ships in, so that order reflects how popular those repositories are rather than any review of Dataset Evaluation against them. Open each page to compare what they document and how they install.