Documentation
README
Creating a new CORAL task
A CORAL task is three things that must line up:
examples/<task>/
├── task.yaml # config: name, description, grader entrypoint, agent count
├── seed/ # starter code agents see when they begin (the repo_path)
│ └── solution.py
└── grader/ # standalone Python package
├── pyproject.toml
└── src/<task>_grader/
├── __init__.py
└── grader.py # class Grader(TaskGrader): ...
The packaged form is the only supported form. The package gives the grader its own venv and ships everything the eval needs — grader code, helper modules, and hidden data (see "Hidden data" below).
Reference implementations
Look at these before writing anything new — copy the closest one and edit:
This is the opening of the README. Read the full README on GitHub.