Documentation
README
LLM Evaluation
Help users build, audit, and improve evaluation systems for LLM pipelines.
Where to Start
Have an existing eval pipeline? Start with an eval audit to surface problems: missing error analysis, unvalidated judges, vanity metrics. See the eval-audit reference.
Starting from scratch? Begin with error analysis on real traces. If no production data exists, generate synthetic data first. See the error-analysis and generate-synthetic-data references.
Test Suites
Test suites are the primary way to test agents in Opik. They combine test items with string assertions checked by an LLM judge, plus execution policies for multi-run reliability testing. Available in both Python and TypeScript SDKs.
Python:
This is the opening of the README. Read the full README on GitHub.