Documentation
README
Model Selection
Purpose
Choose the model that meets the task's requirements at the lowest cost and latency. Most production LLM features run on a model several times more expensive than the task requires, because nobody measured the cheaper one.
When to Use
- Choosing a model for a new feature.
- Reducing the cost or latency of an existing feature.
- Evaluating whether a newly released model is worth migrating to.
- Designing a routing strategy across several models.
Capabilities
- Task-to-capability matching.
- Cost and latency modeling at real volume.
- Routing: cheap model first, escalate on difficulty.
- Benchmark interpretation and its limits.
- Migration and re-evaluation.
Inputs
This is the opening of the README. Read the full README on GitHub.