Documentation
README
Benchmarking
llmprobe is the measurement layer. tests/bench.sh boots mlx-serve (one model at a time: boot, probe, kill, settle) and llmprobe takes every number via --bench-only. We do not hand-roll timing loops — llmprobe discards a warmup per scenario, reports median-of-3 as median (min-max), refuses to fabricate a number when usage is missing, records the machine it ran on, and applies the same protocol to every engine.
./tests/bench.sh # every model (~did we regress)
./tests/bench.sh --only qwen36-27b # one row
./tests/bench.sh --url 127.0.0.1:1234 -m <id> # a server someone else started
./tests/bench.sh --full # median of 3 per rung, to 64k
This is the opening of the README. Read the full README on GitHub.