Documentation
README
Implement the scoring rule you intend to condemn
A leaderboard number is not the model's output. It is the output of a path: a split, a subset of its rows, an order, a batch loader that may discard a partial batch, a checkpoint or an epoch chosen by some rule, an aggregation across tasks or targets, a statistic across seeds. Every one of those steps is worth points of metric, and any one of them left out of your re-implementation goes into your reproduction gap without a label on it.
This is the opening of the README. Read the full README on GitHub.