Scoring

Spearman correlation score

Once you have submitted, your prediction will be confronted with what is happening on the market. It will be scored daily using spearman rank correlation.

Post-processing ranking (scaled leaderboard)

  • The last 12 rounds of the master dataset are taken into account.
  • Non-submissions on a round get the score of the less performant player -1. (Incentivises consistency)
  • Scores are normalized (between range [-1,1]) per round.
  • Users scoring above the 90th percentile get the same score of +1. (This is to disincentivize overfit models as anyone above a threshold gets the same score).
  • Finally, the scores for all rounds are averaged.
This post-processing ranking has been voted :
First proposal:
Snapshot
[Ranking] Rounds Equal Weightage proposal
  • Second proposal:
Snapshot
[Ranking] Normalization, Top percentile, Non-submitters penalty
  • Third proposal on the non submission penalty: