Leaderboard

The only requirement to appear on the leaderboard is to have a successful Run.

Being on the leaderboard makes you eligible for rewards based on the rewards system of the crunch you are participating in.

Submission Phase

During the Submission Phase, the leaderboard is updated when a Run is scored.

If there is no public leaderboard, the leaderboard will be updated when a Prediction is validated to make sure it is ready for scoring.

Out-of-Sample Phase

During the Out-of-Sample Phase, the leaderboard is only updated once when all the Run that are running on unseen data are scored.

Columns

Only a subset of columns will be visible, depending on the current state of the crunch.

Name	Description
Rank	Rank of the participant / model
Participant	Name of the participant / model
Participation	Distinguish if a participant is ready for the private leaderboard (only if there is no public leaderboard)
Team	The participant's team name (only if the competition allows teams)
Best metric (unit)	The best submission score for the metric metric (reset between each rounds)
Last metric (unit)	The last submission score for the metric metric
Mean	The weighted average of the metrics (only if there are multiple)
metric (unit)	The score value of the metric metric
Run Success	The number of successful runs / the number of failed runs
Previous LB Chn	Rank change from the previous round
Public LB Chn	Rank change from the public leaderboard
Weekly Change	Rank change from the previous week
User Hist. Rewards	The amount of rewards ever received by the participant
Proj. Rewards	The amount of rewards the participant is expected to receive at the end of the month

A small view of the leaderboard, including rank changes and a metric.

Badges

Icon	Description
	The participant has won the crunch.
	The participant has earned a certificate.
	The participant has received a prize.
	The participant only submitted a Quickstarter and is disqualified from receiving any rewards.
	The prediction is too close to another (grouped by same user or same team, if correlation > 95%*).
	The code is not deterministic and is not eligible for any rewards. It is tested by running the infer function twice (on the same data) and comparing if the predictions are the same.
	The score is not within the allowed range. Hover over to see which limit(s) the model did not fit.
	Only the team leader is considered for ranking on the leaderboard.

Participant

If the crunch allows more than one model, the model name is displayed right after the participant name. Otherwise, only the participant name is displayed.

Only one Model on the Leaderboard

Some competitions (like Mid+One) allow more than one model, but only ONE can be displayed on the leaderboard at any given time.

The selection can be made during the Submission Phase. After that, the "Use this one" button will no longer be available.

The model will be displayed on the leaderboard and will be eligible for rewards.

The model will NOT be displayed on the leaderboard and will NOT be eligible for rewards.

Ties

Some competitions (such as Broad #2) allow ties when models achieve the same scores, meaning they receive the same rank.

The rank assigned to all tied models is the first rank of the tie.

The reward is distributed equally among the tied models, calculated as the sum of the rewards for the tied models divided by the number of models in the tie.

Example

For a prize pool that rewards the top 10 models:

• If the 1st, 2nd, and 3rd place models are tied, they will all be ranked 1st. The next model will be ranked 4th. Their total reward will be: (1st place prize + 2nd place prize + 3rd place prize) / 3.

• If the 8th, 9th, 10th, 11th and 12th place models are tied, they will all be ranked 8th. The next model will be ranked 13th. Their total reward will be: (8th place prize + 9th place prize + 10th place prize) / 5.