# Welcome Welcome to the global collective intelligence network. ## What is Crunch? At its core, Crunch is a collective intelligence protocol that distributes machine learning workloads across a global community. * 10,000+ ML engineers and 1,200+ PhDs contribute models. * Participants span 100+ countries, each bringing unique perspectives and methods. * The protocol structures this effort into Crunches, prediction challenges with clear rules, rewards, and evaluation criteria. This system has already delivered results for world-class institutions: * [ADIA Lab (research arm of the Abu Dhabi Investment Authority)](https://www.adialab.ae/) – achieved double-digit improvements in financial forecasting. * [Broad Institute of MIT & Harvard](https://www.broadinstitute.org/) – outperformed internal benchmarks in cancer research using predictive genomics. ## Why Crunch? Traditional predictive modeling is limited by silos. Large in-house teams face bottlenecks, while independent researchers lack resources and exposure. Crunch solves this by: * **Lowering barriers**: Anyone with skill can compete, regardless of geography or institutional backing. * **Unlocking diversity**: A network of independent approaches outperforms centralized teams stuck in single modes of thinking. * **Ensuring security**: Trusted Execution Environments (TEEs) guarantee models remain private, verifiable, and fully owned by their creators. * **Delivering results**: Aggregated predictions (ensembles, composites, mixtures of experts) are continuously tested, updated, and deployed in real-world environments. With Crunch, you focus on building the best model. We handle infrastructure, evaluation, and reward distribution ## Who can Participate? Everyone. Crunch is open and permissionless. * **Machine Learning engineers**: Submit models, earn rewards, and prove your expertise. * **Researchers and students**: Gain access to real-world data challenges and contribute fresh perspectives. * **Institutions and coordinators**: Launch Crunches to harness global intelligence for finance, science, and beyond. Python knowledge helps, but Quickstarter notebooks and community resources make onboarding fast and accessible. ## How it works? 1. [**Create an account**](https://hub.crunchdao.com/signup) – and explore active Crunches. 2. [**Join a Crunch**](https://hub.crunchdao.com/competitions) – each challenge has structured rules, data, and a prize pool. 3. **Build & submit models** – start with Quickstarters or bring your own innovation. 4. **Evaluation & rewards** – models are tested against live outcomes, ranked, and rewarded in USDC. 5. Scale & repeat – high-performing models can be aggregated into prediction feeds, securing long-term impact and recognition ## When Can You Crunch? Crunches are always live. From financial markets to biomedical research, you’ll find competitions suited to your expertise and curiosity. New challenges are continuously published, offering opportunities to contribute, learn, and earn. ## The Future with Crunch Crunch aims to be the bridge for humans and machines to collectively reduce uncertainty about the future. By participating, you join a growing movement to: * Democratize predictive intelligence. * Align incentives between researchers, institutions, and innovators. * Create a resilient, transparent, and global marketplace for models. ***

Welcome to Crunch. The future of predictive intelligence is collective.

# Register your account Joining Crunch gives you access to competitions, datasets, and the global collective intelligence network. With your account, you can submit models, track results, and earn rewards. How to Register You can sign up in two ways: 1. **Email** * Enter your email address in the registration form. * Check your inbox for a **6-digit one-time password (OTP)**. * Enter the OTP to verify your email. 2. **Google/Social Login** * Select Google, Discord or LinkedIn or Passkey * Authorize the connection, then complete your name and country to finalize your account. {% embed url="" %} Sign up here {% endembed %}
{% hint style="warning" %} If the country list does not appear, try again using a VPN. {% endhint %} ## Complete your Profile After registration, you’ll go through a short onboarding process: 1. Confirm your name and country. 2. Accept CrunchDAO’s participation rules. 3. Optionally, add details to your profile (bio, socials, etc.).
{% hint style="info" %} Linking your socials makes it easier to log in and helps other Crunchers discover your work. {% endhint %} ### Link your Kaggle Account To prove ownership of your Kaggle profile: 1. First, connect GitHub or Twitter on both CrunchDAO and Kaggle. 2. Go to your Crunch account page, click Link Kaggle, and enter your Kaggle username. This step ensures secure verification. You can later disconnect the link(s) from Kaggle if you wish.
## Secure your Account For wallet use and enhanced security, Crunch requires multi-factor authentication (MFA). You can choose: * **Passkey** – allows direct passwordless login. * **Authenticator App** (e.g. Google Authenticator). You may disable MFA later if needed, but enabling it is strongly recommended.
## All set! Once registered, you’re ready to [explore active Crunches](https://hub.crunchdao.com/competitions), join competitions, and start building models! # Navigate the Dashboard The gateway to the Crunch ecosystem, where it all begins. ## Dashboard A single place to consolidates the latest news, crunches, and your model's performance to provide a quick overview of current activity. ### The Event Calendar To stay informed about upcoming events, CrunchDAO maintains a calendar widget that displays: * Your activities * Upcoming crunches * Times when you joined a crunch * Upcoming events (webinars, physical meetings, conferences, etc.).

The calendar widget

### The Contribution Graph Similar to GitHub, it tracks your activity within the CrunchDAO ecosystem. The deeper the colors, the more invested you are!

The contributions widget

{% hint style="info" %} [Understand how activities are counted](/crunch-hub/dashboard/activity-graphs) {% endhint %} ### The Rewards Graph Track how much CrunchDAO gave to the community via leaderboard prizes. To ensure full transparency regarding the rewards given to the community, all transactions are made on the blockchain and are accessible via the [payouts page](https://hub.crunchdao.com/account/payouts).

The rewards widget

### The Competition Summary Card Take a quick look at how your models are performing in the crunches you have joined that are currently active.
## Currently Ongoing Crunches You can find a directory of available crunches. We encourage users to explore a variety of crunches to expand their knowledge. If you are curious, you can also consult previous crunches that have already ended.

Currently open crunches

# Activity Graphs Understand how activities are counted. ## Daily and Weekly Contributions Contributions are tasks related to data scientists and civilians (community). The data scientist activities are: * When you submit your code * When you run your code The civilian activities are: * When you send a message on the [Community Discord server](https://discord.com/invite/veAtzsYn3M) * When you join the [Community Discord server](https://discord.com/invite/veAtzsYn3M) * When you tweet or retweet about our [Twitter Account](https://x.com/crunch_fdn) **(soon)** * When you post about our [LinkedIn Page](https://www.linkedin.com/company/crunchdao/) **(soon)**

Daily contributions

Weekly contributions

{% hint style="info" %} You must first [connect your social accounts](https://hub.crunchdao.com/account#connections) so the system can identify you. {% endhint %} ## Privacy No content is ever stored. Just the fact that you did something, where, and when. For Discord messages, the channel name and message ID are stored, but never the content of the message itself. If you decide to remove your message, you can be sure that the content will be lost forever. ## Ghost Activities Activities are never removed, so the activity graph may show a larger number of activities if the original resource has been deleted. # Competitions ## Why we use the Solana Blockchain? Integrating blockchain technology enhances the CrunchDAO platform in several key ways: * **Open Participation**: Third parties, beyond Crunch Lab, can post challenges, increasing the earning opportunities for Crunchers. * **Privacy Guarantees**: Trusted Execution Environments (TEEs) and Multi-Party Computation (MPC) ensure that model code and data feeds remain private and secure. * **Continuous Rewards**: Ongoing challenges pay out dynamically, enabling top-performing models to generate a steady revenue stream. Solana is currently the best fit for CrunchDAO's real-time competitions and prediction feeds, offering fast transaction processing and low fees that fit perfectly with our requirements. More importantly, Solana's vision—to be the leading platform for low-latency information transfer and financial software—closely aligns with CrunchDAO's goal of becoming the leading decentralized prediction network. # Numinous: Predictive Agents For Real World Outcome ## Overview Prediction markets like Polymarket have become one of the most watched phenomena in forecasting, aggregating real-time information into probability estimates that consistently outperform polls, expert panels, and traditional models. In this competition, you'll build a forecasting agent that predicts the outcomes of live binary events: return a probability for each question, get scored when it resolves. Crunch is launching this competition in partnership with [Numinous](https://numinouslabs.io/), a decentralized forecasting subnet on Bittensor (SN6) founded by Cambridge mathematician Marc Graczyk. Its goal is to aggregate AI agents into a collective forecaster that outperforms any individual model. Every prediction target is a live binary market sourced from [Polymarket](https://polymarket.com/): questions like "*Will the US enter a recession in 2026?*" or "*Will BTC exceed $120,000 before June?*" Your agent receives the question, the current market price, and a resolution deadline. It returns a probability between 0 and 1. The collective output is sold to traders and institutions through the Eversight API. The most competitive strategies involve LLMs analyzing event descriptions, scraping news for recent developments, anchoring against historical base rates, and calibrating against live market prices. In this Crunch, you build a TrackerBase model that processes Polymarket events in real time and outputs probability estimates. The best-performing models get aggregated into an ensemble forecast that mines the Numinous subnet directly. ## How to Participate Trackers (models) must return a probability between 0.0 and 1.0 for each event and maximize accuracy across all resolved questions. See the open-source Crunch framework [here](https://github.com/crunchdao/crunch-numinous). **Event types covered:** * Macroeconomic and geopolitical outcomes * Cryptocurrency and financial market milestones * Elections, sports, and public events * Technology and science announcements ## Phases * **Phase 1:** 1-month model calibration and warmup phase, where predictions are scored but not rewarded. * **Phase 2**: 2 months with $5,000 USDC. * **Phase 3**: Ongoing mining rewards from the Numinous SN6 subnet currently averaging $3K / a day. ## Prediction Target For each active event, your tracker receives an `EventInput` via `predict()`: {% code expandable="true" %} ```json5 { "event_id": "62dadbf3-fc7d-4e76-8a60-7df9fc66a1ad", "run_id": "a4d13d7b-...", // Mandatory to forward to the Gateway "title": "Will the US enter a recession in 2026?", "description": "This market will resolve to 'Yes' if...", "cutoff": "2026-12-31T00:00:00Z", "metadata": { ... } } ``` {% endcode %} When called to predict, it returns a `ForecastOutput`: {% code expandable="true" %} ```json5 { "event_id": "62dadbf3-fc7d-4e76-8a60-7df9fc66a1ad", "prediction": 0.72, // 72% chance of Yes "reasoning": "Its because ...", // Reasoning behind the prediction, can be omitted } ``` {% endcode %} Predictions are clipped to \[0.01, 0.99] during scoring to prevent degenerate edge cases. ## The Challenge Beating it requires genuine alpha: information the market hasn’t priced in yet, faster reaction to breaking news, better long-run calibration, or reasoning that cuts through noise. Your model needs to find signal beyond what the crowd has already priced. ## Game Rules ### Available Services The competition only allows you to access the following services to generate your predictions: * **Chutes AI**: LLM inference with multiple open-source models * **Desearch AI**: Web search, social media search, and content crawling * **OpenAI**: GPT-5 series models with built-in web search * **Perplexity**: Reasoning LLMs with built-in web search * **Vericore**: Statement verification with evidence-based metrics * **OpenRouter**: Model router with access to hundreds of LLM models (Claude, Gemini, Llama, etc.) * **LunarCrush**: Social media intelligence and sentiment data for any topic * **Numinous Indicia**: Geopolitical and OSINT signals intelligence (X/Twitter, LiveUAMap) * **Numinous Signals**: Event-relevant news signals scored by relevance and impact, causal driver graphs, and deep research reports * **Unusual Whales**: Financial news headlines with filtering by source, ticker, and sentiment * **Public Data Proxy**: Generic proxy any number of free public APIs across sports, economics, weather, finance, and more. No cost. {% hint style="info" %} [Read the official Numinous documentation to find out how to use them.](https://github.com/numinouslabs/numinous/blob/main/docs/gateway-guide.md) {% endhint %} ### Start * The game begins with a 1-month model calibration and warmup phase, where predictions are scored but not rewarded. * Leaderboard ranking is based on a [weighted score](https://github.com/crunchdao/crunch-numinous#scoring). * A model must accumulate [enough resolved predictions](https://raw.githubusercontent.com/crunchdao/crunch-numinous/refs/heads/main/docs/emission-weights.png) to receive a ranking. * Each player may run up to two model, which can be updated at any time. ### Prediction Phase Events are continuously arriving from the Polymarket feed. For each active event, your model is called via `_predict()` and must return a probability within the prediction interval. Only models registered before an event is broadcast can predict on it. ### Scoring Once an event’s resolution horizon elapses, the score worker searches for a matching resolution record in the feed. A market with a final Yes price ≥ 0.95 resolves to 1; a price ≤ 0.05 resolves to 0. The Brier score is then computed: $$ brier\_score=(prediction-outcome)^2 $$ Where `outcome` is 1 (event happened) or 0 (event didn’t happen). Missing, invalid or default (`0.5`) predictions will receive the score of `0.25`. The Brier score is strictly proper: the optimal strategy is to report your honest probability estimate, and no gaming is possible. Scores are bounded between 0.0 (perfect) and 1.0 (worst possible).
You predictOutcomeBrier ScoreQuality
0.90Yes (1)0.01Excellent: confident and correct
0.50Yes (1)0.25Uninformative: no better than guessing
0.10Yes (1)0.81Terrible: confident and wrong
0.20No (0)0.04Good: low probability for a non-event
0.80No (0)0.64Bad: expected it to happen, but it didn’t
The leaderboard ranks in ascending order. Lower Brier is better. ## Leaderboard The events window is long enough to smooth noise from individual events, and short enough to reward models that adapt as new information arrives. The event count requirements are as follows: * Global: a minimum of **200 events** is required, and **only the last 600** will be taken into account. * Geopolitical: a minimum of **100 events** is required, and **only the last 200** are taken into account. Those who are still below the event threshold will appear at the bottom of the leaderboard. ## Payouts The prize pool is $5,000 USDC, since we are in Phase 2. Rewards are calculated every Monday at 12 p.m. GMT and are then frozen for the distribution period. The first period begins on May 25, 2026. In order to be eligible for rewards, a model must: * Outperform the benchmark, which is available on the leaderboard as `enzo/benchmark`. * Rank in the top 10 based on your [weighted score](https://github.com/crunchdao/crunch-numinous#scoring) in **the Signal track**. * If there are fewer than 10 eligible participants, the undistributed share is retained and not redistributed. * Only the top model from each participant is rewarded. * Have both **Global Brier** and **Geopolitics Brier** scores below `0.25`. ## Build Your Tracker ### Code Interface Subclass `TrackerBase` from the [`numinous.tracker`](https://pypi.org/project/crunch-numinous/) module, and implement the \``predict(event)` method, to return your probability estimate when called. {% code title="Python Notebook Cell" expandable="true" %} ```python from numinous.tracker import TrackerBase class MyForecaster(TrackerBase): """Your binary event forecasting model.""" def _predict(self, event: dict): """Return your probability estimate.""" event_id = event.get("event_id") run_id = event.get("run_id") # Your signal here: use NLP, LLMs, external data, etc. prediction = your_forecasting_logic(event) return { "event_id": event_id, "prediction": max(0.0, min(1.0, prediction)), "reasoning": None, # This can be omitted } ``` {% endcode %} {% embed url="" %} ### Authentication You need to authenticate via the event's `run_id` property, which you must forward with all web requests to your Gateway. Depending on the endpoint, you also need to provide the [necessary provider API key via the correct header.](https://github.com/crunchdao/crunch-numinous?tab=readme-ov-file#authentication) We recommend storing them in constants for reuse in your code. {% code title="Python Notebook Cell" expandable="true" %} ```python import os import httpx # Specify your OpenAI's API Key OPENAI_API_KEY = ... # Get the URL of the Gateway GATEWAY_URL = os.environ.get("SANDBOX_PROXY_URL", "https://public-gateway.numinous.competition.crunchdao.com") def your_forecasting_logic(event: dict): run_id = event.get("run_id") response = httpx.post( f"{GATEWAY_URL}/api/gateway/openai/responses", json={ # IMPORTANT: Always forward the `run_id` to the Gateway otherwise the request will fail "run_id": run_id, "model": "gpt-5-mini", "input": [ { "role": "user", "content": "Will BTC hit 100k?" } ], }, headers={ # IMPORTANT: Send the API Key header to the Gateway "x-openai-api-key": OPENAI_API_KEY, }, timeout=30, ) ``` {% endcode %} ### Directions for Competitive Models The market price is your baseline. From there, a few directions have shown real edge: * **LLM-based reasoning**: Use GPT-5, Claude, or local models to analyze event descriptions and resolution criteria, then estimate how likely the described outcome is. * **News sentiment**: Query news APIs for recent coverage related to each question. A surge of negative headlines on a "Yes" question is a signal. * **Historical base rates**: Build a database of similar past events and their outcomes. Questions about recessions, elections, and technological milestones all have reference classes. * **Ensemble methods**: Combine market price, text analysis, and base rates with learned weights. No single signal holds up on its own. * **Calibration**: Post-process raw probabilities using isotonic regression or Platt scaling on your own historical prediction data to correct systematic overconfidence or underconfidence. # Synth: Synthetic Price Data Forecast prices of Bitcoin, Ethereum, Solana, Tether Gold, and tokenized stocks (Apple, Nvidia, Tesla, Alphabet, S\&P 500). ## Overview [Synth](https://www.synthdata.co/) is a decentralized prediction subnet on Bittensor that forecasts crypto and financial asset prices through probabilistic modeling. Instead of predicting a single price, participants build models that output probability distributions, capturing not just where an asset might go, but the range of possibilities and their likelihoods. This approach helps model real market behavior like volatility clustering and tail risks that traditional forecasts often miss. In this Crunch, you need to build probabilistic forecasting models for Bitcoin, Ethereum, Solana, Tether Gold, and tokenized stocks (Apple, Nvidia, Tesla, Alphabet, S\&P 500). Your models will predict over 1-hour and 24-hour price distributions scored on statistical accuracy. The best-performing models will be aggregated into an ensemble forecast that captures collective intelligence from our community. ## How to participate Enter Synth and submit full return density forecasts for selected assets. **Trackers (models)** must generate returns density predictions, not single predictions and maximize accuracy overall forecasts. See open-source Crunch framework**:** [Crunch-Synth](https://github.com/crunchdao/crunch-synth) **Covered Assets:** * Bitcoin (BTC) * Ethereum (ETH) * Solana (SOL) * Tether Gold (XAUT) * [SP500 tokenized ETF (SPYX)\*](#user-content-fn-1)[^1] * S\&P 500 Index (SP500) * NVIDIA tokenized stock (NVDAX) * Tesla tokenized stock (TSLAX) * Apple tokenized stock (AAPLX) * Alphabet tokenized stock (GOOGLX) * Ripple (XRP) * Hyperliquid (HYPE) * Crude Oil WTI (WTIOIL) ## Phases * \[Phase 1] - 1-month warmup period before Live Trading * \[Phase 2] - 3 months live with a fixed $30,000 USDC prize pool * \[Phase 3] - Rewards pools will be generated from mining Synth subnet and may be higher. ## Prediction Targets Trackers must predict the **probability distribution of returns**, defined as: $$ r\_{t,k} = P\_t - P\_{t-k} $$ For each defined step $$k$$ (e.g., 5 minutes, 1 hour, …), your tracker must return a full **probability density function (PDF)** over the future price change $$r\_{t,k}$$. ## Visualize the challenge The Synth coordinator is evaluated on **incremental return predictions**, not raw prices.\ Incremental returns capture the *relative* change in price and produce a stationary series that is easier to model and compare across assets. Below is an example of a **density forecast over incremental returns for the next 24h at 5-minute intervals**:
Below is a minimal example showing what your tracker might return: ```python # Expected output >>> model.predict(asset="SOL", horizon=86400, step=300) [ { "step": (k + 1) * step, "prediction": { "type": "builtin", "name": "norm", "params": { "loc": -0.01, # mean return "scale": 0.4 # standard deviation of return } } } for k in range(0, horizon // step) ] ``` Here is the **return forecast mapped into price space**:
### Game Rules #### Start * The game begins with a **model calibration /** **warm-up phase of 4 weeks,** where predictions are scored but **not rewarded.** * Leaderboard is based on **7-day rolling average** of CRPS scores. * A player must wait **7 days** to get a meaningful ranking. * Players may **enter or exit the game at any time**. * Each player may run **two active models**, which can be updated at any time. #### Prediction Phase In each prediction round, players must submit **a set of** **density forecasts**. A **prediction round** is defined by **one asset**, **one forecast horizon** and **one or more step resolutions**. * A **24-hour horizon** forecast * Triggered **every 1 hour** for each asset * Step resolutions: **{5-minute, 1-hour, 6-hour, 24-hour}** * Supported assets: ```python ["BTC", "SOL", "ETH", "XAUT", "SPYX", "SP500", "NVDAX", "TSLAX", "AAPLX", "GOOGLX", "XRP", "HYPE", "WTIOIL"] ``` * A **1-hour horizon** forecast * Triggered **every 12 minutes** for each asset * Step resolutions: **{1-minute, 5-minute, 15-minute, 30-minute, 1-hour}** * Supported assets: ```python ["BTC", "SOL", "ETH", "XRP", "HYPE"] ``` All required forecasts for a prediction round must be generated **within 40 seconds**. {% hint style="info" %} Don't submit models that needs more than 40 seconds to infer! {% endhint %} #### Scoring * Once the full horizon has passed, each prediction is scored using a [**CRPS**](https://en.wikipedia.org/wiki/Scoring_rule#:~:text=%5B8%5D-,Continuous%20ranked%20probability%20score,-%5Bedit%5D) **scoring function**. * A lower **CRPS score** reflects more accurate predictions. * Missing or invalid predictions receive the **worst CRPS score of the round**. Leaderboard ranking is based on a **7-day rolling average** of CRPS scores across **all assets and horizons**, evaluated **relative to other participants**, for each prediction round: * The **best CRPS score is assigned a normalized score of 1** * The **worst 5% of CRPS scores are assigned** **a score of 0** * Intermediate scores are scaled accordingly ### Leaderboard & Game State The leaderboard displays several **relative performance indicators**, computed across **all assets and horizons**: * **Primary Metric** * **Anchor CRPS (7 days):** Rolling relative CRPS average over the past 7 days. * **Secondary Metrics** * **Steady CRPS (3 days):** Rolling relative CRPS average over the past 3 days. * **Recent CRPS (24 hours):** Rolling relative CRPS average over the past 24 hours. #### Payouts * Rewards are distributed at target resolution + 24h (every 7 + 1 days). * Real mining rewards from Synth Miners (currently up to 50K / months). * A 500 USD/month [(or 115.07 USD/week)](#user-content-fn-2)[^2] model hosting fee is applied. * A 20% platform fee (after hosting fee) is applied. * **The rest is the Pot.** * Price is determined at the creation of the checkpoint by converting [Alpha to TAO](#user-content-fn-3)[^3] and then converting [TAO to USD](#user-content-fn-4)[^4]. * Maximum of 8 players **receive 100% of the Pot.** * Top 4 of the 1-hour horizon. * Top 4 of the 24-hour horizon. * If a participant appears on both horizons, their prize is the sum. * The miners that Crunch manages can be found [here](https://taostats.io/subnets/50/metagraph?order=stake%3Adesc\&filter=5ECfazM69yvhX4SLX2CDzW4iSJvzJx3XESkhdncqM3HmyfKf). * The stake is not withdrawn each week; but only the new emissions are taken into account for the checkpoint. ## Probabilistic Forecasting Probabilistic forecasting provides **a distribution of possible future values** rather than a single point estimate, allowing for uncertainty quantification. Instead of predicting only the most likely outcome, it estimates a range of potential outcomes along with their probabilities by outputting a **probability distribution**. A probabilistic forecast models the conditional probability distribution of a future value $$(Y\_t)$$ given past observations $$(\mathcal{H}\_{t-1})$$. This can be expressed as: $$ P(Y\_t \mid \mathcal{H}\_{t-1}) $$ where $$(\mathcal{H}\_{t-1})$$ represents the historical data up to time $$(t-1)$$. Instead of a single prediction $$(\hat{Y}\_t)$$, the model estimates a full probability distribution $$(f(Y\_t \mid \mathcal{H}{t-1}))$$, which can take different parametric forms, such as a Gaussian: $$ Y\_t \mid \mathcal{H}\_{t-1} \sim \mathcal{N}(\mu\_t, \sigma\_t^2) $$ where $$(\mu\_t)$$ is the predicted mean and $$(\sigma\_t^2)$$ represents the uncertainty in the forecast. Probabilistic forecasting can be handled through various approaches, including **variance forecasters**, **quantile forecasters**, **interval forecasters** or **distribution forecasters**, each capturing uncertainty differently. For example, you can try to forecast the target location by a gaussian density function (or a mixture), thus the model output follows the form: ```python { "density": { "type": "builtin", "name": "norm", "params": { "loc": y_mean, "scale": y_var } }, "weight": weight } ``` A **mixture density**, such as the gaussian mixture $$\sum\_{i=1}^{K} w\_i \mathcal{N}(Y\_t | \mu\_i, \sigma\_i^2)$$ allows for capturing multi-modal distributions and approximate more complex distributions.
The meaning of probabilistic forecast is made more precise by means of the Python [density](https://github.com/microprediction/density) package which provides a dict specification of continuous univariate density function mixtures using the pydantic Python package. The function [validatedensitydict](https://github.com/microprediction/density/blob/main/density/validatedensitydict.py) will tell you whether or not your specification is valid. ## Create your Tracker A **tracker** is a model that processes real-time asset data to **predict future price changes**. It uses past prices to generate a **probabilistic forecast** of incremental returns. **You can use the data provided by the challenge or any other datasets to improve your predictions.** It operates incrementally: prices are pushed to the tracker as they arrive and predictions are requested at specific times by the framework. **To create your tracker, you need to define a class that implements the `TrackerBase` interface, which already handles:** * price storage and alignment via `PriceStore` * multi-resolution forecasting through `predict_all()` As a participant, you only need to implement **one method**: `predict()`. **Required method: `predict(self, asset: str, horizon: int, step: int)`** It must return a sequence of **predictive density distributions** for the **incremental price change** of an asset: * Forecast horizon: horizon seconds into the future * Temporal resolution: one density every step seconds * Output length: `horizon // step` Each density prediction must comply with the [density\_pdf](https://github.com/microprediction/densitypdf/blob/main/densitypdf/__init__.py) specification. {% hint style="info" %} You can refer to the [Tracker examples](https://github.com/crunchdao/crunch-synth/tree/master/crunch_synth/examples) for guidance. {% endhint %} ## Additional Resources * [Literature](https://github.com/crunchdao/crunch-synth/blob/master/LITERATURE.md) * Useful Python [packages](https://github.com/crunchdao/crunch-synth/blob/master/PACKAGES.md) [^1]: Still scored during the SP500 rollout; new prompts stop after the migration cutover, existing predictions age out naturally. [^2]: Equivalent to 500 / 4.34524, see why [here](https://www.cuemath.com/questions/what-is-the-average-number-of-weeks-in-a-month/). [^3]: Conversion rate is coming from taostats' API. [^4]: Conversion rate is coming from Coingecko's API. # Falcon: The Collective Pricing Engine Can you outsmart the falcons and predict where the dove will go next? ## Overview Enter a real-time forecasting game where players use probabilistic models to forecast the dove’s future location, based on the movement of the dove and the chasing falcons. Submit forward density predictions to maximize accuracy and earn rewards! It’s a blend of strategy, live data, and statistical forecasting. ## Data Your model must process a sequence of records that will be received in real time. Each record provides: * `time`: The current time; * `falcon_location`: A location of one falcon; * `dove_location`: The current dove location; * `falcon_id`: The falcon identity; * `falcon_wingspan`: A figurative measure of dexterity, precision and aggression whose utility or lack thereof is up to you to decide;
The falcon locations are shown as colored dots on the figure above. Some falcons may provide useful information as to (track) the future location of the dove, or the uncertainty of the same, whereas others may not. Their utility, or otherwise, is for you to determine. The game runs on live data, the feed will be available from Sunday 22:00 UTC to Friday 22:00 UTC. ### Expected Interruptions of the Data Feeds Each day, around 21:00 UTC, there is a window lasting 5–30 minutes during which data may be delayed, unavailable or the market might behave different than during other times in the day. This is caused by the underlying data streams going into a rollover/maintenance period. Your model should be prepared to handle this change in the underlying data gracefully to not loose wealth! ### Unexpected interruptions of the Data Feeds As the game relies on real-time data occasional outages may occur. During these periods, models will not receive new inputs. Once the feed resumes, there will be a burn-in period where fresh data is collected before model performance is evaluated again. This ensures fairness and allows models to recalibrate to current market conditions. ## Game Rules ### Start * Each player begins with a starting **wealth of 1000**. * The game opens with a **warm-up phase** where predictions are scored, but wealth is reset at the beginning of each day so players can calibrate their models. * This phase **will last until November 17**, giving players additional time to refine and test their approaches. * Players can enter and exit the game at any time. * Players have a single active model they can update at any time. ### Prediction Phase * For each prediction round, players automatically invest a **fraction of their active wealth** into the pot. * This amount is subtracted from their active wealth. * The total pot is **inflated** slightly by a game-defined **inflation rate**. * The model must **generate predictions in under 50 Milliseconds**. ### Scoring & Distribution * Once the **true dove location** is revealed, each prediction is scored using a **likelihood function**. * The pot is then distributed **proportionally** based on these likelihood scores. * More accurate predictions earn a **larger share** of the pot. * Player wealth will never go below 0. * Players can skip predictions. Doing so means they cannot lose or gain wealth, as they are not participating in prize distribution. ### Payouts * When a player’s wealth exceeds a defined **wealth threshold of 2000**, they receive a **prize payout** equal to **10% of their wealth**. * This payout is treated like a **withdrawal**: it’s subtracted from their active wealth and moved to **Realized Wealth**. * Realized Wealth is **distributed weekly**. A fixed pool of rewards is allocated, and each participant receives a share proportional to their earned wealth relative to the total earned by all players that week: $$ \text{Payout}*i = \frac{W\_i}{\sum*{j} W\_j} \times R $$ ### Game Duration * After the warm-up phase ends on November 17, the official game begins with real rewards. * The initial season **will run for 3 months of live play**, followed by the announcement of the winner and the start of a new season. * Rules and mechanics may evolve with player feedback, especially after the first season. ### Winning * The player with the most Total Wealth (The sum of Active Wealth and Realized Wealth) wins the game. ## Game State Explanation of the game state parameters displayed on the leaderboard for each player: * **Total Wealth**: The sum of a player’s *Realized* and *Active Wealth*. * **Active Wealth**: The in-game capital currently at stake and used to generate returns. * **Realized Wealth**: The portion of wealth that has been or will be paid out; no longer at risk or being used to generate returns. * **Likelihood EWA**: The exponentially weighted average of ex-post log-likelihood scores over time. * **Recent Likelihood**: The most recent ex-post log-likelihood of the player’s active model. * **Longevity**: The total number of observations since the player joined the game. * **Predicting**: Indicates whether the player’s model submitted predictions in the current round, can be Live / Idle. ## Probabilistic forecast Probabilistic forecasting provides **a distribution of possible future values** rather than a single point estimate, allowing for uncertainty quantification. Instead of predicting only the most likely outcome, it estimates a range of potential outcomes along with their probabilities by outputting a **probability distribution**. A probabilistic forecast models the conditional probability distribution of a future value $$(Y\_t)$$ given past observations $$(\mathcal{H}\_{t-1})$$. This can be expressed as: $$ P(Y\_t \mid \mathcal{H}\_{t-1}) $$ where $$(\mathcal{H}\_{t-1})$$ represents the historical data up to time $$(t-1)$$. Instead of a single prediction $$(\hat{Y}\_t)$$, the model estimates a full probability distribution $$(f(Y\_t \mid \mathcal{H}{t-1}))$$, which can take different parametric forms, such as a Gaussian: $$ Y\_t \mid \mathcal{H}\_{t-1} \sim \mathcal{N}(\mu\_t, \sigma\_t^2) $$ where $$(\mu\_t)$$ is the predicted mean and $$(\sigma\_t^2)$$ represents the uncertainty in the forecast. Probabilistic forecasting can be handled through various approaches, including **variance forecasters**, **quantile forecasters**, **interval forecasters** or **distribution forecasters**, each capturing uncertainty differently. For example, you can try to forecast the target location by a gaussian density function (or a mixture), thus the model output follows the form: ```python { "density": { "name": "normal", "params": { "loc": y_mean, "scale": y_var } }, "weight": weight } ``` A **mixture density**, such as the gaussian mixture $$\sum\_{i=1}^{K} w\_i \mathcal{N}(Y\_t | \mu\_i, \sigma\_i^2)$$ allows for capturing multi-modal distributions and approximate more complex distributions.
### Mathematical Definition The informal meaning of probabilistic forecast is a mixture of parametric univariate density functions where each is taken from a standard family (such as exponential, or gaussian).

A mixture of gaussian densities conspire to match a fat-tailed distribution.

### Engineering Definition The meaning of probabilistic forecast is made more precise by means of the Python [density](https://github.com/microprediction/density) package which provides a dict specification of continuous univariate density function mixtures using the pydantic Python package. The function [validatedensitydict](https://github.com/microprediction/density/blob/main/density/validatedensitydict.py) will tell you whether or not your specification is valid. ## Create your Tracker A tracker is a framework that processes real-time data to track the dove’s movement and predict its future location. It considers inputs like the dove’s position and falcon locations to generate a probabilistic forecast. To create your tracker, you need to define a class that implements the `TrackerBase` interface. Specifically, your class must implement the following methods: * `tick(self, payload: dict, performance_metrics: dict) -> None`\ This method is called at every time step to process new payloads. Use this method to update your internal state or logic as needed. * `predict(self) -> dict`\ This method should return your prediction of the dove's location at a future time step. Ensure that the return format complies with the [density\_pdf](https://github.com/microprediction/densitypdf/blob/main/densitypdf/__init__.py) specification. {% hint style="info" %} You can refer to the [Tracker examples](https://github.com/microprediction/birdgame/tree/main/birdgame/examples) for guidance. {% endhint %} ## Challenge your Tracker against the benchmark To compare your Tracker's performance against the benchmark Tracker, use the `test_run` method provided in the `TrackerBase` class. This method evaluates your Tracker's efficiency over a series of time steps using [density\_pdf](https://github.com/microprediction/densitypdf/blob/main/densitypdf/__init__.py) scoring. A higher score reflects more accurate predictions. ## Additional Resources * [Literature](https://github.com/microprediction/birdgame/blob/main/LITERATURE.md) * Useful Python [packages](https://github.com/microprediction/birdgame/blob/main/PACKAGES.md) # Participate To get started and submit your first model, you will need to pass through the following steps. ## Create your Solana Wallet Before you can participate in CrunchDAO's real-time competitions, you'll need to set up a Solana wallet. We recommend using the Phantom wallet for its ease of use and robust features. 1. **Download Phantom**\ Visit the [Phantom's "How to create a new wallet" post](https://phantom.com/learn/guides/how-to-create-a-new-wallet) and download the wallet extension for your browser or mobile app. 2. **Install and Setup**\ Follow the on-screen instructions to install Phantom. During the setup process, you'll create a new wallet, which includes generating a secure recovery phrase. 3. **Secure your Recovery Phrase**\ Write down your recovery phrase and keep it in a safe place. This phrase is essential for recovering your wallet if you lose access to your device. {% hint style="info" %} For testing and development on the Devnet, you'll need to add test SOL tokens to your wallet. Visit the official [Solana Faucet](https://faucet.solana.com/), enter your wallet address, connect to your GitHub account, and request the desired amount of test tokens. These tokens are provided for staging purposes only and have no real-world value. {% endhint %} ## Accept the Rules (Onchain/Offchain) Go to a real-time competition you want to join, which is indicated by a spinning clock icon.
You'll need to formally accept the competition rules. To do so, navigate to the "**Overview**" section.
And click the "**Read the rules**" button. This action will trigger a Solana transaction that records your acceptance on the blockchain. Additionally, your wallet's public key will be stored on our platform for future withdrawals, though you'll have the option to update this information later if necessary.
## Create a Model Next, it's time to create a model. Go to "**My Models**".
Click on "**+ New Model**", enter the name of your model and click on the "**Create**" button. Once this is done, you're ready to submit on this model.
Depending on the competition, you may have the option to create and submit one or multiple models. ## Submit your Code To submit your code, go to the "**Submit**" page.
Select your preferred submission method. [The submission process is the same as for other competitions.](/competitions/participate#submit)
## Deploy your Submission Submission is just as easy. Go to your models page where you'll find a new entry in the "**Submissions**" section. Simply click the "**Deploy**" button and follow Phantom's instructions to record the transaction on the blockchain.
By default, your deployment will appear under the "**Model Runner**" section as "**On**". You can adjust this status at any time by simply toggling the switch.
Once the deployment is complete, a new entry is added to the "**Deployments**" section where you can monitor and manage your deployed models. From there, you'll also have access to Model Runner logs to track performance and troubleshoot if necessary.
## Check your Runner/Builder Logs The logs are divided into two sections: * **Builder**: the logs generated while building the image of your model * **Runner**: the logs of your model running live
{% hint style="info" %} Only the last 100 lines are available. {% endhint %} # Competitions ## What are Competitions? The competitions are the real fuel of the Crunch Foundation. The data quality is top tier and clients offer big cash prize in order to crowdsource predictions. There are for now different format of collaboration: * Limited in time competitions: The client buy the models to the rewarded winners. * Continuous competitions: last in time. The client pays to query the model through our API to receive the community's prediction each hour/day/week/months. * Rallies: Limited in time. It allows clients to test a dataset and the performance of the community models before launching a Continuous Competition. * Code Competitions: Participant must submit the code of their model in order for the customer to query inferences from the submission. * Predictions Competitions: Participant submit a simple prediction not the model or the code of the model. ## Competition Stages A competition is structured in multiple stages: ### 1. The [Submission Phase](/other/glossary#submission-phase) Participants will have access to the training data and can submit their solutions to the challenge. A public leaderboard is usually available for you to use to compare yourself with other participants. ### 2. The Selection Phase The Selection Phase comes after the Submission Phase, which gives you extra time to make your selection for the Out-of-Sample phase. If you start a run at the very end of the Submission Phase, it gives you time to wait for it to finish before making your final selection. However, the submission period has now closed and it is no longer possible to create new runs. If you don't select anything, **the last run will be selected automatically**. ### 3. The [Out-of-Sample Phase](/other/glossary#out-of-sample-phase) The private leaderboard will be computed by running your model(s) on unseen data. Thanks to the Submission Phase, your submission is considered valid, and you should not encounter any issues when running it on the new dataset. However, the same Submission Phase rules apply: * If your code exceed the quota, your run will end. * If your code crashes, your run will end. A run that isn't successful means the end of the competition for you. #### Data update In rare cases where we decide to update the training data, we will ensure that it is sufficient to cover the new dataset size. This change may be unannounced, so your code must be able to handle it. This is why we ask for the training code in advance.\ \ Failure to provide this will result in you losing access to the new training data. However, most of the time, only the new data will be used for inference.\ This means that training will be skipped (as the training data will be the same), and you can use your entire quota for inference. #### Early contact If you are among the [top N](#user-content-fn-1)[^1] on the private leaderboard, we will contact you to complete the KYC process. This is necessary in order to be eligible to receive the prize. This ensures that we have everything ready to distribute the prize on time. Of course, you must not tell anyone that you were contacted.\ This is also why we are contacting N participants: * To not reveal the leaderboard too early. * To make sure that we have enough winners in case someone refuses their prize. #### Write up While waiting for the private leaderboard, we encourage all participants to write a description of their submission, research, and reasoning process on their preferred platform so that others can learn from it! The following platforms are often used: * [Notion.site](https://www.notion.com/product/sites) * [GitHub Pages](https://docs.github.com/en/pages/getting-started-with-github-pages/creating-a-github-pages-site) (code could also be included) * [Medium](https://medium.com) You can find examples from previous competitions: * [smoggy-mahcih's write up (ADIA Lab Causal Discovery Challenge)](https://thetourney.github.io/adia-report/) * [mutian-hong's write up (ADIA Lab Causal Discovery Challenge)](https://stream-physician-14c.notion.site/ADIA-Lab-Causal-Discovery-Challenge-Rank3-Solution-1397f010c9428099aa82e4503cad1c20#1457f010c94280cfb541c60cc9f55b97) * [semantic-alexander's write up (ADIA Lab Causal Discovery Challenge)](https://medium.com/@alexkiechlecornish/4th-place-solution-100k-causal-discovery-challenge-adia-lab-x-crunchdao-e0f88c9eda8e) * [hi-cdoz's write up (ADIA Lab Causal Discovery Challenge)](https://medium.com/@htnu/the-5th-place-solution-to-the-adia-causal-discovery-challenge-2024-236d5036e03a) * [datatech team's write up (ADIA Lab Structural Break Challenge)](https://messy-plain-a65.notion.site/ADIA25-211402a1a1b780428471eaed714e285b) * [farukcan-saglam's write up (ADIA Lab Structural Break Challenge)](/competitions/competitions/adia-lab-structural-break-challenge) When you're ready to share your write-up, specify the URL in the model settings:

The Model Settings dialog is available by clicking the Edit button on your model page.

{% hint style="danger" %} Data cannot be shared directly, whether in file format or as visualizations.\ Only your work should be explained, and the visualizations should focus on your model. {% endhint %} ### 4. The Reveal The reveal will happen at the very end, after the private leaderboard is published. The rankings will then be final. The prizes will be distributed in the coming days. If you are one of winner, we suggest reading the [Prizes Winners documentation](/competitions/faqs/prize-winners) to know what to do next. [^1]: e.g. top 30 if the prize reward the top 10 # ADIA Lab Structural Break Challenge: Real Time Edition Monitor time series in real time and detect when their behaviour changes. ## Overview Detecting structural changes in time series data in real time is a critical task across various scientific and engineering domains. In this competition, you monitor a stream of univariate time series data one observation at a time and, after each new observation, report how confident you are that a structural break has **already occurred** somewhere in the online segment up to and including the current step. ## Problem Statement The task of this competition is to **monitor a univariate time series in real time** and, at each new observation, quantify whether a permanent structural break has already occurred somewhere up to that point. Each series is comprised of a long historical segment (1,000 to 5,000 observations, with no break), and an online segment (10 to 1,000 observations, possibly with a structural break). The observations in this online segment are revealed one at a time: after each of them, your detection algorithm must output a score between `0` and `1`, reflecting cumulative confidence that a structural break has already occurred - `0` if absolutely confident no break has occurred, `1` if absolutely confident a break has already occurred. The training data (with known structural break locations) combines a large collection of synthetic and real-world time series exhibiting a wide variety of break types - including changes in mean, variance, distributional shape, and dependence structure. Submissions are evaluated on an independent test set using the Time-Stratified AUC (`TS-AUC`): at each online time step, a standard AUC is computed cross-sectionally across all series, and the weighted average over time steps is the final score. ### Differences from the 2025 Edition If you participated in the 2025 edition, the core concept is the same, but the mechanics are fundamentally different:
20252026 (this one)
Data deliveryBoth segments given at onceOnline segment arrives one step at a time
Break locationAlways at the known boundaryUnknown -- anywhere in the online segment
OutputOne score per seriesOne score per time step
This edition mirrors a realistic monitoring scenario: you watch a stream of data and, after each new observation, you report how confident you are that the process has already changed. ## Competition Timeline * Start Date: May 6th, 2026 at 4:00 p.m. UTC * Quota Refresh: every Wednesday at 4:00 p.m. UTC * End Date: September 17th, 2026 at 4:00 p.m. UTC (Thursday)[^1] * Final Evaluation: End of October, 2026 * Winners Announcement: [During the ADIA Lab 2026 Symposium](https://www.adialab.ae/upcoming-events/adia-lab-symposium-2026) (26–28 October) ## What is a Structural Break? A **structural break** occurs when the statistical behaviour of a time series changes permanently at some point in time. Before the break, the data follows one process; from the break onwards, it follows a different one. The figure below shows a simple example: the series has a constant mean before the break and a different mean after it. The dashed line marks the break time.
Example: a time series with a structural break. The dashed line marks the break.

Example: a time series with a structural break.

Structural breaks appear in many domains: * **Climatology:** shifts in weather patterns that may signal climate anomalies or long-term change. * **Industry:** changes in machinery sensor readings that anticipate equipment failures or maintenance needs. * **Healthcare:** sudden changes in physiological signals that may indicate critical health events. * **Finance:** shifts in market or strategy behaviour relevant to risk management and portfolio decisions. ## Dataset The dataset contains a large and diverse collection of univariate time series exhibiting many different kinds of structural breaks -- changes in mean, variance, distribution shape, correlation structure, and more. All series are pre-processed into a common z-scored format. Each series is split into two parts, a **historical segment** and an **online segment**.
Time series structure: historical segment (blue), online segment before the break (green), online segment after the break (red). The dashed vertical lines mark the start of the online segment and the break position.

Time series structure

### **Historical segment** The **historical segment** is a long reference sequence provided to you in full at the start, typically between 1,000 and 5,000 observations. It represents the behaviour of the series before any potential break. ### Online segment The **online segment** follows the historical segment and is revealed to you **one observation at a time,** typically between 10 and 1,000 observations long. After you submit your score for the current observation, the next one is released. There is no way to look ahead. Each series contains **at most one** structural break, and the historical segment is always break-free. Any break, if present, falls somewhere within the online segment: * With probability `0.5`, a break occurs at some unknown point within the online segment. You must infer its position from the data. * With probability `0.5`, no break occurs during the online segment at all. ### Break position For the training set only, the **break position** `tau` is a 0-indexed position within the online segment at which the break occurs, or `None` if no break occurs. ### Data Size The dataset is divided into multiple parts: | Split | Availability | Number of series | | ----------------------------- | ------------- | ---------------- | | Public training set | Local & Cloud | 10,000 | | Public **(reduced)** test set | Local | 100 | | Public test set | Cloud | 10,000 | | Private test set | Cloud | 10,000 | [True values](#user-content-fn-2)[^2] are only available for the training set (both locally and in the cloud) and the reduced test set (only locally). ## Scoring The competition uses a single metric: **Time-Stratified** [**AUC**](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html) **(TS-AUC)**. At each online time step $$t$$, the metric computes a standard AUC cross-sectionally across all series alive at that step: * A series is **positive** at step $$t$$, if the break has already occurred by that step (ideal score = `1`). * A series is **negative** at step $$t$$, otherwise (ideal score = `0`). The TS-AUC is the weighted average of these per-step AUCs, with weight $$w(t) = n\_\text{pos}(t) \cdot n\_\text{neg}(t)$$ (the number of positive-negative pairs at step $$t$$): $$ \text{TS-AUC} = \frac{\sum\_t w(t),\text{AUC}(t)}{\sum\_t w(t)} $$ * `0.5`: equivalent to random guessing.\ To score above 0.5, a predictor must use the content of the series: at every fixed $$t$$, the metric compares series against each other, so a score that does not depend on the series cannot discriminate. * `1.0`: perfect detection. ## Code Submission This is a code competition where participants are required to submit their Python code (files or notebooks) directly to the Crunch Hub. Your submission should: 1. Process and analyze the data; 2. Output a score between `0` and `1` for each time series steps in the test set, representing the likelihood of a structural break; 3. Your code must produce deterministic output, or it will be ineligible for any rewards; 4. If you participate as a team, only [the team leader](/competitions/teams#leaders) will be ranked on the leaderboard, [rewards are split among all members](/competitions/teams/rewards). Your submitted code will be executed on the platform and automatically scored against a portion of the test set. Shortly after submission, your score will appear on the public leaderboard of the competition.

Visual animation.

### Interface At each new online observation, produce a **score between `0` and `1`** representing your cumulative confidence that a structural break has **already occurred** somewhere in the online segment up to and including the current step: * **`0`**: no break detected so far. * **`1`**: a break has definitely already occurred. You produce one score per time step, so for a series with an online segment of length `T` you output `T` scores. 
def infer(
    datasets: Iterable[Tuple[List[float], Iterable[float]]],
    model_directory_path: str,
):
    """
    Load your trained model, then use the `yield` keyword to indicate that it is ready.
    Then iterate over the datasets and points to provide a result using `yield <prediction>`.

    Args:
        datasets: the data object to iterate.
        model_directory_path: the path to the directory where you model has been saved in the train function.
    """

    model = joblib.load(os.path.join(model_directory_path, 'model.joblib'))

    # Mark as ready
    yield

    for x_historical, x_online in datasets:
        for point in x_online:

            # Consume the point (float)
            result = model.consume(point)

            # Provide your result, one at a time
            yield result
There are a few constraints on the data that will stop your code if you try to ignore them: * You must provide your result before you can get the next point from `x_online`. * The online segment cannot be read twice. * You must `yield` at each point. {% hint style="warning" %} Rely on the testing tool to make sure your code is working as intended locally. {% endhint %} ### Requirements Your solution must include two functions: * `train()`: to train your model on the training set.\ [You must provide it if your model requires training](/competitions/faqs#can-i-train-a-model-locally). If not, you can leave it empty. * `infer()`: to returns predictions on the test set. The execution time of your solution should not exceed the platform's time limits: **15 hours per week**. Your solution must be deterministic: when [**re-run on 10% of the data**](#user-content-fn-3)[^3], the predicted values should be the same (within a **tolerance of 1e-8**). #### What a good score sequence looks like The ideal score is a step function: it stays at `0` as long as no break has occurred, then jumps to `1` as soon as the break happens. If there is no break, the ideal score is `0` throughout. The figure below shows how your detection algorithm's score sequence (solid line) compares to the ideal sequence (dashed step). The shaded area between them reflects how early and how cleanly the break was detected.
Evaluation example: participant scores (solid) vs ideal step function (dashed). A good submission keeps the shaded area small.

Evaluation example

### Computational note With 10,000 series and up to 1,000 online steps each, solutions that recompute everything from scratch at every step may run into time budget constraints. Incremental approaches, which are maintaining a compact running state and updating it with each new observation, are worth considering for efficiency, though any solution that fits within the time budget is acceptable. ### Parallelism Infering so many points can be slow. That is why we offer a parallel approach, but **only at the time series level**. Your model **must still process each point separately**. Depending on your model's capacity, the dataset will be split into n equal parts. Your model will start n times **in different processes** [(not threads)](#user-content-fn-4)[^4], and each process will receive and fully process one part. #### How to use it To ensure optimal performance, your model must follow a few restrictions: * Because of the concurrency, your model should avoid writing any files. * Make sure the cloud environment can handle the RAM and CPU consumption of your model. * If you want 6 processes and your model consumes 4 GB of RAM, the runtime must have 4 \* 6 = 12 GB of RAM plus some overhead. * The same applies to CPU cores. Overallocation can actually decrease performance. You can enable parallel processing by simply specifying the number of workers you want via a global constant: {% code title="Python Cell" expandable="true" %} ```python # @crunch/keep:on INFER_PARALLELISM = 4 ``` {% endcode %} {% hint style="info" %} The [`@crunch/keep:on` command](https://docs.crunchdao.com/competitions/participate/notebook-processor#automatic-line-commenting) is only required for notebook users to prevent the line from being commented out. Keep the constant in a dedicated cell, or add `@crunch/keep:off` after the assignation. {% endhint %} #### Known issues 1. Exceptions and Crashes: When using multiple processes that all print to a single terminal, it is expected that lines will mix with each other. This makes errors harder to debug, as traces cannot be printed properly. We have made sure to report the first error trace, but subsequent errors will be ignored. 2. CPU over-allocation: NumPy uses [OpenBLAS](https://github.com/OpenMathLib/OpenBLAS) behind the scenes to try to parallelize some computations which could potentially conflict with the parallelism mechanism. To help resolve this issue, we recommend using [threadpoolctl](https://pypi.org/project/threadpoolctl/) at the correct location(s). ## Methodology Suggestions * **Statistical tests** comparing the distribution of the historical segment to the online observations seen so far (t-tests, KS tests, CUSUM). * **Change-point detection algorithms** designed for online or streaming data. * **Feature extraction** summarising the online window incrementally, fed into a trained classifier. * **Probabilistic and Bayesian models** tracking the likelihood of a change sequentially. * **Deep learning** models trained to score (series, time step) pairs using the labeled training data. * **Foundation models for time series** pre-trained on large corpora, used as feature extractors or fine-tuned on the labeled training data. Whatever approach you choose, the training set provides full supervision: the known break positions let you construct labeled (series, time step) pairs and apply standard binary classification training. ## Prizes All prizes are in [USDC](https://www.usdc.com/), a cryptocurrency with the same value as the US dollar.
Winners’ rankPrize value
1st place$40,000
2nd place$20,000
3rd place$10,000
4th place$5,000
5th place$5,000
6th place$5,000
7th place$5,000
8th place$3,500
9th place$3,500
10th place$3,000
## FAQ
What data is used to compute the mean and standard deviation for standardizing each series? **Only the historical (reference) segment**, not the online period or the full series.
Is standardization done separately for each series? **Yes**, each series is standardized independently.
Are the normalization parameters (mean/std) updated once the online period starts? **No**. They are fixed before the online stream begins and never updated afterward.
Are real and synthetic series normalized the same way? **Yes**, Both go through the same standardization pipeline.
Is the reference window assumed to be free of structural breaks or irregularities? **Yes and no**: there are no structural breaks by definition (the reference window defines what is "normal"), but there can be irregularities (jumps, etc.).
Can I use an AI assistant or LLM to help with the competition? **Yes**, as long as you don't copy the full dataset into its context.
[^1]: Sept. 16 will be the last quota refresh. [^2]: Also known as Y train/test. [^3]: Only applies to inference, not training. [^4]: This means that memory is not shared. Your running model cannot communicate with other running models. # DataCrunch Equity Market Neutral #2 This weekly cross-sectional problem target the expected returns of the 3000 most liquid US equities. [DataCrunch](https://datacrunch.com/) uses the quantitative research of the CrunchDAO to manage its systematic market-neutral portfolio. The long-term strategic goal of the fund is capital appreciation by capturing idiosyncratic return at low volatility. In order to achieve this goal, DataCrunch needs the community to assess the relative performance of all assets in a subset of the [Russell 3000](https://www.investopedia.com/terms/r/russell_3000.asp) universe. In other words, DataCrunch is expecting your model to predict the performance of the constituent of its investment universe. This dataset is the new of the original DataCrunch dataset that can be found [here](https://hub.crunchdao.com/competitions/datacrunch). To read more about the evolutions between #1 and #2 of the DataCrunch datasets, please [read the full article](/competitions/competitions/datacrunch-2/from-dataset-1-to-dataset-2). ## Reward Scheme DataCrunch is distributing 1,000 USDC every weeks. The reward scheme is calculated as follows for both metrics: {% code title="Pseudo code (Python flavored)" %} ```python # Reward Calculation weekly_rewards = 1000 # 0 = worst, 1 = best percentile_rank = your_rank / nb_participants if percentile_rank <= 0.5: reward = 0 else: # excess above median, scaled 0–1 e = 2 * (percentile_rank - 0.5) weight = e ** 20 reward = weekly_rewards * (weight / sum(participants_weights)) ``` {% endcode %}
All rewards are computed on the leaderboards. The Historical Rewards are the sum of every payout you have received from DataCrunch. The Projected Rewards are the current estimated rewards yet to be distributed. ## Dataset Each row of the dataset represents a single stock at a given weekly timestamp. ### `X_train` * `moon`: A sequentially increasing integer representing a date. Time between subsequent moons is constant, denoting a weekly fixed frequency at which the data is sampled. * `id`: A unique identifier representing a stock at a given `moon`. Note that the same asset has a different `id` in different `moon`. * `Feature_1`, …, `Feature_1150`: Anonymised features that describe the state of assets on a given `moon`. They are ways of assessing the relative performance of each stock on a given `moon`. ### `y_train` * `moon`: Same as in `X_train`. * `id`: Same as in `X_train`. * `target`: The targets that will help you build your models. It is derived from the 28 days forward returns. ### `X_test` and `y_test` `X_test` and `y_test` have the same structure as `X_train` and `y_train` but comprise only one `moon` at each iteration. These files are used to simulate the submission process locally via `crunch.test()` (within the code), or `crunch test` (via the CLI). The aim is to help participants debug their code and have successful submissions. A successful local test usually means no errors during execution on the submission platform. ### Embargo The embargo is defined by the length of the target and is thus 4 moons. ## Tournament Structure ### Data Splits The DataCrunch competition follows a fixed and transparent structure: * Local data: * The first 15 years of the dataset. A smaller version of the latest 2.5 years of these 15 years is provided for small configurations. * Cloud data: * A public Out-of-Sample allowing participants to test your submission in the cloud. This data is on the first two month of 2020 year. * A private Out-of-Sample allowing DataCrunch to have historical performance of the models in order to do meta-modelling and ensemble research. (see below) * The last available date of the dataset will be scored on a weekly basis, after the target is resolved. {% hint style="info" %} Your code will have access to the entire dataset when running in the cloud. {% endhint %} ### Submission Cut-off The window to submit and lock your model is removed. You can submit your code / model whenever you want. If the run completes before Sunday 12pm UTC, it will be taken into account for the week. Otherwise, the run will be terminated and ignored for the week. The prediction target takes 1 day + 4 weeks to be fully resolved, so the score for a model submitted in week #1 will be available and published in week #6. ### Private Historical Performance Dataset The first time your model is run for the Out-of-Sample, you must first infer a mandatory \~300-moon test set, which is not scored but used for historical performance analysis. After that, you will only need to infer one moon per week. If you decide to submit (and select) a new model, you will need to predict this dataset again. ### Train Frequency The train frequency is representing when your model's `train()` function will be called: * If set to 0, `train()` will never be called. * If set to 1, `train()` will be called at every moon. * If set to 2, `train()` will be called at every even moon. * If set to 10, `train()` will be called at every tenth moon. The number is based on the modulo of the moon itself, not the one of the iteration, meaning that a train frequency of 5 will run at moon 300, 305, 310, ... It is not based on the loop. If the cloud environment starts at moon 303, only after two moon (305) the `train()` function will be called. You can put frequency, but the smaller, the more time the `train()` function will be called, and the longer your code will take to run. Some models may not be able to run within the time constraint, considering that you must first pass the private historical performance dataset. When you run your model for the first time, you can choose the train frequency. This frequency will be used for the Out-of-Sample phase. It is not possible to change the frequency during an Out-of-Sample phase. If you need to change the frequency, you must submit again.

train() function calling matrix for a moon and a train frequency

## Scoring and Evaluation Participants are evaluated using the [Pearson correlation](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient) between their predictions and the target on the last date of the dataset. As mentioned in the Submission Cut-off section, it takes 5 weeks for a prediction to be scored and appear on the leaderboard. ## Computing Resources Participants will be allocated a specified quantity of computing resources within the cloud environment for the execution of their code. Participants are entitled to **15 hours** of GPU or CPU compute time per week. The competition being weekly, you will have to manage your weekly computing resources consumption to comply with this constraint. Quota will be reset each Sunday at 12pm UTC. ## Getting Started Participants can begin by implementing the required `train` and `infer` functions and validating locally using the [provided quickstarter notebook](https://colab.research.google.com/github/crunchdao/quickstarters/blob/master/competitions/datacrunch-2/quickstarters/quickstarter/quickstarter.ipynb). The API is as follow: {% code title="Python Notebook Cell" expandable="true" %} ```python def train( X_train: pd.DataFrame, y_train: pd.DataFrame, model_directory_path: str, loop_moon: int, embargo: int, ) -> None: """ At each retrain this function will have to save an updated version of the model under the model_directory_path, as in the example below. Note: You can use other serialization methods than joblib.dump(), as long as it matches what reads the model in infer(). Args: X_train, y_train: the data to train the model. model_directory_path: the path to save your updated model. loop_moon: the moon currently being processed. embargo: data embrago. Returns: None: Returned value is ignored. """ ``` {% endcode %} {% code title="Python Notebook Cell" expandable="true" %} ```python def infer( X_test: pd.DataFrame, model_directory_path: str, loop_moon: int, embargo: int, ) -> pd.DataFrame: """ This function will load the model(s) saved at the previous iteration and use it/them to produce your inference on the current moon. It is mandatory to send your inferences with the ids and moon so the system can match it correctly. Args: X_test: the independant variables of the current moon passed to your model. model_directory_path: the path to the directory to the directory in wich we will be saving your updated model. loop_moon: the moon currently being processed. embargo: data embargo. Returns: A pd.DataFrame (moon, id, prediction) with the inferences of your model for the current moon. """ ``` {% endcode %} {% hint style="info" %} Running a local test will validate your API. All parameters are optional and be commented if not needed. If the test passes locally, it will also pass in the cloud environment. {% endhint %} # From Dataset #1 to Dataset #2 ## Datacrunch New Data Release ⍺H With this released Datacrunch may come with one of the most significant dataset upgrades since the fund launched. ⍺H is built around the team internal strategy and represents a fundamental return to the source of the original competition setup: Simpler metric, longer datasets and one target to rule them all. This upgrade brings the competition dataset into closer alignment with our internal production strategy. The changes, informed by extensive backtesting and confirmed under live trading, remove legacy constraints and consolidate around the signals that demonstrate the strongest and most stable correlation to returns. For participants, this means more transparent evaluation and a clearer understanding of what drives success. ## What's New in ⍺H ### Extended Historical Coverage: More data, More Signal The dataset now reaches further back in time, capturing a wider range of market conditions. This matters because models trained only on recent data tend to overfit to the prevailing regime, whether that's a low-volatility bull market, a crisis period, or a recovery phase. By including more years of history, ⍺H as more memory and exposes models to recessions, rate cycles, sector rotations, and volatility spikes they might otherwise never encounter during training. The team's internal Benchmark’s results are more robust and hold up when market conditions inevitably shift. ### Bigger Cross-Sectional Universe: More assets, more edge Each weekly snapshot now contains more stocks than ever before. A larger cross-section strengthens the statistical foundation for relative rankings, when you're predicting which stocks will outperform, having more candidates creates cleaner separation between winners and losers. It also reduces the risk of overfitting to idiosyncratic behavior in a narrow universe. For a market-neutral strategy that relies on going long some stocks and short others, breadth is essential for constructing diversified portfolios that isolate alpha from market beta. ### Refined Features: Less is more The 1,150 anonymized features have been reworked based on our internal Benchmark’s performance. Some features that seemed promising in backtests but failed to generate alpha in production have likely been removed or transformed. Others may have been re-scaled, combined, or orthogonalized to reduce redundancy. While participants still can’t identify exactly what each feature represents, the refinement improves signal-to-noise ratio and makes the feature space more tractable for machine learning models. ### One target survived: 28-Day Forward Returns The prediction target has fundamentally changed. Rather than 4 return horizons, models need now to focus all their attention to forecast performance over a 28-day window. This longer horizon smooths out short-term noise and market microstructure effects, focusing instead on moves driven by fundamental revaluation. It also aligns better with the fund’s weekly rebalancing. Finally it avoids participants to compromise between horizons and focus all their attention to one and only one target. The four-moon embargo period directly corresponds to this target window, preventing any information leakage. As always only live performance will prevail. ### Dataset Architecture The data maintains a weekly frequency, with each row representing a single stock at a given timestamp. Features are anonymized (1,150 in total) and describe each stock's relative state at that moment. The "moon" identifier marks time periods sequentially, while stock IDs are unique to each moon: the same company receives different IDs across different time periods. The embargo period spans four moons, matching the target's forward-looking window. ## New Tournament Mechanics: ### No more submission Window The submission window constraint is gone. Participants can submit whenever they want, and any run completed before Sunday at 12pm UTC counts for that week. Given the target's resolution timeline, scores appear five weeks after submission. ### A Stronger Reward Scheme Toward Top Performers The payout structure now follows an aggressive power-law distribution. Below-median performers receive nothing, and above median, rewards scale with the 20th power of excess rank. To put this in perspective: a participant at the 75th percentile captures far less than someone at the 90th, who in turn captures far less than someone at the 99th. This concentration has strategic implications: incremental improvements near the top are worth dramatically more than climbing from average to good. ### Staking incoming As many of you requested the feature the Staking will arrive in Q1 2026. Participants will be able to put tokens behind their models, creating direct economic exposure tied to signal quality. The team views this skin-in-the-game mechanism as a great tool for trusting models faster under this new dataset and with larger capital allocations. ### Getting Started New participants can begin with the provided quickstarter notebook, implementing the required `train` and `infer` functions and validating locally before submitting to the cloud environment. Weekly compute allocation stands at **15 hours** of GPU or CPU time, resetting each Sunday. # Obesity ML Competition: Tackling Metabolic Diseases Can you design algorithms that identify genes driving obesity and metabolic disease? You will get to work with cutting-edge biological data collected in partnership with the [**Eric and Wendy Schmidt Center at the Broad Institute**](https://www.broadinstitute.org/), the [**Broad Diabetes Initiative**](https://www.broadinstitute.org/diabetes/diabetes-genetics-initiative), [**Massachusetts General Hospital**](https://www.massgeneral.org/), and [**Beth Israel Deaconess Medical Center**](https://www.bidmc.org/). Your algorithms could directly guide biological discoveries in obesity! ## Quick TL;DR * **The Goal**: Identify genetic "switches" that can trick human cells into burning fat instead of storing it. * **The Data Science**: You will be given Single-Cell RNA sequencing (scRNA-seq) data from cells where specific genes have been knocked out (turned off) using CRISPR/Cas9. * **The Challenge**: Build a model to predict how a cell’s behavior and development change when a new, unseen gene is turned off. ## Introduction ### The Biology: Storing Energy vs. Burning Energy Not all body fat is created equal. Our bodies primarily rely on two types of fat cells (adipocytes) to manage energy: 1. **White Fat**: The "storage" cells. They hold onto excess energy from food. When we store too much, it leads to obesity and metabolic diseases like Type 2 Diabetes. 2. **Brown Fat**: The "furnace" cells. Instead of storing energy, they burn sugar and fat to generate heat (thermogenesis). Most current obesity drugs work by making people feel less hungry. However, scientists are exploring a different approach: **What if we could convince the body to produce more brown fat, or trigger white fat to start burning energy?** ### The Experiment: Flipping Genetic Switches To find the biological "switches" that control these processes, researchers are conducting massive experiments. They harvest human fat-cell precursors and use CRISPR/Cas9 technology to turn off specific genes, one by one. They then observe: * Does the cell still become a fat cell? * Does it act like a white fat cell (storage) or a brown fat cell (burning)? * How does the cell's internal machinery (gene expression) change? ### Why cannot we physically test every single gene in the human genome? There are 20,000 genes in our genomes! It would take too much time and money. We need Machine Learning to fill in the gaps. By training on the existing experimental data, your model will predict the biological outcome of turning off genes that haven't been tested yet. ### Why this Matters? Obesity affects over 890 million people worldwide and is a primary driver of cardiovascular disease and cancer. While weight-loss drugs exist, they don't work for everyone and often carry side effects. To develop better therapies, we need to understand the fundamental code of metabolism: * Which genes tell fat cells how to develop? * Which genes help turn white fat cells into brown/heat-producing fat cells? * Which genes change how cells store or burn fat? If your model can successfully simulate how perturbing genes affect fat cells, it will allow researchers to screen thousands of potential drug targets purely computationally. This could dramatically accelerate the discovery of medicines that restore metabolic balance and fight disease. ### The Challenge In this competition, you are working with single-cell RNA sequencing data, which indicates how much each gene is expressed in each single cell. Scientists have "knocked out" (deleted) different genes in fat-cell precursors and measured how the cells changed. Your job is to **build a model that predicts what happens when scientists turn off new genes that appear in the test set**. You must predict two specific outcomes for these unseen gene knockouts: #### 1. The "Internal State" (Gene Expression Profiles) You must predict the gene expression profile of single cells after the target gene is knocked out. This is a high-dimensional vector representing the activity levels of thousands of genes within the cell. #### 2. The "Cell Identity" (Cell Type differentiation) Normally, these precursor cells differentiate into specific types of fat cells. You must estimate the resulting proportions of four distinct cell states: * **Pre-adipocyte**: Early-stage cells that haven't differentiated yet. * **Adipocyte**: Mature fat cells (the standard white fat). * **Lipogenic**: Specialized fat-producing cells. * **Other**: Cells that followed a different developmental path. {% hint style="success" %} Explore the [full specifications](/competitions/competitions/broad-obesity/full-specifications) for in-depth details. {% endhint %} ### Crash Course Similar to last year, the Eric and Wendy Schmidt Center team [prepared an 80-minute lecture series](https://www.youtube.com/playlist?list=PLlMMtlgw6qNi-WSkY-UiJMvO2TTa0T4Ga) containing a biology overview needed for this challenge. {% embed url="" %} ## Phases The challenge is broken down into three Crunches. ### Crunch 1 – Predicting the effect of held-out single-gene perturbations From Dec 8 to Feb 28, Crunchers will build a model to predict the single-cell transcriptomic response to unseen single-gene perturbations. ### Crunch 2 – Predicting the effect of held-out double-gene perturbations From Feb 28 to April 13, Crunchers will build a model to predict the single-cell transcriptomic response to unseen double-gene perturbations. It will be very similar to the Crunch 1, so we highly encorage participation to both! {% hint style="success" %} More details about this challenge will be announced soon! {% endhint %} ### Crunch 3 – Identifying combinatorial perturbations to drive white and brown adipocyte differentiation Crunchers will predict combinatorial perturbations to drive adipocyte differentiation, and The Eric and Wendy Schmidt Center will test these perturbations directly in the lab! {% hint style="success" %} More details about this challenge will be announced soon! {% endhint %} ## Timeline * **December 2025:** * Beginning of Crunch 1 * **February 2026:** * Closing of Crunch 1 * Beginning of Crunch 2 * **April 2026:** * Closing of Crunch 2 * **May 2026:** * Beginning of Crunch 3 * **July 2026:** * Closing of Crunch 3 ## Evaluation Criteria For each Crunch, participants must submit predictions in `h5ad` format along with matrices containing the predicted cell state proportions for each perturbation. Outputs will be evaluated using: * **Pearson Delta** (Crunch 1 & 2) * **Maximum Mean Discrepancy or MMD** (Crunch 1 & 2) * **L1-Distance** (Crunch 1 & 2) To avoid overfitting, Crunch will only publish the public leaderboard once each week. ## Prizes All parts will use multiple metrics to rank participants. Your final prize will be the sum of the prizes for each metric that you or [your team](/crunch-hub/teams) ranks in. Only those who achieve a score above the baseline will be eligible for a prize. All prizes are in [USDC](https://fr.usdc.com/), a cryptocurrency with the same value as the US dollar. ### Crunch 1 | Winner's Rank | Pearson Delta | MMD | L1 Distance | | ------------- | ------------- | ----- | ----------- | | 1st place | 1,400 | 1,400 | 1,400 | | 2nd place | 800 | 800 | 800 | | 3rd place | 480 | 480 | 480 | | 4th place | 320 | 320 | 320 | | 5th place | 240 | 240 | 240 | | 6th place | 200 | 200 | 200 | | 7th place | 200 | 200 | 200 | | 8th place | 160 | 160 | ~~160~~ | | 9th place | 120 | 120 | ~~120~~ | | 10th place | 80 | 80 | ~~80~~ | | Total | 4,000 | 4,000 | 3,640 | {% hint style="info" %} Ranks 8, 9 and 10 for the L1 Distance metrics were not rewarded as they did not managed to beat the baseline. {% endhint %} ### Crunch 2 | Winner's Rank | Pearson Delta | MMD | L1 Distance | | ------------- | ------------- | ----- | ----------- | | 1st place | 1,400 | 1,400 | 1,400 | | 2nd place | 800 | 800 | 800 | | 3rd place | 480 | 480 | 480 | | 4th place | 320 | 320 | 320 | | 5th place | 240 | 240 | 240 | | 6th place | 200 | 200 | 200 | | 7th place | 200 | 200 | 200 | | 8th place | 160 | 160 | 160 | | 9th place | 120 | 120 | ~~120~~ | | 10th place | 80 | 80 | ~~80~~ | | Total | 4,000 | 4,000 | 3,800 | {% hint style="info" %} Ranks 9 and 10 for the L1 Distance metrics were not rewarded as they did not managed to beat the baseline. {% endhint %} ### Crunch 3 | Winner's Rank | Total Prize Pool | Evaluation 1 (10%) | Evaluation 2 (90%) | | ------------- | ---------------- | ------------------ | ------------------ | | 1st place | 7,000 | 700 | 6,300 | | 2nd place | 6,500 | 650 | 5,850 | | 3rd place | 5,000 | 500 | 4,500 | | 4th place | 3,000 | 300 | 2,700 | | 5th place | 1,000 | 100 | 900 | | 6th place | 900 | 90 | 810 | | 7th place | 800 | 80 | 720 | | 8th place | 700 | 70 | 630 | | 9th place | 600 | 60 | 540 | | 10th place | 500 | 50 | 450 | | Total | 26,000 | 2,600 | 23,400 | ## External Resources Crunchers are encouraged to use publicly available external resources, including gene perturbation datasets and pre-trained models, as long as they are properly credited. ## References Below are a few references meant to provide more background and some of the approaches researchers are applying in the fields relevant to these Crunches. This is not meant to be an exhaustive list and many important works are not listed here.
Single cell transcriptomic adipocytes datasets * [Dissecting the impact of transcription factor dose on cell reprogramming heterogeneity using scTF-seq](https://www.nature.com/articles/s41588-025-02343-7) * [A single-cell atlas of human and mouse white adipose tissue](https://www.nature.com/articles/s41586-022-04518-2) * [Human subcutaneous and visceral adipocyte atlases uncover classical and nonclassical adipocytes and depot-specific patterns](https://www.nature.com/articles/s41588-024-02048-3) * [Adipose tissue retains an epigenetic memory of obesity after weight loss.](https://www.nature.com/articles/s41586-024-08165-7) * [Unveiling adipose populations linked to metabolic health in obesity](https://www.cell.com/cell-metabolism/fulltext/S1550-4131\(24\)00452-2?_returnURL=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1550413124004522%3Fshowall%3Dtrue) * [Spatial mapping reveals human adipocyte subpopulations with distinct sensitivities to insulin](https://www.cell.com/cell-metabolism/fulltext/S1550-4131\(21\)00363-6?_returnURL=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1550413121003636%3Fshowall%3Dtrue) * [snRNA-seq reveals a subpopulation of adipocytes that regulates thermogenesis](https://www.nature.com/articles/s41586-020-2856-x) * [Wnt signaling preserves progenitor cell multipotency during adipose tissue development](https://www.nature.com/articles/s42255-023-00813-y) * [Adipogenic and SWAT cells separate from a common progenitor in human brown and white adipose depots](https://www.nature.com/articles/s42255-023-00820-z) * [Single-Nucleus Analysis of Human White Adipose Tissue Reveals Adipocyte Subsets with Distinct Metabolic Profiles](https://www.biorxiv.org/content/10.1101/2025.09.14.673351v1) * [Mapping the transcriptional landscape of human white and brown adipogenesis using single-nuclei RNA-seq](https://www.sciencedirect.com/science/article/pii/S2212877823000807?via%3Dihub)
CRISPR/Cas9 genetic perturbation screens * [Mapping information-rich genotype-phenotype landscapes with genome-scale Perturb-seq](https://www.sciencedirect.com/science/article/pii/S0092867422005979?via%3Dihub) * [Exploring genetic interaction manifolds constructed from rich single-cell phenotypes](https://www.science.org/doi/10.1126/science.aax4438) * [X-Atlas/Orion: Genome-wide Perturb-seq Datasets via a Scalable Fix-Cryopreserve Platform for Training Dose-Dependent Biological Foundation Models](https://www.biorxiv.org/content/10.1101/2025.06.11.659105v1.full.pdf)
Modeling perturbations and causality * [Machine learning for perturbational single-cell omics](https://www.sciencedirect.com/science/article/pii/S2405471221002027) * [Elements of Causal Inference: Foundations and Learning Algorithms](https://library.oapen.org/handle/20.500.12657/26040) * [Causal Structure and Representation Learning with Biomedical Applications](https://arxiv.org/abs/2511.04790) * [scPerturb: Information Resource for Harmonized Single-Cell Perturbation Data](https://pubmed.ncbi.nlm.nih.gov/38279009/) * [GeneDisco: A Benchmark for Experimental Design in Drug Discovery](https://arxiv.org/abs/2110.11875) * [Systema: a framework for evaluating genetic perturbation response prediction beyond systematic variation](https://www.nature.com/articles/s41587-025-02777-8) * [Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines](https://www.nature.com/articles/s41592-025-02772-6) * [MORPH Predicts the Single-Cell Outcome of Genetic Perturbations Across Conditions and Data Modalities](https://pmc.ncbi.nlm.nih.gov/articles/PMC12236822/) * [Learning Genetic Perturbation Effects with Variational Causal Inference](https://www.biorxiv.org/content/10.1101/2025.06.05.657988.abstract?__cf_chl_tk=OYKEAztwqhTV5wR88BPPWiPAPeKXBcS5JGYnXetMlkI-1764097427-1.0.1.1-vxNBhPFfL036W6lSNAyhsc2qWS4ZOJGP.EQamGrNcco) * [Squidiff: predicting cellular development and responses to perturbations using a diffusion model](https://www.nature.com/articles/s41592-025-02877-y) * [Predicting cellular responses to perturbation across diverse contexts with State](https://www.biorxiv.org/content/10.1101/2025.06.26.661135v2) * [TxPert: Leveraging Biochemical Relationships for Out-of-Distribution Transcriptomic Perturbation Prediction](https://arxiv.org/abs/2505.14919) * [GEARS: Predicting transcriptional outcomes of novel multi-gene perturbations](https://www.nature.com/articles/s41587-023-01905-6) * [Learning Causal Representations of Single Cells via Sparse Mechanism Shift Modeling](https://arxiv.org/abs/2211.03553) * [Predicting cellular responses to complex perturbations in high-throughput screens](https://pubmed.ncbi.nlm.nih.gov/37154091/) * [PerturbNet predicts single-cell responses to unseen chemical and genetic perturbations](https://pubmed.ncbi.nlm.nih.gov/40640612/) * [Predicting Cellular Responses to Novel Drug Perturbations at a Single-Cell Resolution](https://arxiv.org/abs/2204.13545) * [Active Learning for Optimal Intervention Design in Causal Models](https://www.nature.com/articles/s42256-023-00719-0) * [Control of cell state transitions](https://www.nature.com/articles/s41586-022-05194-y)
{% hint style="info" %} Reading these articles is not necessary to complete the challenge, but we believe these can be a helpful resource. {% endhint %} # Crunch 1 – Predicting the effect of held-out single-gene perturbations Predict the single-cell transcriptomic response to unseen single-gene perturbations. ## Evaluation Phases In Crunch 1, you will have the opportunity to evaluate the predictive performance of your model on a validation dataset. There will be multiple validation checkpoints, with one occurring every Monday at 6:00 p.m. UTC: * Checkpoint 1 - December 22th * Checkpoint n - every Monday * Last checkpoint - February 23th * Last submission - February 28th * Start of the selection period * End of the selection period - March 4th {% hint style="info" %} You can still submit and run your code multiple times onto the platform. At a checkpoint, all of your (non scored) predictions will be scored. \ Predictions will also be scored at the beginning of the selection period. {% endhint %} ## Overview In Crunch 1, we will explore how well we can predict the single-cell transcriptomic response to single-gene perturbations that were not measured and provided in the training dataset. ## Dataset The dataset includes perturbations targeting 157 genes, of which 150 are [transcription factors (TFs)](#user-content-fn-1)[^1]. For each perturbation, we provide single-cell gene expression (RNA-seq) profiles measured at the day 14 of [adipocyte differentiation](https://pubmed.ncbi.nlm.nih.gov/9674695/#:~:text=Committed%20preadipocytes%20undergo%20growth%20arrest,protein%20and%20lipid%2Dmetabolizing%20enzymes.), annotated with gene perturbation identity, quality control (QC) metrics, and cell metadata. The training dataset contains a subset of these perturbations, while a distinct set of single-gene perturbations is held out for validation and test. The layout is as follow: * The dataset is provided in [AnnData format (.h5ad)](https://anndata.readthedocs.io/en/stable/) as `obesity_challenge_1.h5ad`. * Normalized gene expression values are stored in `adata.X`. Raw counts were normalized to a target sum of 100,000 per cell, followed by a $$log\_2(1+x)$$ transformation (standard single-cell RNA-seq normalization; see [lecture 2 of the crash course](/competitions/competitions/broad-obesity/crash-course#lecture-2)). * Raw gene expression counts prior to normalization are stored in `adata.layers['counts']` for reproducibility and alternative preprocessing. * The perturbation target gene information is provided in `adata.obs['gene']`, with values corresponding to either “NC” for control cells or to the target gene name if the cell is perturbed. Control cells receive a perturbation that has no effect on the cell’s RNA-Seq profile. * Cell state/program enrichment information is provided in `.obs`, with columns `pre_adipo`, `adipo`, `lipo`, and `other` indicating whether each cell was enriched for pre-adipocyte, adipocyte, or lipogenic programs. `other` was defined as cells that were not enriched for either pre-adipocyte or adipocyte programs. Program enrichment assignments were based on expert-curated canonical signature genes, and the list of signature genes is provided in `signature_genes.csv`. * The full analysis workflow used to determine program enrichment is provided in the [accompanying notebook (in R)](https://github.com/julielaffy/obesity-broad-ml-competition-2025?tab=readme-ov-file), which can be consulted for additional methodological details. * We provide the cell state proportion for each of the perturbations in a separate file `program_proportion.csv`. * During preprocessing, standard single-cell quality control (QC) was applied to remove low-quality cells and cell doublets based on sequencing library complexity, gene detection rate, and mitochondrial gene content. The dataset was then restricted to cells with a single confident guide assignment to a perturbation, and guides represented by fewer than 10 cells were excluded. Genes detected in fewer than 10 cells were removed, and known signature genes from `signature_genes.csv` were subsequently re-introduced. The `.obs` columns are defined as: * `orig.ident`: The original sample ID. * `nCount_RNA:` The number of UMIs detected per cell. * `nFeature_RNA`: The number of genes detected per cell. * `nCount_guide`: The number of sgRNA UMIs detected per cell. * `nFeature_guide`: The number of sgRNAs detected per cell. * `percent.mt`: The fraction of UMIs per cell that map to mitochondrial transcripts. * `SampleID`: The sample ID. * `Day`: The day of sample collection. * `num_features`: The number of guides per cell (for low MOI data, after qc, only the cells with 1 guide are kept). * `feature_call`: The guide assignment of each cell. * `num_umis`: The number of guide umis per cell. * `gene`: The perturbation target gene (or perturbation identity). * `positive_control`: Whether the perturbation is one of the positive controls. ## Expected Output Participants must submit three outputs: ### File: `prediction.h5ad` An [AnnData](https://anndata.readthedocs.io/en/stable/) file containing predicted gene expression profiles normalized and log-transformed post-perturbation for 2,863 gene perturbations indicated in [`predict_perturbations.txt`](#user-content-fn-2)[^2]. Predictions should be stored in `adata.X` matrix with the corresponding perturbation identity recorded in `adata.obs['gene']`. The set of genes (columns) included in the prediction is defined explicitly by `genes_to_predict` provided at inference time and the columns of `adata.X` must follow this order. Note that the `genes_to_predict` list may change between validation (N=10,237) and test phases, and your model must generate predictions for whichever set of genes is supplied. The maximum number of genes that could be included in `genes_to_predict` is 21,592 corresponding to the total number of genes in the dataset. For each gene perturbation, we ask you to predict the gene expression profiles for 100 cells to quantify the distribution of each perturbation prediction. With N = len(genes\_to\_predict), the final prediction file is therefore required to have dimensions: \[286,300 × N] (cells × genes\_to\_predict). ### File: `predict_program_proportion.csv` A [CSV](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_csv.html) file reporting the predicted proportion of cells with enriched programs for each gene perturbation listed in `predict_perturbations.txt`. The file should contain one row per perturbation with the following columns: * `gene`: should contain the perturbation name, * `pre_adipo`, `adipo`, `lipo`, and `other`: should specify the predicted proportion of cells in each corresponding state for that perturbation, * `lipo_adipo`: should be the ratio of `lipo` to `adipo` (representing the proportion of adipocytes with enriched lipogenic programs). This file should thus have 2,863 rows and [6 columns](#user-content-fn-3)[^3]. An example is available in the `data/` directory. ### File: `Method description.md` We ask you to please write a small document outlining the approaches used to generate both the predictions and the estimated proportions of cells enriched for each program. This should include sufficient details of the computational models employed and the procedures used to derive cell proportions. The document should be organized into three sections, represented as titles in a Markdown file: * **Method Description**: Explain how your method works. *(5-10 sentences)* * **Rationale**: Describe the reasoning behind your model. *(5-10 sentences)* * **Data and Resources Used**: Specify the datasets and any other resources utilized. *(5-10 sentences)* **Notes:** * A human will validate the content at the end of the competition.\ Work deemed unsatisfactory may be disqualified. * This file must be **provided during submission**.\ If content needs to be changed, you **must re-submit with the new version**. * The name must be `Method Description.md`; case does not matter. * Only non-empty and non-comment lines are considered. Below is an example of how to format the file: {% code expandable="true" %} ```markdown # Method Description Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam eget augue quis metus viverra vehicula sit amet lacinia odio. # Rationale Praesent dignissim ipsum vel leo eleifend, eget pulvinar mauris ornare. Duis efficitur lectus posuere iaculis dictum. # Data and Resources Used Donec feugiat eros vel odio gravida venenatis. Nam et sem sit amet nisi vestibulum semper bibendum et libero. ``` {% endcode %} {% hint style="info" %} If there's an obvious issue regarding the format, you'll receive an immediate notification. Notebook users are required to use [embedded files](/competitions/participate/notebook-processor#embed-files). {% endhint %} ## Scoring Each metric will be displayed in a different leaderboard. Each will have a different ranking and opportunity for a prize. The metrics are classed into 2 categories: * Transcriptome-wide metrics that will be computed computed using **a subset of genes** (i.e., the columns of the predicted matrix) for each perturbation. * Metrics include: * **Pearson Delta** between predicted and observed perturbation effects relative to perturbed mean. * **Maximum mean discrepancy (MMD)** between predicted and observed distributions of single-cell profiles. * **Public leaderboard / validation** (updated weekly): Evaluation uses **1,000 hidden genes**. * **Private leaderboard / test phase**: Both the **number** and **identity** of scoring genes will remain unknown. * A Program-level metric that will evaluate whether models capture meaningful biological outcomes, which is: * **L1-distance** between predicted and observed four cell state proportions for each perturbation (i.e. pre-adipogenic, adipogenic, lipogenic, and other) {% hint style="info" %} The evaluation code is available [on GitHub](https://github.com/crunchdao/competitions/blob/master/competitions/broad-obesity-1/scoring/scoring.py). Code for local scoring will be available in the [quickstarter](https://colab.research.google.com/github/crunchdao/quickstarters/blob/master/competitions/broad-obesity-1/quickstarters/perturbed-mean-baseline/perturbed-mean-baseline.ipynb). For more details about how the metrics formulas, please consult the [Full Specifications](/competitions/competitions/broad-obesity/full-specifications). {% endhint %} ## Submit To build a valid submission, your model needs to be coded within the infer function, effectively respecting the crunch code submission interface. {% code title="Python Notebook Cell" %} ```python def train( data_directory_path: str, model_directory_path: str, ) -> None: """ Train a perturbation prediction model. Parameters: data_directory_path: Directory where the data is located. model_directory_path: Directory where your model state should be persisted (usually named `resources/`). Return: None: Returned value is ignored. """ ``` {% endcode %} {% hint style="warning" %} We recommend training locally and submitting weights because the dataset is large and cloud resources are limited. Make sure that the [`Method description.md`](#file-method-description.md) file properly documents your model, so that the Eric and Wendy Schmidt Center team can reference your work in their publications. {% endhint %} {% code title="Python Notebook Cell" %} ```python def infer( data_directory_path: str, prediction_directory_path: str, prediction_h5ad_file_path: str, program_proportion_csv_file_path: str, model_directory_path: str, predict_perturbations: list[str], genes_to_predict: list[str], ): """ Run inference for a set of perturbations. Parameters: data_directory_path: Path to the training AnnData file. prediction_directory_path: Directory where prediction files can be written. prediction_h5ad_file_path: Direct path where to write the `prediction.h5ad` file. program_proportion_csv_file_path: Direct path where to write the `predict_program_proportion.csv` file. model_directory_path: Directory containing your persisted model files. predict_perturbations: The perturbations for which to generate predictions. genes_to_predict: List of gene names (columns) to include in the prediction.h5ad AnnData object. Return: None: Returned value is ignored. Expected files: prediction_directory_path / "prediction.h5ad": anndata.AnnData AnnData matrix of size (n_perturbations * cells_per_perturbation, n_genes_to_predict), containing the predicted gene expression values. prediction_directory_path / "predict_program_proportion.csv": pd.DataFrame DataFrame (index=False) containing estimated cell-type proportions for each perturbation. """ ``` {% endcode %} {% hint style="info" %} An example is available in the [quickstarter](https://colab.research.google.com/github/crunchdao/quickstarters/blob/master/competitions/broad-obesity-1/quickstarters/perturbed-mean-baseline/perturbed-mean-baseline.ipynb). {% endhint %} ## FAQ
I missed a checkpoint, can I participate to the next one? Yes. There are checkpoints every Monday, and missing one will not affect your final ranking. Once your model is ready, submit it!
Why must external resources be published or in the public domain? While releasing the full model is encouraged, it is not strictly required if the weights are sufficient for reproducibility. When constraints limit what can be released, we ask that the methods and training procedures be clearly documented. These cases can be reviewed individually to ensure transparency and fairness.
[^1]: A transcription factor (TF) is a protein that controls the rate of transcription of genetic information from DNA to messenger RNA, by binding to a specific DNA sequence. (source [Wikipedia](https://en.wikipedia.org/wiki/Transcription_factor)) [^2]: File is available in the `data/` directory. [^3]: Use `.to_csv(index=False)` when saving the file. # Crunch 2 – Predicting the effect of held-out double-gene perturbations Predict the single-cell transcriptomic response to double-gene perturbations. ## Evaluation Phases In Crunch 2, you will have the opportunity to evaluate the predictive performance of your model on a validation dataset. There will be multiple validation checkpoints, with one occurring every Monday at 6:00 p.m. UTC: * Checkpoint 1 - March 9th * Checkpoint n - every Monday * Last checkpoint - April 6th * Last submission - April 13th * Start of the selection period * End of the selection period - April 17th {% hint style="info" %} You can still submit and run your code multiple times onto the platform. At a checkpoint, all of your (non scored) predictions will be scored, which also include the "last submission". Predictions will also be scored at the end of the selection period. {% endhint %} ## Overview In this Crunch, we will explore how well we can predict the single-cell transcriptomic response to double-gene perturbations that were not provided in the training dataset. ## Dataset This dataset contains single-gene and pairwise perturbations targeting a curated set of 18 genes. This includes a set of: * 153 heterotypic perturbations (i.e., Gene A+Gene B) * 18 homotypic perturbations (i.e., Gene A+Gene A); * and 18 monogenic perturbations (i.e., Gene A+NC). Each cell receives either 1 guide RNA (resulting in a single genetic perturbation, similar to [Crunch #1](/competitions/competitions/broad-obesity/crunch-1) or two guide RNAs (leading to heterotypic, homotypic, or monogenic perturbations). Importantly, the number of guides received by a cell has an effect on its underlying transcriptomic distribution: * two non-targeting guides (NC+NC) may have a different effect from a single non-targeting guide (NC); * similarly, Gene A+NC may have a different effect from Gene A alone. Although the dataset includes some cells that received three guides, the evaluation in this Crunch focuses exclusively on cells that received a single guide or two guides. For each cell, we provide its single-cell gene expression (RNA-seq) profile measured at day 14 of adipocyte differentiation, annotated with the identity of the perturbed genes, quality control (QC) metrics, and cell metadata. The training dataset contains a subset of these perturbations, while a distinct set of perturbations is held out for validation and test. The layout is as follow: * The dataset is provided in [AnnData format (.h5ad)](https://anndata.readthedocs.io/en/stable/) as `obesity_challenge_2.h5ad`. * Normalized gene expression values are stored in `adata.X`. Raw counts were normalized to a target sum of 100,000 per cell, followed by a $$log\_2(1+x)$$ transformation (standard single-cell RNA-seq normalization; see [lecture 2 of the crash course](/competitions/competitions/broad-obesity/crash-course#lecture-2)). * Raw gene expression counts prior to normalization are stored in `adata.layers['counts']` for reproducibility and alternative preprocessing. * We provide data for cells that received single-guide, double-guide or three-guide perturbation. The perturbation target information for each cell is provided in `adata.obs['gene']`. Control cells, which receive perturbations with minimal transcriptomic effect, are labeled as `"NC"` (single-guide) or `"NC+NC"` (double-guide). * For perturbed cells, single-guide perturbations are indicated simply by the target gene name (e.g., 'Gene A'). * All double-guide perturbations are denoted using a '+' format, which includes heterotypic gene pairs ('Gene A+Gene B'), homotypic pairs ('Gene A+Gene A'), and monogenic double-guide perturbations ('Gene A+NC'). * Similarly, three-guide perturbations extend this format by linking three targets separated by a '+' (e.g., 'Gene A+Gene B+Gene C'). * Cell state/program enrichment information is provided in `.obs`, with columns `pre_adipo`, `adipo`, `lipo`, and `other` indicating whether each cell was enriched for pre-adipocyte, adipocyte, or lipogenic programs. `other` was defined as cells that were not enriched for either pre-adipocyte or adipocyte programs. Program enrichment assignments were based on expert-curated canonical signature genes, and the list of signature genes is provided in `signature_genes.csv`. * We provide the cell state proportion for each of the perturbations in a separate file `program_proportion.csv`. * During preprocessing, standard single-cell quality control (QC) was applied to remove low-quality cells and cell doublets based on sequencing library complexity, gene detection rate, and mitochondrial gene content.
Definitions of the columns in adata.obs * `orig.ident`: The original sample ID. * `nCount_RNA:` The number of UMIs detected per cell. * `nFeature_RNA`: The number of genes detected per cell. * `nCount_guide`: The number of sgRNA UMIs detected per cell. * `nFeature_guide`: The number of sgRNAs detected per cell. * `percent.mt`: The fraction of UMIs per cell that map to mitochondrial transcripts. * `SampleID`: The sample ID. * `Day`: The day of sample collection. * `num_features`: The number of guides per cell (for low MOI data, after qc, only the cells with 1 guide are kept). * `feature_call`: The guide assignment of each cell. * `num_umis`: The number of guide umis per cell. * `gene`: The perturbation target gene (or perturbation identity). * `positive_control`: Whether the perturbation is one of the positive controls.
## Expected Output Participants must submit three outputs: ### File: `prediction.h5ad` An [AnnData](https://anndata.readthedocs.io/en/stable/) file containing predicted gene expression profiles normalized and log-transformed post-perturbation for 62 gene perturbations indicated in [`predict_perturbations_2.txt`](#user-content-fn-1)[^1]. across 36,601 genes listed in [`predict_genes_2.txt`](#user-content-fn-2)[^2]. Predictions should be stored in `adata.X` matrix with the corresponding perturbation identity recorded in `adata.obs['gene']`. The set of genes (columns) included in the prediction is defined explicitly by `genes_to_predict` provided at inference time and the columns of `adata.X` must follow this order. Note that the `predict_genes_2.txt` list may change between validation and test phases, and your model must generate predictions for whichever set of genes is supplied. The maximum number of genes that could be included in `genes_to_predict` is 36,601 corresponding to the total number of genes in the dataset. For each gene perturbation, we ask you to predict the gene expression profiles for 100 cells to quantify the distribution of each perturbation prediction. With `N = len(predict_genes)`, the final prediction file is therefore required to have dimensions: \[6,200 × N] (cells × genes\_to\_predict). ### File: `predict_program_proportion.csv` A [CSV](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_csv.html) file reporting the predicted proportion of cells with enriched programs for each gene perturbation listed in `predict_perturbations.txt`. The file should contain one row per perturbation with the following columns: * `gene`: should contain the perturbation name, * `pre_adipo`, `adipo`, `lipo`, and `other`: should specify the predicted proportion of cells in each corresponding state for that perturbation, * `lipo_adipo`: should be the ratio of `lipo` to `adipo` (representing the proportion of adipocytes with enriched lipogenic programs). This file should thus have 62 rows and [6 columns](#user-content-fn-3)[^3]. An example is available in the `data/` directory. ### File: `Method description.md` We ask you to please write a small document outlining the approaches used to generate both the predictions and the estimated proportions of cells enriched for each program. This should include sufficient details of the computational models employed and the procedures used to derive cell proportions. The document should be organized into three sections, represented as titles in a Markdown file: * **Method Description**: Explain how your method works. *(5-10 sentences)* * **Rationale**: Describe the reasoning behind your model. *(5-10 sentences)* * **Data and Resources Used**: Specify the datasets and any other resources utilized. *(5-10 sentences)* **Notes:** * A human will validate the content at the end of the competition.\ Work deemed unsatisfactory may be disqualified. * This file must be **provided during submission**.\ If content needs to be changed, you **must re-submit with the new version**. * The name must be `Method Description.md`; case does not matter. * Only non-empty and non-comment lines are considered. Below is an example of how to format the file: {% code expandable="true" %} ```markdown # Method Description Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam eget augue quis metus viverra vehicula sit amet lacinia odio. # Rationale Praesent dignissim ipsum vel leo eleifend, eget pulvinar mauris ornare. Duis efficitur lectus posuere iaculis dictum. # Data and Resources Used Donec feugiat eros vel odio gravida venenatis. Nam et sem sit amet nisi vestibulum semper bibendum et libero. ``` {% endcode %} {% hint style="info" %} If there's an obvious issue regarding the format, you'll receive an immediate notification. Notebook users are required to use [embedded files](/competitions/participate/notebook-processor#embed-files). {% endhint %} ## Scoring Each metric will be displayed in a different leaderboard. Each will have a different ranking and opportunity for a prize. The metrics are classed into 2 categories: * Transcriptome-wide metrics that will be computed computed using **a subset of genes** (i.e., the columns of the predicted matrix) for each perturbation. * Metrics include: * **Pearson Delta** between predicted and observed perturbation effects relative to perturbed mean. * **Maximum mean discrepancy (MMD)** between predicted and observed distributions of single-cell profiles. * **Public leaderboard / validation** (updated weekly): Evaluation uses **1,000 hidden genes**. * **Private leaderboard / test phase**: Both the **number** and **identity** of scoring genes will remain unknown. * A Program-level metric that will evaluate whether models capture meaningful biological outcomes, which is: * **L1-distance** between predicted and observed four cell state proportions for each perturbation (i.e. pre-adipogenic, adipogenic, lipogenic, and other) {% hint style="info" %} The evaluation code is available [on GitHub](https://github.com/crunchdao/competitions/blob/master/competitions/broad-obesity-2/scoring/scoring.py). Code for local scoring will be available in the [quickstarter](https://colab.research.google.com/github/crunchdao/quickstarters/blob/master/competitions/broad-obesity-2/quickstarters/perturbed-mean-baseline/perturbed-mean-baseline.ipynb). For more details about how the metrics formulas, please consult the [Full Specifications](/competitions/competitions/broad-obesity/full-specifications). {% endhint %} ## Submit To build a valid submission, your model needs to be coded within the infer function, effectively respecting the crunch code submission interface. {% code title="Python Notebook Cell" expandable="true" %} ```python def train( data_directory_path: str, model_directory_path: str, ) -> None: """ Train a perturbation prediction model. Parameters: data_directory_path: Directory where the data is located. model_directory_path: Directory where your model state should be persisted (usually named `resources/`). Return: None: Returned value is ignored. """ ``` {% endcode %} {% hint style="warning" %} We recommend training locally and submitting weights because the dataset is large and cloud resources are limited. Make sure that the [`Method description.md`](#file-method-description.md) file properly documents your model, so that the Eric and Wendy Schmidt Center team can reference your work in their publications. {% endhint %} {% code title="Python Notebook Cell" expandable="true" %} ```python def infer( data_directory_path: str, prediction_directory_path: str, prediction_h5ad_file_path: str, program_proportion_csv_file_path: str, model_directory_path: str, predict_perturbations: list[str], predict_genes: list[str], ): """ Run inference for a set of perturbations. Parameters: data_directory_path: Path to the training AnnData file. prediction_directory_path: Directory where prediction files can be written. prediction_h5ad_file_path: Direct path where to write the `prediction.h5ad` file. program_proportion_csv_file_path: Direct path where to write the `predict_program_proportion.csv` file. model_directory_path: Directory containing your persisted model files. predict_perturbations: The perturbations for which to generate predictions. predict_genes: List of gene names (columns) to include in the prediction.h5ad AnnData object. Return: None: Returned value is ignored. Expected files: prediction_directory_path / "prediction.h5ad": anndata.AnnData AnnData matrix of size (n_perturbations * cells_per_perturbation, n_genes_to_predict), containing the predicted gene expression values. prediction_directory_path / "predict_program_proportion.csv": pd.DataFrame DataFrame (index=False) containing estimated cell-type proportions for each perturbation. """ ``` {% endcode %} {% hint style="info" %} An example is available in the [quickstarter](https://colab.research.google.com/github/crunchdao/quickstarters/blob/master/competitions/broad-obesity-2/quickstarters/perturbed-mean-baseline/perturbed-mean-baseline.ipynb). {% endhint %} ## FAQ
I missed Crunch 1, can I participate to this one? Yes. All crunches are unique. There is no requirement to have participated in the first one in order to participate in the others.
I participated to Crunch 1, can my model be adapted? Yes. The model can be adapted with minimal modifications because: * Some file names have changed: * The prefix `_2` has been added. * They are now matching the specifications more closely:\ `perturbations_to_score.txt` is now `score_perturbations_2.txt` * Some parameters have been renamed to reflect the file names.\ The old names are still available for convenience. * There are more masks (see the quickstarter): * `single_perturb_mask` * `double_perturb_mask` * `overall_perturb_mask`
I missed a checkpoint, can I participate to the next one? Yes. There are checkpoints every Monday, and missing one will not affect your final ranking. Once your model is ready, submit it!
Why must external resources be published or in the public domain? While releasing the full model is encouraged, it is not strictly required if the weights are sufficient for reproducibility. When constraints limit what can be released, we ask that the methods and training procedures be clearly documented. These cases can be reviewed individually to ensure transparency and fairness.
[^1]: File is available in the `data/` directory. [^2]: File is available in the `data/` directory. [^3]: Use `.to_csv(index=False)` when saving the file. # Crunch 3 – Identifying combinatorial perturbations to drive adipocyte differentiation Nominate combinatorial gene perturbations that are predicted to most strongly increase a target transcriptional program. ## TL;DR; * **The Goal**: Find pairs of genes that, when switched off together, make fat cells burn energy instead of storing it. * **The Task**: Score and rank \~4.47 million candidate pairs by how strongly they trigger fat-burning. Top picks get tested in a real lab experiment after the competition. * **The Scoring**: 12 fat-burning scores plus one overall score per pair. ### What's new vs Crunch 1 and 2 * **Scale**: Crunch 1 tested single genes. Crunch 2 tested \~170 pairs from an 18-gene shortlist. Crunch 3 covers 4.47M pairs across 2,991 genes. * **Output**: Crunch 1 and 2 wanted detailed cell-by-cell predictions. Crunch 3 wants one ranking score per pair. * **Feedback**: Crunch 1 and 2 had a weekly public leaderboard. Crunch 3 **has no scoring** during the competition. The final leaderboard arrives after the lab experiment. ## Evaluation Phases In Crunch 3, you will not have the opportunity to evaluate your model's predictive performance. A final leaderboard will only be published after a lab experiment. **All participants may submit up to two times:** * For the first submission window, **participants from Crunch 2 are especially encouraged** to **adapt their existing models and submit** them for Crunch 3 within the first 3 weeks of the competition. * After that, **all participants, including both Crunch 2 participants and new participants**, will have an **additional 3 weeks to submit or revise** their work before the competition closes. This division will enable the EWSC team to conduct a preliminary selection based on the results of Crunch 2. ### Timeline There will be multiple quota refresh, with one occurring every Tuesday at 4:00 p.m. UTC: * Refresh 1 – May 19th * Refresh 2 – May 26th * Refresh 3 – June 2nd * **Proposal 1 deadline – June 9th** * If no selection is made, the last proposal is used. * There will be a one-day downtime. * Refresh 5 – June 16th * Refresh 6 – June 23th * Refresh 7 – June 30th * **Proposal 2 deadline – July 3rd** * If no selection is made, the last proposal is used. * If the same as Proposal 1, it is ignored. * Peer review begins – July 4th * More information will be provided later. * **Peer review ends – July 10th** ## Overview Participants must predict the effect of two-gene perturbations on thermogenesis-related gene expression in adipocyte cells. The search space covers all possible pairs drawn from 2,991 genes (those with >5 transcripts per million, plus transcription factors and marker genes), yielding \~4.47 million gene pairs (excluding pairs already tested in Crunch 2). Participants can build on work from Crunches 1 and 2, but it's not required. **Newcomers are also welcome** and may use any approach they choose, including modeling, biological knowledge, public resources, or other strategies. ### Objective The objective in Crunch 3 is to predict the mean z-scored enrichment score for each of the 12 thermogenic gene sets per perturbation, and then to derive the final score by averaging the top 3 scores per perturbation to rank perturbations from highest to lowest. Therefore, for each perturbation, participants should predict 13 scores (12 gene-set-level scores and 1 final aggregated score).

Overall Workflow

## Dataset For the training datasets released in Crunches 1 (`TF150`) and 2 (`TF15_MOI2`), we will provide the raw and z-scored signature scores for each adipocyte cell in [`{...}_ThermoScores_cell.csv` files](#user-content-fn-1)[^1]: * Each row is a cell, identified by `cell_id` and its assigned perturbation (under column `gene`). * The raw enrichment score for each of the 12 signatures appears as a [named columns](#user-content-fn-2)[^2], and the corresponding z-score computed across all adipocyte cells is provided in the [`z_`-prefixed columns](#user-content-fn-3)[^3]. The perturbation-level score for each perturbation is provided in [`{...}_ThermoScores_perturbation.csv` files](#user-content-fn-4)[^4]: * Each row is a perturbation. * The signature columns (named according to the signature gene sets, 12 in total) contain the mean z-score across all cells belonging to that perturbation. * The column `agg_top3_z` is the final aggregated score (the mean of the top 3 signature z-scores) used to rank perturbations. The datasets come from two separate experiments and were processed separately. Therefore, the absolute expression values may differ across the two datasets, even though they are z-normalized. When combining them, participants are free to decide how to combine or adjust them if needed. The datasets includes cells with more than two perturbations, but the new experiment mainly focuses on double perturbations. **When computing the threshold to beat using the existing data, we require more than 16 adipocyte cells for a perturbation to receive a valid score.** The signature gene sets used to define the score can be found in `thermogenic_signatures.csv`. The notebook showing how the scoring procedure was derived and computed is [available here](https://julielaffy.github.io/obesity-broad-ml-competition-2025/program_analysis_py.html#Part-4:-Thermogenic-Scoring). ### Crunch 1 & 2 Training Data The Crunch 1 and Crunch 2 training datasets are available via the `TF150.h5ad` and `TF15_MOI2.h5ad` files, respectively. These are the same files provided in previous parts, just renamed for consistency with the thermo score files. Since the files are large, a smaller version of the data is available to make it easier and faster to get and load the data. In the cloud environment, only the "default" one will be available. You can find out more about the content of the training data for [Crunch 1 here](https://docs.crunchdao.com/competitions/competitions/broad-obesity/crunch-1#dataset) and for [Crunch 2 here](https://docs.crunchdao.com/competitions/competitions/broad-obesity/crunch-2#dataset). ## Expected Output Your submission should include the following: ### File: `prediction.parquet` A [PARQUET](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_parquet.html) file containing all candidate gene pairs, with one row per gene pair and at least the following columns: * `GenePairID`: **Gene pair ID**, in the format `GeneA+GeneB` or `GeneA+GeneA`.\ The IDs should match the pairs in `predict_perturbations_3.parquet` file. * `nonucp1.all`, * `REACTOME_AMPK_inhibits_chREBP_R-HSA-163680`, * `REACTOME_Integration_of_energy_metabolism_R-HSA-163685`, * `REACTOME_Mitochondrial_biogenesis_R-HSA-1592230`, * `emont.hAd6`, * `lit.thermogenic`, * `C2.WP_THERMOGENESIS`, * `C5.GOBP_ADAPTIVE_THERMOGENESIS`, * `C5.GOBP_BROWN_FAT_CELL_DIFFERENTIATION`, * `C5.GOBP_NEGATIVE_REGULATION_OF_COLD_INDUCED_THERMOGENESIS`, * `C5.GOBP_POSITIVE_REGULATION_OF_COLD_INDUCED_THERMOGENESIS`, * and `yi.Thermogenesis_Village`: **Predicted scores for each of the 12 thermogenic signatures**, with one column per signature.\ Column names should match the signature names in the `thermogenic_signatures.csv` file. * `FinalAggScore`: **Predicted final aggregated score for the gene pair**, defined as the mean of the top-3 predicted signature scores among the 12 signatures. * `Rank`: **Predicted rank** of the gene pair based on the final aggregated score. This file should thus have 4,474,413 rows and [15 columns](#user-content-fn-5)[^5]. ### File: `Report.md` We ask you to please write a small document that should include sufficient details outlining the approaches used by your algorithm to compute the scores. The document should be organized into three sections, represented as titles in a Markdown file: * **Method description**: Describe the algorithm used to generate the predicted scores. *(5-10 sentences)* * **Design rationale**: Explain the rationale behind the proposed gene panel design. *(5-10 sentences)* * **Data and Resources**: Describe the datasets, prior knowledge, and any additional resources used. *(5-10 sentences)* * A reference list may be included. **Notes:** * A human will validate the content at the end of the competition.\ Work deemed unsatisfactory may be disqualified. * This file must be **provided during submission**.\ If content needs to be changed, you **must re-submit with the new version**. * The name must be `Report.md`; case does not matter. * Only non-empty and non-comment lines are considered. Below is an example of how to format the file: ```markdown # Method Description Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam eget augue quis metus viverra vehicula sit amet lacinia odio. # Design rationale Praesent dignissim ipsum vel leo eleifend, eget pulvinar mauris ornare. Duis efficitur lectus posuere iaculis dictum. # Data and Resources Donec feugiat eros vel odio gravida venenatis. Nam et sem sit amet nisi vestibulum semper bibendum et libero. ``` ## Requirements Your algorithm should include an `infer()` function that returns predictions on the test set. The prediction file should not exceed 1 GB. Use `int32` and `float32` to optimize the size. The execution time of your algorithm should not exceed the platform's time limits: **10 hours per week**. ## Evaluations For this part, there will be no direct feedback that participants can use to judge their ranking against others', as scores will only be known after experimental validation is complete. The leaderboard will only show users (or teams) who submitted an algorithm that can generate a valid prediction. The scoring, and thus the prize, is divided into two parts: ### Evaluation 1 – Peer-review Evaluation 1 will award **10% of the prize pool** and is focus on the submitted algorithms and rationales used to select gene pairs for experimental testing. These submissions will be assessed through a [peer review](#peer-review) process, and winners for this evaluation will be announced at the **end of Summer 2026**. {% hint style="info" %} A proposal will only be considered if its author has also peer-reviewed other proposals. {% endhint %} ### Evaluation 2 – Wet lab experiment Evaluation 2 will award **90% of the prize pool** and will assess algorithmic performance after new laboratory experiments have been completed. Specifically, all submitted algorithms will be evaluated based on how accurately they predict the effects of combinatorial perturbations using the newly generated experimental data. Winners based on this evaluation will be announced in **Fall 2026 after the new laboratory experiments have been performed**. {% hint style="info" %} Prizes will only be awarded to the top 10 teams whose proposed gene pairs achieve experimentally validated scores that exceed the maximum perturbation score observed in the training data released from Crunches 1 and 2. {% endhint %} ## Peer Review The peer review component is a mandatory part of the competition. In order to qualify for prizes, you must review 3-4 submissions from other participants. The peer review process is crutial for selecting the most promising gene panels for [experimental validation](#experimental-validation). Your evaluations help identify submissions with strong justifications and innovative approaches. For each assigned submission, reviewers should: * Assign an overall rating on a 1-3 scale: * `1` – poor proposal / justification * `2` – adequate proposal / justification * `3` – excellent proposal and justification * Provide a short written evaluation of 200-400 words covering: * Rationale of the design. * Novelty of the design. * Compliance with the required format. ## Experimental Validation To validate the most promising gene panels, **we will select a total of 150 unique gene pairs for experimental evaluation**. This selection will occur via two routes: ### Route 1: nominations from Crunch 2 winners A total of \~110 unique gene pairs will be selected from top-performing teams in **Crunch 2 that also participate in Crunch 3**. Selection will proceed as follows: * The 30 top-performing teams from Crunch 2 (top 10 teams for each of 3 evaluation metrics) will be considered. * For each team, the top 4-6 gene pairs from that team’s first Crunch #3 submission will be used. This process yields up to 180 nominations, which will be consolidated to \~110 unique gene pairs after accounting for overlap across evaluation metrics and teams. ### Route 2: nominations from Crunch 3 winners A total of \~40 unique gene pairs will be selected from top-performing Crunch 3 submissions based on the [peer review](#peer-review). Selection will proceed as follows: * The top 10 teams will be identified based on peer-review rankings. * For each selected team, the top 4-6 gene pairs will be taken from the winning proposals. This process yields up to 60 nominations, which will be consolidated to \~40 unique gene pairs after accounting for overlap. Together, these two routes define the full experimental validation set of 150 gene-pair combinations. ## FAQ
I missed the other crunches, can I participate to this one? Yes. All crunches are unique. You don't need to have participated in the first or second one to participate in the last one.
I participated to Crunch 2, can my model be adapted? Yes. In Crunch 2, you had to predict the perturbation of gene pairs. In Crunch 3, you must do the same and then rank them. Updating your one-page report is also very crutial so the EWSC team can understand your reasoning.
Why must external resources be published or in the public domain? While releasing the full model is encouraged, it is not strictly required if the weights are sufficient for reproducibility. When constraints limit what can be released, we ask that the methods and training procedures be clearly documented. These cases can be reviewed individually to ensure transparency and fairness.
[^1]: `TF150_ThermoScores_cell.csv` and `TF15_MOI2_ThermoScores_cell.csv` [^2]: e.g. `C2.WP_THERMOGENESIS` [^3]: e.g. `z_C2.WP_THERMOGENESIS` [^4]: `TF150_ThermoScores_perturbation.csv` and `TF15_MOI2_ThermoScores_perturbation.csv` [^5]: Use `.to_parquet(index=False)` when saving the file. # Full Specifications This specification document comes directly from the Eric and Wendy Schmidt Center team. A careful reading of it may provide more detailed answers than the simplified information available on the overview pages. {% embed url="" %} {% hint style="info" %} If the document does not load, the PDF is [available here](https://crunchdao--competition--production.s3.eu-west-1.amazonaws.com/competitions/broad-obesity-1/documents/broad-obesity.pdf). {% endhint %} # Crash Course The following videos dive into the concepts underlying the Obesity ML Competition and how machine learning can help diabetes research. Each lecture guides participants through data handling, model training, and algorithm testing, emphasizing practical applications for advancing diagnosis, treatment prediction, and patient outcomes in autoimmune diseases. It’s ideal for those participating in the challenge or interested in applying AI solutions in medical research. {% hint style="info" %} [The full playlist is available here.](https://www.youtube.com/playlist?list=PLlMMtlgw6qNi-WSkY-UiJMvO2TTa0T4Ga) {% endhint %} {% hint style="success" %} Even if YouTube is unavailable in your country, [you can still watch all the videos via this link.](https://videos.crunchdao.com/broad-obesity-1/) {% endhint %} ## Overview {% embed url="" %} ## Introduction {% embed url="" %} ## Lecture 1 * [Part A - Biology - What is Adipose Tissue?](https://www.youtube.com/watch?v=eBsCT8BZaiw\&list=PLlMMtlgw6qNi-WSkY-UiJMvO2TTa0T4Ga) * [Part B - Biology - Adipose Tissue and Metabolic Diseases](https://www.youtube.com/watch?v=ytT4ks5GExw\&list=PLlMMtlgw6qNi-WSkY-UiJMvO2TTa0T4Ga) * [Part C - Biology - How is Adipose Tissue Formed?](https://www.youtube.com/watch?v=4ZGuYPANmng\&list=PLlMMtlgw6qNi-WSkY-UiJMvO2TTa0T4Ga) ## Lecture 2 * [Part A - Tech - Gene Expression Programs](https://www.youtube.com/watch?v=WGoWHWMr1hY\&list=PLlMMtlgw6qNi-WSkY-UiJMvO2TTa0T4Ga) * [Part B - Tech - Measuring Gene Expression in Cells](https://www.youtube.com/watch?v=-qOs32IFDvY\&list=PLlMMtlgw6qNi-WSkY-UiJMvO2TTa0T4Ga) * [Part C - Tech - Measuring Perturbation Effects in Cells](https://www.youtube.com/watch?v=dhqQ42Fl2MI\&list=PLlMMtlgw6qNi-WSkY-UiJMvO2TTa0T4Ga) ## Lecture 3 * [Part A - Data - Experiment Overview](https://www.youtube.com/watch?v=Vq7Mtc1qMYM\&list=PLlMMtlgw6qNi-WSkY-UiJMvO2TTa0T4Ga) * [Part B - Data - Data Overview](https://www.youtube.com/watch?v=rcKNTCraeG4\&list=PLlMMtlgw6qNi-WSkY-UiJMvO2TTa0T4Ga) * [Part C - Data - Crunches Overview](https://www.youtube.com/watch?v=jZIr7z_PGJo\&list=PLlMMtlgw6qNi-WSkY-UiJMvO2TTa0T4Ga) # ADIA Lab Structural Break Open Benchmark Challenge New edition of the ADIA Lab Structural Break Challenge with a new dataset. ## Overview The ADIA Lab Structural Break Open Benchmark Challenge is a continuation of the [original ADIA Lab Structural Break Challenge](/competitions/competitions/adia-lab-structural-break-challenge). The benchmark addresses the same scientific problem, uses data with the same structure and characteristics, and preserves the original evaluation philosophy. It extends the original challenge into a long lived, continuously evaluated public benchmark. ## Problem Statement The Open Benchmark is a continuation of the 2025 ADIA Lab Structural Break Challenge, run in response to popular demand for additional time to test new ideas. Each series is comprised of two consecutive segments separated by an explicitly given boundary point: a pre-boundary segment, representing the reference behaviour, and a post-boundary segment, representing the period under scrutiny. Each series contains roughly 1,000 to 5,000 observations in total, and the boundary position is known to the participant. Both segments are given in full and simultaneously: there is no streaming and no hidden data. For each series, the code contributed by the participant must output a single score between `0` and `1`, reflecting the confidence that the data-generating process changed at the boundary - `0` if absolutely confident no break occurred, `1` if absolutely confident a break occurred. The training data combines a large collection (around 10,000 series) of synthetic time series with known break labels, exhibiting a wide variety of break types - including changes in mean, variance, distributional shape, dependence structure, and tail behaviour. The benchmark is drawn from the same data-generating distribution as the original competition, with newly generated, never-disclosed test sets (10,000 public and 10,000 private) so that it remains a fair Out-of-Sample evaluation. To preserve the value of the original leaderboard while still encouraging new submissions, a money prize is offered, the top ten models from the original competition are pinned to the leaderboard as fixed reference baselines, and an originality constraint rejects submissions too highly correlated with those reference models. Submissions are evaluated on the held-out test sets using `ROC AUC` computed across all series, with the final out-of-sample scoring carried out at the end of the benchmark period.

Structural Break Example

## Scientific Continuity The open benchmark preserves: * The original research question * The data generating philosophy * The evaluation metric * The definition of structural breaks The objective is not to reset the problem, but to extend it over time under comparable conditions. ### Changes Introduced Originality Constraint: * An originality metric has been introduced to prevent solution cloning and excessive convergence. * Submissions that are highly correlated with any of the original top models will be rejected. Leaderboard Initialization: * The top ten models from the original competition are visible on the leaderboard as fixed reference benchmarks. Out of Sample Evaluation Schedule: * Models are evaluated on a rolling out of sample basis. * Scoring occurs at the end of each calendar quarter, starting with Q1 2026. \ The first official scoring date is March 31st, 2026. ### Possible Use This benchmark is intended for: * Academic research on structural breaks * Method comparison under regime change * Long horizon robustness evaluation It is designed to remain open and relevant beyond a single competition cycle. ## Timeline * **Opening of the new competition**: December 18, 2025 * **Quarterly Out-of-Sample**: March 31, 2026 * **Quarterly Out-of-Sample**: June 30, 2026 * **Quarterly Out-of-Sample**: September 30, 2026 * **Quarterly Out-of-Sample**: December 31, 2026 {% hint style="info" %} * All quarterly dates are at midnight UTC (23:59 to be precise). * A downtime of one week will be necessary to compute the Out-of-Sample score. {% endhint %} ## Evaluation For each time series in the test set, your task is to predict a score between `0` and `1`, where values towards `0` mean that no structural break occurred at the specified boundary point, and values towards `1` mean that a structural break did occur. The evaluation metric will be the [ROC AUC (Area Under the Receiver Operating Characteristic Curve)](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html), which measures the performance of detection algorithms regardless of their specific calibration. A ROC AUC value around `0.5` means that the algorithm is not able to detect structural breaks better than random chance, while values approaching `1.0` indicate perfect detection. The ROC AUC allows us to compare the output of different detection methods by removing the specific bias of each method towards false positives or false negatives. The competition follows a two-stage evaluation process: 1. **Public Leaderboard**: Each submission is immediately scored against a portion of the test data, and results appear on the public leaderboard; 2. **Private Leaderboard**: At the end of the competition, selected submissions are evaluated on the remaining test data to determine the final rankings. This approach ensures that models are evaluated on their ability to generalize rather than potentially overfitting to the public leaderboard data. ## Code Submission This is a code competition where participants are required to submit their Python code (files or notebooks) directly to the CrunchDAO platform. Your submission should: 1. Process and analyze the data; 2. Output a score between `0` and `1` for each time series `id` in the test set, representing the likelihood of a structural break; 3. Your code must produce deterministic output, or it will be ineligible for any rewards; 4. Only the team leader will be ranked on the leaderboard, and be eligible for a reward. Your submitted code will be executed on the competition platform and automatically scored against a portion of the test set. Shortly after submission, your score will appear on the public leaderboard of the competition. ## Submission Requirements * Your main solution file should follow the template provided by the competition host here[^1]; * Your solution must include `train()` and `infer()` functions. The first one is meant to train your model on the training set, in case your model needs that, otherwise it can be left empty. The second one takes the test data as input and returns predictions; * The execution time of your solution should not exceed the platform's time limits: 15 hours per week. {% hint style="success" %} The `yield`-based approach is now optional.\ If no such use is detected, the entire `X_test` will be provided as a `DataFrame`. {% endhint %} ## Dataset Description The dataset for this competition comprises tens of thousands of synthetic univariate time series, each containing approximately 1,000 to 5,000 values with a designated boundary point. For each training time series, a label (`True` for break, `False` for no break) indicates whether a structural break occurred at this boundary point. The time series in this competition are designed to represent various real-world scenarios where structural breaks may occur, with different levels of difficulty in detection. This includes scenarios similar to those found in financial markets, climate data, industrial sensor readings, and biomedical signals, among others. The challenge is to develop algorithms that can generalize across these scenarios and accurately detect structural breaks in new, unseen data. ### Data Format For the training data, the `X` variable will be provided as a `pandas.DataFrame` with a `MultiIndex` structure. Here's an example of the format: ``` value period id time 0 0 0.001858 0 1 -0.001664 0 2 -0.004386 0 3 0.000699 0 4 -0.002433 0 ... ... ... 10000 1890 -0.005903 1 1891 0.007295 1 1892 0.003527 1 1893 0.007218 1 1894 0.000034 1 ``` The `DataFrame` has the following structure: * A `MultiIndex` with two levels: * `id`: Identifies the unique time series (each time series has a unique ID) * `time`: The timestep within each time series * The columns include: * `value`: The actual time series value at that timestep; * `period`: A binary indicator where `0` represents the period before the boundary point, and `1` represents the period after the boundary point. The structural break occurs at the change in value, but it may take some time (values) to become detectable/apparent. The `y` variable is a boolean `pandas.Series`, with `id` as index, indicating whether a structural break \ occurred at the boundary point for that time series (`True` if there was a break, `False` otherwise). The test data will follow the same format. Your code will need to process these time series and generate predictions of the likelihood of a structural break for each unique time series ID. ### Data Size The training set consists of 10,000 datasets and is the same as last year's. There will also be another 10,000 new datasets for each of the [public](/other/glossary#submission-phase) and [private](/other/glossary#out-of-sample-phase) test sets. The testing data provided for local usage only consists of 100 datasets. Participants must consider that their code will run on **100 times more** datasets, with a maximum limit of 15 hours of computing time. A determinism check will re-run the infer function on 30% of the data (3,000 datasets) to ensure your model is deterministic. The results must be equal, with a tolerance of `1e-8`. ### Comparison against the Top 10 The participants with the highest scores in the [original challenge](https://hub.crunchdao.com/competitions/structural-break/leaderboard) will be used as fixed reference points. These models will only be used as a baseline for benchmarking purposes and will not be eligible for prizes. | Leaderboard Rank | Cruncher(s) | ROC AUC Score\* | | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- | | 1st place |

Humberto Brandão
Mário Augusto Filho
Rafael Alencar
João Pedro Peinado

| 90.14% | | 2nd place | [Farukcan Saglam](https://hub.crunchdao.com/users/farukcan-saglam) | 89.85% | | 3rd place | [Lucas Morin](https://hub.crunchdao.com/users/lcrm) | 89.59% | | 4th place | [Julian Mukaj](https://hub.crunchdao.com/users/valuable-j) | 89.32% | | 5th place |

Guoqin Gu
Mutian Hong

| 89.28% | | 6th place | [Leo Huu Dung Nguyen](https://hub.crunchdao.com/users/leo-nguyen) | 89.17% | | 7th place | [Tuah Jihan](https://hub.crunchdao.com/users/tuah-jihan) | 88.34% | | 8th place | [Abhishek Gupta](https://hub.crunchdao.com/users/abhishek-gupta) | 87.78% | | 9th place | [Arina Streltsova](https://hub.crunchdao.com/users/alright-jho) | 87.72% | | 10th place | [Rakesh Jarupula](https://hub.crunchdao.com/users/stealth-rakesh) | 86.72% | \*Previous edition scores. {% hint style="info" %} A Spearman correlation test will be used to assess similarity with the Top 10 models. Models deemed too similar, with an out of sample correlation above 95 percent, will not be eligible for rewards. {% endhint %} {% hint style="info" %} [You can watch the Talks and Award Ceremony here.](https://www.youtube.com/watch?v=LFWdWgAcOqU) {% endhint %} ## Methodology Suggestions Methods such as change point detection algorithms, tests for equality of distributions, anomaly detection, or supervised learning models can be utilized to recognize patterns associated with structural breaks. Some approaches to consider include: * Statistical tests comparing the distributions before and after the boundary point; * Feature extraction from both parts of the time series for comparative analysis; * Time series modeling to detect deviations from expected patterns; * Deep learning approaches for automated pattern recognition. Careful preprocessing of the time series data is an essential step in developing robust detection models. The ultimate goal is to develop reliable algorithms for detecting structural breaks in time series data across various domains where such changes have significant implications for decision-making and risk management. ## Prizes | Winners’ rank | Prize value per quarter | | ------------- | ----------------------- | | 1st place | $3,000 USD | | 2nd place | $2,000 USD | | 3rd place | $1,000 USD | Scoring occurs at the end of each calendar quarter, starting with Q1 2026. ## FAQ
What is a Structural Break? A structural break in a time series can be defined explicitly as an alteration in the underlying [data-generating process (DGP)](#intervention-on-the-data-generating-process-dgp), or implicitly through illustrative examples of series exhibiting structural breaks versus those that do not. #### Intervention on the Data-Generating Process (DGP) Formally, a structural break occurs at a specific time point when the characteristics of the DGP governing a time series change. For instance, consider a random walk characterized by parameters such as drift `μ` and volatility `σ`. A structural break is said to occur at the moment one or more of these parameters experience a change - for example, volatility `σ` changing from `1.0` to `2.0` at a certain point in time. Structural breaks need not always involve explicit mathematical equations or parameter adjustments. Another scenario might involve constructing a single time series by combining segments with inherently different behaviors or data sources - for instance, measuring daily temperature changes with one thermometer initially, then switching to a different thermometer after a specified breakpoint. In short, a structural break is explicitly characterized by a change in the fundamental nature or parameters of the DGP. This change can be either abrupt or smooth and could manifest as parameter changes, functional form changes, regime transitions, or combinations thereof. If no such change occurs, the series does not contain a structural break. #### Implicit Definition via Examples An alternative approach involves implicitly defining a structural break through examples. Under this approach, a substantial and diverse collection of time series, some exhibiting structural breaks and others remaining stable, can implicitly define the concept. Such example-based definitions are particularly useful in machine learning and statistical inference contexts, where explicit parameter-level descriptions might not be directly available or practical. This is especially relevant when working with real-world data, where the underlying data generation process is typically unknown. Importantly, failing to account for structural breaks in time series analysis can lead to misleading results, including inaccurate forecasts and invalid statistical inferences.
Can I reuse my old code from the original competition? **Yes**, you can reuse your old code, but the **originality constraint** will be applied. This means that if your model or approach closely resembles one of the original top 10 models from the original competition, it may not be accepted for new submissions. The goal is to foster innovation and prevent exact replications of previous solutions.
How is originality measured? Originality is measured using a spearman correlation that compares your model's predictions with those of the original top models. If your submission is highly correlated with any of the original models, it will be flagged and will not be accepted. This ensures that the solutions in the benchmark remain innovative and not overly similar to previous submissions.
How many datasets will my infer() function be called with? A total of 16,000. The details are as follows: * 3,000 for the top 10 correlation check, * 10,000 for the regular dataset, * and 3,000 for the determinism check. Everything must fit within the 15-hour quota.
[^1]: TODO Use correct link # DataCrunch Competition This weekly prediction contrast ranks 3000 most liquid US equities for DataCrunch's Hedge Fund. ## Overview DataCrunch uses the quantitative research of the CrunchDAO to manage its systematic market-neutral portfolio. DataCrunch built a dataset covering thousands of publicly traded U.S companies. The long-term strategic goal of the fund is capital appreciation by capturing idiosyncratic return at low volatility. In order to achieve this goal, DataCrunch needs the community to assess the relative performance of all assets in a subset of the [Russell 3000](https://www.investopedia.com/terms/r/russell_3000.asp) universe. In other words, DataCrunch is expecting your model to rank the constituent of its investment universe. ## Prize Reward are split in targets as follow. Each target represent an investment horizon and can be predicted using the DataCrunch dataset. Reward are distributed every month based on crunchers performance: * 60,000 $USDC yearly on target\_b + $10k bonus for cumulative alpha target. * 20,000 $USDC yearly on target\_g * 20,000 $USDC yearly on target\_r * 10,000 $USDC yearly on target\_w ## Weekly Crunches Every week, two phases: * The `Submission Phase:` every Friday at 8PM UTC to Tuesday 12PM UTC, the system will release an additional `moon`. Competitors will be able to submit their code or model * The `Out-Of-Sample Phase:` the models will be run on the Out-Of-Sample data (the live data). The score of each target are published as they are resolved against live market data as reward. ## Data Each row of the dataset describes a stock at a certain date. The dataset is composed of three files, `X_train y_train and X_test`. ### X\_train * `moon`: A sequentially increasing integer representing a date. Time between subsequent dates is constant, denoting a weekly fixed frequency at which the data is sampled. * `id`: A unique identifier representing a stock at a given Moon. Note that the same asset has a different `id` in different Moons. * `Feature_Industry`: the industry to which a `stock` belongs at a given `moon.` * (`gordon_Feature_1`, …, `dolly_Feature_30`): Anonymised `features` that describe the state of assets on a given date. They are grouped into several families, or ways of assessing the relative performance of each stock on a given month. Note: All features have the string "Feature" in their name, prefixed by a code name for the feature family. ### y\_train * `moon`: Same as in `X_train`. * `id`: Same as in `X_train`. * (`target_w`, …, `target_b`): the targets that may help you build your models. Target\_w, r, g, b refer to 7, 28, 63, 91 days compounding of returns. ### X\_test - y\_test `X_test` and `y_test` has the same structure as `X_train` and `y_train` but comprises only 13 moons. These files are used to simulate the submission process locally via `crunch.test()` (within the code), or `crunch test` (via the cli). The aim is to help participants debug their code and have successful submissions. A successful local test usually means no errors during execution on the submission platform. The data of these files is composed of the 13 moons on which the longest target (target\_b) is not resolved. The missing data for each target were replaced by -1 values. *Note*: the features are split in two groups. The legacy features and the v2 features which are suffixed by "\_v2" ## The Performance Metric The infer function from your code will return your predictions. A Spearman rank correlation will be computed against the live targets. ### Reward Scheme All reward are computed on the leaderboards.
The **Hist**orical **Rewards** are the sum of every payout you have received from the DataCrunch competition. The **Proj**ected **Rewards** are the current estimated rewards yet to be distributed. ## Payouts calculation Payouts are computed based on your the rank of your prediction for each target. The higher the Spearman Rank between your prediction and market realisation, the higher your rank on the leaderboard. The payouts are distributed according to an exponential function of your position on the leaderboards, as shown in the graph below, the top 20 crunchers earn approximately 30% of the total rewards.

Cumulative distribution of rewards (2023-03-03 leaderboard)

## Computing Resources Competitors will be allocated a specified quantity of computing resources within the cloud environment for the execution of their code. During the phase, they are entitled to **10 hours** of GPU or CPU compute time per week, and for the `OOS` phase, this allocation increased 10% in case of slower deployement by the system. During the `SUBMISSION` phase, you are entitled to 10 hours of GPU or CPU computing time per week, and during the OOS phase, this allocation is increased by 10%. ## Quickstarter Notebook A Quickstarter notebook is available below so you can get familiar with what is expected from you. {% embed url="" %} ## Legacy Endpoints {% hint style="warning" %} The old data format is still available on the legacy endpoint, but will be removed at some point in the future. We encourage people who still rely on this data to migrate to the new submission format. {% endhint %}
NameParquetCSV (deprecated)
X_train/data/X_train.parquet/data/X_train.csv
y_train/data/y_train.parquet/data/y_train.csv
X_test/data/X_test.parquet/data/X_test.csv
example_submission/data/example_submission.parquet/data/example_submission.csv
# ADIA Lab Structural Break Challenge Detect structural changes in time series data for accurate insights. ## Overview Detecting structural changes in time series data is a critical task across various scientific and engineering domains. In this competition, you are given univariate time series data which, at an explicitly given point, may have experienced a structural break, meaning that the process governing the data generation may have changed from that point on. Given the entire time series and the indication of that time point, your task is to determine whether a structural break has occurred or not.

Structural Break Example

## Problem Statement The task of this competition is to determine whether a permanent structural break has occurred in a univariate time series at a specified point in time. Each series is comprised of two consecutive segments separated by an explicitly given boundary point: a pre-boundary segment, representing the reference behaviour, and a post-boundary segment, representing the period under scrutiny. Each series contains roughly 1,000 to 5,000 observations in total, and the boundary position is known to the participant. Both segments are given in full and simultaneously: there is no streaming and no hidden data. For each series, the code contributed by the participant must output a single score between `0` and `1`, reflecting the confidence that the data-generating process changed at the boundary - `0` if absolutely confident no break occurred, `1` if absolutely confident a break occurred. The training data combines a large collection (around 10,000 series) of synthetic time series with known break labels, exhibiting a wide variety of break types - including changes in mean, variance, distributional shape, dependence structure, and tail behaviour. Submissions are evaluated on an independent test set of comparable size using `ROC AUC` computed across all series: a single score per series is compared with the corresponding ground-truth label, and the area under the ROC curve is the final score. ### Competition Timeline * Start Date: May 14th, 2025 at 10:00 AM CET * End Date: September 30th, 2025 at 6:00 PM CET * Final Evaluation: End of October, 2025 * Winners Announcement: [During the ADIA Lab 2025 Symposium](https://www.adialab.ae/upcoming-events/adia-lab-2025-symposium) (27–29 October) ## Evaluation For each time series in the test set, your task is to predict a score between `0` and `1`, where values towards `0` mean that no structural break occurred at the specified boundary point, and values towards `1` mean that a structural break did occur. The evaluation metric will be the [ROC AUC (Area Under the Receiver Operating Characteristic Curve)](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html), which measures the performance of detection algorithms regardless of their specific calibration. A ROC AUC value around `0.5` means that the algorithm is not able to detect structural breaks better than random chance, while values approaching `1.0` indicate perfect detection. The ROC AUC allows us to compare the output of different detection methods by removing the specific bias of each method towards false positives or false negatives. The competition follows a two-stage evaluation process: 1. **Public Leaderboard**: Each submission is immediately scored against a portion of the test data, and results appear on the public leaderboard; 2. **Private Leaderboard**: At the end of the competition, selected submissions are evaluated on the remaining test data to determine the final rankings. This approach ensures that models are evaluated on their ability to generalize rather than potentially overfitting to the public leaderboard data. ## Code Submission This is a code competition where participants are required to submit their Python code (files or notebooks) directly to the CrunchDAO platform. Your submission should: 1. Process and analyze the data; 2. Output a score between `0` and `1` for each time series `id` in the test set, representing the likelihood of a structural break; 3. Your code must produce deterministic output, or it will be ineligible for any rewards; 4. Only the team leader will be ranked on the leaderboard, and be eligible for a reward. Your submitted code will be executed on the competition platform and automatically scored against a portion of the test set. Shortly after submission, your score will appear on the public leaderboard of the competition. ### Submission Requirements * Your main solution file should follow the template provided by the competition host [here](https://colab.research.google.com/github/crunchdao/quickstarters/blob/master/competitions/structural-break/quickstarters/baseline/baseline.ipynb); * Your solution must include `train()` and `infer()` functions. The first one is meant to train your model on the training set, in case your model needs that, otherwise it can be left empty. The second one takes the test data as input and returns predictions; * The execution time of your solution should not exceed the platform's time limits: 15 hours per week. ## Dataset Description The dataset for this competition comprises tens of thousands of synthetic univariate time series, each containing approximately 1,000 to 5,000 values with a designated boundary point. For each training time series, a label (`True` for break, `False` for no break) indicates whether a structural break occurred at this boundary point. The time series in this competition are designed to represent various real-world scenarios where structural breaks may occur, with different levels of difficulty in detection. This includes scenarios similar to those found in financial markets, climate data, industrial sensor readings, and biomedical signals, among others. The challenge is to develop algorithms that can generalize across these scenarios and accurately detect structural breaks in new, unseen data. ### Data Format For the training data, the `X` variable will be provided as a `pandas.DataFrame` with a `MultiIndex` structure. Here's an example of the format: ``` value period id time 0 0 0.001858 0 1 -0.001664 0 2 -0.004386 0 3 0.000699 0 4 -0.002433 0 ... ... ... 10000 1890 -0.005903 1 1891 0.007295 1 1892 0.003527 1 1893 0.007218 1 1894 0.000034 1 ``` The `DataFrame` has the following structure: * A `MultiIndex` with two levels: * `id`: Identifies the unique time series (each time series has a unique ID) * `time`: The timestep within each time series * The columns include: * `value`: The actual time series value at that timestep; * `period`: A binary indicator where `0` represents the period before the boundary point, and `1` represents the period after the boundary point. The structural break occurs at the change in value, but it may take some time (values) to become detectable/apparent. The `y` variable is a boolean `pandas.Series`, with `id` as index, indicating whether a structural break \ occurred at the boundary point for that time series (`True` if there was a break, `False` otherwise). The test data will follow the same format. Your code will need to process these time series and generate predictions of the likelihood of a structural break for each unique time series ID. ### Data Size The training set consists of 10,000 datasets. There will also be another 10,000 new datasets for both the [public](/other/glossary#submission-phase) and [private](/other/glossary#out-of-sample-phase) test sets. The testing data provided for local usage only consists of 100 datasets. Participants must consider that their code will run on **100 times more** datasets, with a maximum limit of 15 hours of computing time. Inference must be performed sequentially. It is not possible to read the entire `X_test` dataset and predict everything at once. Therefore, the code must be optimized to consume one dataset at a time within the limited computing time. ## Methodology Suggestions Methods such as change point detection algorithms, tests for equality of distributions, anomaly detection, or supervised learning models can be utilized to recognize patterns associated with structural breaks. Some approaches to consider include: * Statistical tests comparing the distributions before and after the boundary point; * Feature extraction from both parts of the time series for comparative analysis; * Time series modeling to detect deviations from expected patterns; * Deep learning approaches for automated pattern recognition. Careful preprocessing of the time series data is an essential step in developing robust detection models. The ultimate goal is to develop reliable algorithms for detecting structural breaks in time series data across various domains where such changes have significant implications for decision-making and risk management. ## Definitions ### What is a Structural Break? A structural break in a time series can be defined explicitly as an alteration in the underlying [data-generating process (DGP)](#intervention-on-the-data-generating-process-dgp), or implicitly through illustrative examples of series exhibiting structural breaks versus those that do not. #### Intervention on the Data-Generating Process (DGP) Formally, a structural break occurs at a specific time point when the characteristics of the DGP governing a time series change. For instance, consider a random walk characterized by parameters such as drift `μ` and volatility `σ`. A structural break is said to occur at the moment one or more of these parameters experience a change - for example, volatility `σ` changing from `1.0` to `2.0` at a certain point in time. Structural breaks need not always involve explicit mathematical equations or parameter adjustments. Another scenario might involve constructing a single time series by combining segments with inherently different behaviors or data sources - for instance, measuring daily temperature changes with one thermometer initially, then switching to a different thermometer after a specified breakpoint. In short, a structural break is explicitly characterized by a change in the fundamental nature or parameters of the DGP. This change can be either abrupt or smooth and could manifest as parameter changes, functional form changes, regime transitions, or combinations thereof. If no such change occurs, the series does not contain a structural break. #### Implicit Definition via Examples An alternative approach involves implicitly defining a structural break through examples. Under this approach, a substantial and diverse collection of time series, some exhibiting structural breaks and others remaining stable, can implicitly define the concept. Such example-based definitions are particularly useful in machine learning and statistical inference contexts, where explicit parameter-level descriptions might not be directly available or practical. This is especially relevant when working with real-world data, where the underlying data generation process is typically unknown. Importantly, failing to account for structural breaks in time series analysis can lead to misleading results, including inaccurate forecasts and invalid statistical inferences. ## Prizes | Winners’ rank | Prize value | | ------------- | ----------- | | 1st place | $40,000 USD | | 2nd place | $20,000 USD | | 3rd place | $10,000 USD | | 4th place | $5,000 USD | | 5th place | $5,000 USD | | 6th place | $5,000 USD | | 7th place | $5,000 USD | | 8th place | $3,500 USD | | 9th place | $3,500 USD | | 10th place | $3,000 USD | # Broad Institute Autoimmune Disease Crunch Foundation, The Eric and Wendy Schmidt Center, and The Klarman Cell Observatory invite you to join the Autoimmune Disease ML Challenge to design algorithms to help millions of people. ## Challenge Overview ### Introduction Autoimmune diseases arise when the immune system mistakenly targets healthy cells. Affecting 50M people in the U.S., with rising global cases, Inflammatory Bowel Disease (IBD) is one of the most prevalent forms. IBD occurs when the barrier between our gut and the microbes living there breaks down, leading to the activation of the immune system and persistent inflammation. This cycle of flares and remission increases the risk of colorectal cancer (up to two-fold). Although modern treatments have improved survival, IBD remains challenging to diagnose and treat due to its complex pathogenic pathways and multifactorial nature. Pathologists rely on **gut tissue images** to diagnose and treat IBD, guiding decisions on the most suitable drug treatments and predicting cancer risk. These tissue images, combined with recent advances in **genomics**, offer a valuable dataset for machine learning models to revolutionize IBD diagnosis and treatment. {% hint style="info" %} [Read the full competition specifications here.](/competitions/competitions/broad-institute-autoimmune-disease/full-specifications) {% endhint %} This challenge is meant for everyone! We have created a three-lecture crash course that provides background on the biology, technology, and data in the three crunches. You do not need a background in biology or medicine to participate. {% hint style="info" %} [Find the lecture crash course here.](https://www.youtube.com/watch?v=9OTvuvr81R0\&list=PLlMMtlgw6qNhqMxU8C2V_zsuhlqIgpW6y) {% endhint %} ## Phases The challenge is broken down into three Crunches, ordered by increasing complexity. ### **Crunch 1 –** Oct 28 to Feb 9 – **P**redict gene expression in spatial transcriptomics data from matched pathology images Crunchers will build a model to predict the expression of 460 genes in held-out patches of colon tissue using **H\&E pathology images** and **Xenium spatial transcriptomics** training data. Hematoxylin and Eosin (H\&E) images provide insight into cell organization, while Xenium data add information on gene expression and cellular pathways of disease. ### **Crunch 2 – Nov 18 to Mar 21 – Predicting Unseen Genes** In this phase, participants will predict the expression of all protein-coding genes, including those that were not measured in the spatial training data, using **single-cell RNA-seq** data as support. This Crunch focuses on leveraging cell transcriptional profiles to enhance the predictive model’s ability to infer the expression of unknown genes in spatial contexts. ### **Crunch 3 – Dec 9 to Apr 30 (submission deadline) / May 15 (peer review deadline) – Identifying Gene Markers for Pre-cancerous Regions** Participants will rank genes by their ability to distinguish between dysplasia (pre-cancerous) regions and noncancerous tissue in IBD patients, increasing our ability to detect cancer early. The final gene panel will be chosen based on participant performance in Crunch 2 and on peer review of participants' methods taking place after the submission deadline. The gene panel will be experimentally validated in a new colon tissue with dysplasia, and all participants' ranked gene lists will be scored. {% hint style="info" %} The best-performing models will be experimentally tested to validate their ability to predict cancer risk, which could lead to early detection and improved treatment options. The best models will be publish in an official publication from Broad Institute. {% endhint %}
### **Participant Output Requirements** For each Crunch, participants must submit predictions in CSV format. Each submission must adhere to the provided **log1p-normalization** standards. Outputs will be evaluated using: * **Mean Squared Error** (Crunch 1). To avoid overfitting Crunch will score all submitted models on a private dataset during two Checkpoints. * **Spearman’s Correlation** (Crunch 2) * **Accuracy** and **Diversity Metrics** (Crunch 3) ### **Evaluation Criteria** Performance will be evaluated through: * **Accuracy in gene expression prediction** (Crunch 1 & 2) * **Gene panel design** for distinguishing between noncancerous and dysplasia regions (Crunch 3) * **Diversity of selected gene programs** in Crunch 3, with extra emphasis on identifying unique biological pathways * **Peer review** of methods to select dysplasia gene panel in Crunch 3 ## Prizes ### Crunch 1 | Winner’s rank | Prize value (in USD) | | ------------- | -------------------- | | 1st Place | 3,500 | | 2nd Place | 2,500 | | 3rd Place | 1,750 | | 4th Place | 900 | | 5th Place | 750 | | 6th Place | 700 | | 7th Place | 600 | | 8th Place | 500 | | 9th Place | 450 | | 10th Place | 350 | | Total | 12,000 | ### Crunch 2 | Winner’s rank | Prize value (in USD) | | ------------- | -------------------- | | 1st Place | 3,500 | | 2nd Place | 2,500 | | 3rd Place | 1,750 | | 4th Place | 900 | | 5th Place | 750 | | 6th Place | 700 | | 7th Place | 600 | | 8th Place | 500 | | 9th Place | 450 | | 10th Place | 350 | | Total | 12,000 | ### Crunch 3 | Winner’s rank | Prize value (in USD) | | ------------- | -------------------- | | 1st Place | 7,000 | | 2nd Place | 6,500 | | 3rd Place | 5,000 | | 4th Place | 3,000 | | 5th Place | 1,000 | | 6th Place | 900 | | 7th Place | 800 | | 8th Place | 700 | | 9th Place | 600 | | 10th Place | 500 | | Total | 26,000 | ## External Resources Crunchers are encouraged to use publicly available external resources, including gene expression datasets and pre-trained models, as long as they are properly credited. A list of potential resources and references is provided in the [full challenge specifications.](/competitions/competitions/broad-institute-autoimmune-disease/full-specifications) **Foundry Institute** offers a computing environment with $10 USD equivalent to around 10h of GPU time. {% hint style="info" %} [Find ML Foundry documentation here.](https://docs.mlfoundry.com/foundry-documentation) {% endhint %} # Crunch 1 – Oct 28 to Feb 9 – Predict gene expression Predict gene expression in spatial transcriptomics data from matched pathology images ## Evaluation Phases In Crunch 1, you will have the opportunity to evaluate your model’s predictive performance on a validation dataset, before submission of your test dataset predictions. There will be multiple validation checkpoints: * **Checkpoint 1** - November 30th (Eastern Time 17:59) * **Checkpoint 2** - December 16th (Eastern Time 17:59) * **Checkpoint 3** - December 30th (Eastern Time 17:59) * **Checkpoint 4** - January 13th (Eastern Time 17:59) * ~~**Checkpoint 5** - January 27th (Eastern Time 17:59)~~ * **Continuous Public Leaderboard** - January 20th * **Last submission -** February 9th (Eastern Time 17:59) ## Overview In **Crunch 1**, you will train an algorithm to predict **spatial transcriptomics data** (gene expression in each cell) from matched **H\&E images**. In other words predict the gene expression (Y) in cells from specific tissue patches based on the **H\&E images** (X) and surrounding **spatial transcriptomics** data. * **X (Input)**: * **`HE_original`**: The **original H\&E image** in its native pixel coordinates. Alignment from H\&E native coordinate system to Xenium coordinate system has been handled from our end. If you prefer to handle alignment yourself, you can check **HE\_original** and **DAPI** (provided in crunch1\_max), but it may require additional processing. * **`HE_nuc_original`**: The **nucleus segmentation mask of H\&E image**, in H\&E native coordinate system. The cell\_id in this segmentation mask matches with the nuclei by gene matrix stored in **anucleus**.
* **Y**: * **`anucleus`**: This file contains the **aggregated gene expression data** for each nucleus. It is log1p-normalized and stores the gene expression profiles for 460 genes per nucleus. This is the primary target (**Y**) for your model.

Anucleus – gene expression for each nucleus

## Linking the H\&E image to spatial transcriptomics Steps to align X and Y: * **Step 1: Identify nuclei in the H\&E image** * Use the **nucleus segmentation masks**: * **H\&E nucleus segmentation (`HE_nuc_original`)**: This mask identifies the location of nuclei in the **original H\&E image** **(**i.e. **HE\_original)**. * **Step 2: Link gene expression to H\&E images** * For each nucleus in the **H\&E image**, use the **`anucleus`** file to get the corresponding gene expression profile (Y) for that nucleus. * The **`anucleus`** file provides the gene expression data, where each row corresponds to a nucleus (cell) and each column corresponds to a gene. * The nuclei IDs from the segmentation masks (e.g., from **`HE_nuc_original`**) will match the IDs used in the **`anucleus`** file.

Matched crop H&E image and its corresponding Gene Expression Heatmap

{% hint style="info" %} If you open the image HE\_nuc\_original, e.g. through `mask=sdata['HE_nuc_original'][0].to_numpy()`. You can directly find the location of that cell, with cell\_id, through `mask==cell_id`. {% endhint %} The datasets are store in a **SpatialData object.** Learn more about this format [here](https://spatialdata.scverse.org/en/stable/generated/spatialdata.SpatialData.html). {% code fullWidth="true" %} ```javascript // SpatialData object structure Images // 'DAPI': DAPI image (validation and test tissue patches are removed) 'DAPI_nuc': DAPI nucleus segmentation 'HE_nuc_original': H&E nucleus segmentation on original image 'HE_nuc_registered': H&E nucleus segmentation on registered image (registered to DAPI image) 'HE_original': H&E original image 'HE_registered': H&E registered image 'group': Defining train(0)/validation(1)/test(2), No_transcript-train(4) tissue patches 'group_HEspace': Defining train(0)/validation(1)/test(2), No_transcript-train(4) tissue patches on the H&E image Points 'transcripts': DataFrame for each transcript (containing x,y,tissue patch,z_location, feature_name,transcript_id,qv,cell_id columns) Tables 'anucleus': AnnData contains .X, .layers['counts'], .obsm['spatial'] 'cell_id-group': AnnData only contains .obs DataFrame for mapping of cell_id to region. with coordinate systems: 'global', with elements: DAPI (Images), 'DAPI_nuc' (Images), 'HE_nuc_original' (Images), 'HE_nuc_registered' (Images), 'HE_original' (Images), 'HE_registered' (Images), 'group' (Images), 6 'group_HEspace' (Images), 'transcripts' (Points) 'scale_um_to_px', with elements: transcripts (Points) ``` {% endcode %} In the minimum version of the data provided for crunch1 (in crunch1\_min.tar), only **HE\_original**, **HE\_nuc\_original**, **anucleus** and **cell\_id-group** are provided. ## Expected Output The output consists of four columns: * **cell\_id**: contains the held-out nuclei (both validation and test tissue regions). * **gene**: the gene among the 460 genes to be predicted. * **prediction**: the gene expression value, rounded to two decimal places. * **sample**: the tissue sample among the 8 samples to process. {% hint style="success" %} Make sure your predictions are `log1p-normalized` with a scale factor of 100 as in `anucleus.X` {% endhint %}
### Scoring The scoring metric is a cell-wise Spearman correlation. A Mean Squared Error metric is also computed, the value must be below 0.2. Since the baseline is 0.1, a model with an MSE that is too high is not considered viable and will not be eligible for rewarded. {% hint style="info" %} [The evaluation code is available on GitHub.](https://github.com/crunchdao/competitions/blob/master/competitions/broad-1/scoring/scoring.py) {% endhint %} ## Submit To build a valid submission, your model need to be coded within the infer function, effectively respecting the crunch code submission interface. {% hint style="info" %} [See how to submit through the quickstarter.](https://github.com/crunchdao/quickstarters/blob/master/competitions/broad-1/quickstarters/random-submission/random-submission.ipynb) {% endhint %} {% hint style="info" %} [Learn about crunch code interface.](/competitions/participate/code-interface) {% endhint %} ## Data Variants Due to the large size of the datasets, Crunch provides both a small (aka. default) and a large version. Depending on your local setup and goals within Crunch, you can choose either one. By default, the small dataset is downloaded. To access the larger dataset, specify it explicitly with a different CLI command: ```bash # setup with the large data crunch setup --size large broad-1 my-model ... # setup with the small data crunch setup broad-1 my-model ... ``` The larger version contain the Xenium transcriptomic data. It allow you to know both the gene expression and the coordinate (x, y, z) of the position of the gene in the Cells. More details about the gene transcriptomic data in the full documentation. {% hint style="warning" %} The large variant is for local use only. The Cloud Environment will always use the default dataset. {% endhint %} # Crunch 2 – Nov 18 to Mar 21 – Predicting Unseen Genes ## Overview In **Crunch 2**, your task is to predict the expression levels of genes that were **not measured** in a spatial transcriptomics dataset. You will use both spatial data and single-cell RNA sequencing (scRNA-Seq) data from similar colon tissue samples to make these predictions. ### **X (Inputs Data)** * **Spatial Data**: The `.zarr` data provided in [**Crunch 1**](/competitions/competitions/broad-institute-autoimmune-disease/crunch-1#linking-the-h-and-e-image-to-spatial-transcriptomics). * **scRNA-Seq Data**: The `Crunch2_scRNAseq.h5ad` file contains gene expression data for **18,615 protein-coding genes**, including the **460 genes** in the Spatial Data object. ### **Y (Targets)** * **Gene Expression Predictions**: The expression levels of **2,000 genes**.

Example of 2,000 genes expressions with random values.

## Evaluation Phases In Crunch 2, you will have the opportunity to evaluate your model’s predictive performance on a validation dataset, before submission of your test dataset predictions. There will be checkpoints every: * **Friday** — to get your scores before the weekend * **Monday** — to see how your weekend work stacks up The final submission must be submitted by **March 21th (Eastern Time 17:59)**. ## The single-cell transcriptomic datasets Colon tissue samples similar to those profiled by Xenium spatial transcriptomics. These datasets contain single-cell gene expression data for **18,615 protein-coding genes**, including the 460 genes in the Spatial Data. ### **Datasets Included** We provide datasets (atlases) from multiple studies to represent all the cell types that are found in the colon tissue. * [**UC Dataset**](https://pubmed.ncbi.nlm.nih.gov/31348891/): An atlas of ulcerative colitis patients, including inflamed, non-inflamed, and healthy colon tissue. * [**ENS Dataset**](https://pubmed.ncbi.nlm.nih.gov/32888429/): An atlas of the enteric nervous system, including glial cells and neurons innervating the colon. * [**Muscle Dataset**](https://pubmed.ncbi.nlm.nih.gov/37206377/): An atlas of the colon muscle layer. ### **Data Format** Provided as an [AnnData object](https://anndata.readthedocs.io/en/latest/) stored in an h5ad file: `Crunch2_scRNAseq.h5ad`. #### **Cell Metadata `scRNA-Seq.obs`** * **Cell Type**: `scRNA-Seq.obs["annotation"]` * **Study**: `scRNA-Seq.obs["study"]` * **Individual**: `scRNA-Seq.obs["individual"]` * **Disease Status**: `scRNA-Seq.obs["status"]`
scRNA-Seq Observations (scRNAseq.obs)
#### **Expression Data`scRNA-Seq.X`** — `log1p-normalized` counts Original raw counts per cell are divided by the sum of counts per cell, multiplied by **10,000**, and then `log1p`-transformed.
Compressed Sparse Row (CSR) Matrix of scRNAseq.X as DataFrame
This representation displays the `scRNAseq.X` matrix in **DataFrame** format to clarify the structure of the CSR matrix. The columns in the DataFrame are as follows: * Row: the row index corresponding to the observation index, accessible via `scRNAseq.obs` * Column: the column index corresponding to the gene index, accessible via `scRNAseq.var` * Value: normalized, log-transformed gene expression counts #### **Raw Gene Counts** — Available in `scRNA-Seq.layers["counts"]` ## Expected Output The output must be provided as a DataFrame with the following structure: **Index** * Contains the `cell_id` values corresponding to the validation and test groups expected in the SpatialData (`.zarr` file provided by the `infer` function). **Columns** * Contains **2,000 genes** randomly selected from the **18,615 protein-coding genes** in the `scRNA-Seq` data, including the **20 genes** already measured by Xenium spatial transcriptomics but excluded from the Spatial Data object. * You can retrieve this list from the `Crunch2_gene_list.csv` file included in the competition dataset. **Values** * Gene expression predictions for each cell and gene. * Predictions must be **log1p-normalized** and rounded to **2 decimal points**. {% hint style="info" %} Refer to the [`random-submission.ipynb`](https://github.com/crunchdao/competitions/blob/master/competitions/broad-2/quickstarters/random-submission/random-submission.ipynb) notebook for an example of how to format your submission. {% endhint %} ## Evaluation Your predictions are evaluated on the **20 held-out genes** using **Spearman’s rank correlation** for cells with non-zero expression. For cells with zero expression, a separate metric applies. Scores combine predictions across **global and local regions** for a balanced final score. # Crunch 3 – Dec 9 to Apr 30 – Identifying Gene ## Overview In **Crunch 3**, your task is to design a gene panel that best distinguishes **dysplasia regions** from **noncancerous mucosa regions** in colon tissue affected by Inflammatory Bowel Disease (IBD). Using provided H\&E images annotated by pathologists and single-cell RNA sequencing (scRNA-Seq) data, you will rank **18,615 protein-coding genes** based on their ability to discriminate between these disease states. If you participated in **Crunch 1** or **Crunch 2**, you may leverage your previously developed models to make gene expression predictions on the annotated regions and design your gene panel based on these predictions. If not, you can design your gene panel from scratch using biological insights or other approaches. Additionally, you are required to: * Provide a **justification** for how you constructed your gene panel. * [#peer-review](#peer-review "mention") three submissions from other participants based on their justifications. ### X (Inputs Data) #### H\&E Images and Annotations * **First H\&E Image**: Includes only noncancerous mucosa (already provided in Crunch 1 and Crunch 2). * **Second H\&E Image**: Entire colon tissue section including both dysplasia and noncancerous mucosa regions (`UC9_I-crunch3-HE.tif`). * **Associated Files**: * **Nucleus Segmentation Masks**. * **Tissue Region Masks with Annotations** (`UC9_I-crunch3-HE-dysplasia-ROI.tif`): * **0**: Other tissue regions. * **1**: Noncancerous mucosa. * **2**: Dysplasia. #### Single-cell RNA-Seq Data * **Dataset**: `Crunch3_scRNAseq.h5ad`. * **Content**: Gene expression data for 18,615 protein-coding genes from colon tissue samples with and without dysplasia. * **Cell Metadata** (`adata.obs`): * **Cell Type**: `adata.obs["annotation"]`. * **Individual**: `adata.obs["individual"]`. * **Disease Status**: `adata.obs["status"]` (Normal, Unaffected tissue, Polyp, Adenocarcinoma). * **Dysplasia Status**: `adata.obs["dysplasia"]` (`y`, `n`, or `ND`). #### Expression Data * **Normalized Counts**: `adata.X` (log1p-normalized). * **Raw Counts**: Available in `adata.layers["counts"]` ### Y (Targets) #### Gene Ranking Rank all 18,615 protein-coding genes from **1** (best discriminator) to **18,615** (worst), based on their ability to distinguish between dysplasia and noncancerous mucosa regions.
{% hint style="info" %} Including genes associated with different biological functions can enhance your gene panel and will be considered in the [#evaluation](#evaluation "mention") {% endhint %} ## Expected Output Your submission should include the following. ### Gene Ranking DataFrame * **Format**: A DataFrame returned by your `infer` function. * **Structure**: * **Index (Rank)**: Unique integers from **1** (best discriminator) to **18,615** (worst). * **Column (Gene Name)**: Gene symbols matching those provided in the dataset. ### Justification Report * **File**: `REPORT.md` * **Length**: Maximum **1 page**. * **Content**: * **Method Description**: Explain how your method works. (5-10 sentences) * **Rationale**: Describe the reasoning behind your gene panel design. (5-10 sentences) * **Data and Resources Used**: Specify the datasets and any other resources utilized. (5-10 sentences) * **References**: May be included (not counted toward the page limit). * **Hard Requirement**: A submission will be rejected outright if the file is missing: * If you are submitting via the CLI, just create a `REPORT.md` at the root of your submission. * If you are submitting via a Notebook, you must write it in a Markdown Cell.\ [Learn more about Embed Files.](/competitions/participate#embed-files) * Only non-empty and non-comment lines are considered.\ This is an example of what is expected: ```markdown # Method Description Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam eget augue quis metus viverra vehicula sit amet lacinia odio. # Rationale Praesent dignissim ipsum vel leo eleifend, eget pulvinar mauris ornare. Duis efficitur lectus posuere iaculis dictum. # Data and Resources Used Donec feugiat eros vel odio gravida venenatis. Nam et sem sit amet nisi vestibulum semper bibendum et libero. ``` {% hint style="warning" %} The report is attached to the submission.\ To modify the `REPORT.md`, you must resubmit with the modified content. {% endhint %} ## Peer Review * **Mandatory Participation**: To qualify for prizes, you must review **three submissions** from other participants. * **Purpose**: The peer review process is crucial for selecting the most promising gene panels for [#experimental-validation](#experimental-validation "mention"). Your evaluations help identify submissions with strong justifications and innovative approaches, contributing to the advancement of dysplasia research. * **Evaluation Criteria**: * Assign a score on a 1-3 scale * **1** - excellent justification * **2** - adequate justification * **3 -** poor justification * Provide a short explanation (200-400 words) covering: * Rationale of design. * Novelty of design. * Compliance with the required format. ## Evaluation We will assess your submissions based on two key criteria: * **Classification Accuracy**: We'll use your top 50 genes to train a model that distinguishes between dysplasia and noncancerous mucosa. The better your genes help the model correctly identify these regions, the higher your accuracy score will be. This is the main factor in determining your ranking. * **Diversity**: We'll also consider the variety of biological functions represented in your gene panel. Including genes from different pathways enhances the panel's usefulness and may provide deeper insights into dysplasia. A more diverse panel is favorable and can help differentiate teams with similar accuracy. Your final ranking will prioritize classification accuracy, with diversity as a supplementary factor to distinguish between submissions with close accuracy scores. ## Experimental Validation To validate the most promising gene panels, **we will select up to 500 genes for experimental evaluation**. This selection will occur via two routes: * **Route 1:** the top performers from Crunch 2 who also participate in Crunch 3 will have up to 50 of their highest-ranked genes included. * **Route 2**: the top performers from Crunch 3, determined by peer review and expert evaluations, will also have up to 50 of their top genes included. We will **order a Xenium gene** **panel** comprising these selected genes, reserving a small number of additional genes to identify important cell types in the colon. This panel will be used to perform spatial transcriptomics measurements on a new colon tissue section diagnosed with dysplasia, enabling **experimental validation of your gene panels.** # Full Specifications {% hint style="info" %} [Open the original document here.](https://crunchdao--competition--production.s3.eu-west-1.amazonaws.com/competitions/broad-1/documents/broad-spatial-challenge.pdf) {% endhint %} {% embed url="" %} # Lectures The following videos dive into the concepts underlying the Broad Institute Challenge and how machine learning can help autoimmune disease research. Each lecture guides participants through data handling, model training, and algorithm testing, emphasizing practical applications for advancing diagnosis, treatment prediction, and patient outcomes in autoimmune diseases. It’s ideal for those participating in the challenge or interested in applying AI solutions in medical research. {% embed url="" %} # ADIA Lab Causal Discovery {% embed url="" %} ### Overview Discovering the causal structure that governs the relationships among variables from their observations is a challenging and valuable problem in many domains of application, like healthcare, economics, social sciences, environmental science, education, etc. In this competition, the basic building block that you are given is a dataset of observations of a set of variables and your task is to discover the causal directed acyclic graph (DAG) that defines the causal relationships between them.

From a pandas DataFrame to a causal directed acyclic graph (DAG)

### Description The task of this competition is *causal discovery*: your goal is to find the causal graph (DAG) for each dataset you will be given. To help you in this endeavor, we provide a large number of example datasets together with their corresponding causal DAGs — as the training set — so that you can calibrate your unsupervised discovery methods, or train your prediction models if you prefer a supervised approach. Your causal discovery algorithm has to be designed to take as input a dataset and to output the causal DAG. All causal graphs in this competition have a specific structure: they have at least two special nodes, X and Y, which are the treatment and the outcome variables, respectively. The treatment variable X is the one that causes effects on the outcome variable Y. All other variables/nodes may or may not influence X and Y, possibly interfering with their relationship X→Y, so each may act as a confounder on X→Y, or as a collider, mediator, or be a cause or consequence of X (or Y), or not have any influence at all, etc. The goal of the competition is to estimate the causal graph behind each dataset. The scores are based on accurately identifying the role of all nodes on X→Y. Both unsupervised and supervised approaches are warmly welcome. ### Evaluation In all datasets, there are two special variables — X and Y — that are the treatment and the effect. We always assume that there is a causal link from X to Y: X→Y. For each predicted graph, the evaluation metric quantifies the correctness of the edges/arrows for all nodes but considers only the edges (or lack of) from each node to X and Y. In other words, the evaluation metric wants to assess the effects of errors in specifying wrong edges affecting X and Y. Each node K (with the exclusion of X and Y) can be in one of these 8 categories: 1. Confounder: K→X, K→Y, X→Y 2. Collider : X→K, Y→K, X→Y 3. Mediator: X→K, K→Y, X→Y 4. Independent: X→Y (no links to X or Y) 5. Cause of X: K→X→Y 6. Consequence of X: X→K, X→Y 7. Cause of Y: K→Y, X→Y 8. Consequence of Y: X→Y→K Each node in your predicted graph will be tested against its true class and the final scoring metric across all datasets is the ***multiclass balanced accuracy***. Participants should submit predicted DAGs for all datasets, and we will transform the predicted DAGs to the corresponding classes for scoring. ### Prediction File For each `example_id` in the test set, which is in the form `__` you must predict a binary value (0 or 1) representing the absence or presence of a causal link between `` and ``. The file should contain a header and have the following format: ``` example_id, prediction 00000_0_0, 0 00000_0_1, 0 00000_0_X, 1 00000_0_4, 0 etc. ``` For example, the row `01234_X_1, 1` means that for the test dataset `01234`, the participant predicts a causal link between X and 1: X→1. ### Dataset Description The whole dataset of the competition, between the training set and test set, comprises 47,000 individual datasets, each of 1000 observations for a certain number of variables, which is between 3 and 10. For the training datasets, the corresponding causal graphs are available. The causal graph is provided via its adjacency matrix, so if the dataset has 8 variables, the adjacency matrix is 8x8 matrix — which becomes 9x9 in the corresponding CSV file because the variable names are indicated for each row and column — where a value of 1 at position (i, j), means that variable i causes variable j, and value 0 means it does not. ### Tutorial #1 {% embed url="" %} ### Prize | Winners’ rank | Prize value | | ------------- | ----------- | | 1st place | $40,000 USD | | 2nd place | $20,000 USD | | 3rd place | $10,000 USD | | 4th place | $5,000 USD | | 5th place | $5,000 USD | | 6th place | $5,000 USD | | 7th place | $5,000 USD | | 8th place | $3,500 USD | | 9th place | $3,500 USD | | 10th place | $3,000 USD | # ADIA Lab Market Prediction Competition ## **A cross-section forecast problem** In finance, predicting asset price returns is a fascinating yet very hard problem. For this reason, alternative prediction problems have emerged in an attempt to circumvent these difficulties and still obtain predictions with tradeable potential. One of the most interesting alternatives is the problem of identifying the relative ordering in performance of an investment vehicle, in the cross-section of a pool or subset of them. This is the *cross-section forecast problem*. In this setting, we track a pool of investment vehicles that are generally obtained through some rule (for example S\&P 500 tracks the stock performance of the 500 largest companies in the US) at different dates. This pool is known as the *universe* in financial jargon and its definition is an object of study by itself. The goal of this competition is to rank the performance of all assets in the universe from best to worst at each given date. The target to predict in this competition is the ranking of the future performance of each asset, remapped to the interval \[-1,1], and the scoring function is Spearman's rank correlation between the predicted vs true rankings. To illustrate an interesting use case of this problem, we can imagine an investment strategy that is long on the best-performing element of the universe, and short in the worst. In this setting, no matter the direction of the market is still possible to obtain positive returns - or to minimize losses. The dataset presented to the competitors is an obfuscated version of high-quality market data. Therefore, details such as the nature of each investment vehicle, the constant frequency at which dates are measured, and the definition of each feature, are not available. We hope you enjoy the challenge! ## **Competition Phases and Format** This competition is focused on forecasting and has two phases. The first is the *submission phase* where participants can submit and test their models. The second phase, which is automatic, involves running the models against unobserved live market data. ### Submission phase - **12 weeks** From **May 16, 2023, 05:00 PM CET** to **August 16, 2023, 23:99 PM CET**. In the first phase, participants are required to submit either a Python notebook (.ipynb) or Python script (.py) file. This file should contain the necessary code to build, load, or update their models trained on the data. The code will be executed by the CrunchDAO platform for every submission, to obtain predictions on unseen data. Participants can either use static models, trained only once on the initial training set, or dynamic models that update or retrain themselves on the unseen data, as explained further in the documentation. ### Out-of-Sample phase - **12 weeks** From **August 16, 2023, 00:00 PM CET** to **November 16, 2023, 00:00 PM CET**. In the second phase, also called [Out-of-Sample](https://en.wikipedia.org/wiki/Cross-validation_\(statistics\)) (OOS), the participant's code will be automatically run by the platform on live market data and evaluated. In this phase, the participants won't be able to modify their code. ### Why the two-phase approach? * Only the performance on Out-of-Sample data will be taken into account. * Reproducibility of the winning solution is ensured. * Participants won't be able to exploit data leaks. {% hint style="info" %} CrunchDAO is acting as a third-party intermediary in this competition and will never communicate the code to the organizer in any way. {% endhint %} ## Evaluation ### The objective of the competition The goal of the participant is to rank the target variable for each stock in the Adia Lab investment universe, from the highest to the lowest, at each given date. This doesn't require estimating the exact target value for each investment; rather, it involves identifying which investments are likely to perform better than others. Participants can obtain this information from the various features (or Xs) describing each investment at each date in the provided dataset. The features' meanings are unknown to both CrunchDAO and the participants to prevent bias and facilitate sharing of the anonymized dataset. ### The scoring metric This competition is evaluated on [Spearman Rank Correlation](https://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient). Each row in the test set represents the predictions (X) associated with a stock of the universe at a given date and its target (Y). ## Data Each row of the dataset describes an investment vehicle at a certain date. Here follows a concise description of the columns of the three files comprising the dataset, `X_train` and `y_train`. `X_train`: * `date`: A sequentially increasing integer representing a date. Time between subsequent dates is a constant, denoting an unknown but fixed frequency at which the data is sampled. The initial training dataset is composed of 268 dates. * `id`: A unique identifier representing the investment vehicle at a given date. Note that the same asset has a different `id` at each date. * `0,...,460`: Anonymized features describing an investment vehicle at a given date. Derived from high-quality market data. `y_train`: * `date`: Same as in `X_train`. * `id`: Same as in `X_train`. * `y`: The target value to predict. It is related to the future performance of the investment vehicle at the given date. The value is normalized between `-1` and `1`. `X_test`: * Same structure as `X_train` but comprises only a few dates. This file is used to simulate the submission process locally via `crunch.test()`, or `cruch test`. The aim is to help participants debug their code and have successful submissions. A successful local test usually means no errors during execution on the submission platform. The dataset is obfuscated. ## Prize The winner's rank will be determined at the end of the [Out-of-Sample](/other/glossary#out-of-sample-phase) period, based on the metric described in the [#evaluation](#evaluation "mention") section.
Winner’s rank Prize value
1st Place$40,000
2nd Place$20,000
3rd Place$10,000
4th Place$5,000
5th Place$5,000
6th Place$5,000
7th Place$5,000
8th Place$3,500
9th Place$3,500
10th Place$3,000
## Original Documentation {% embed url="" %} # Rallies A Rally's objective from a client's perspective is to be an iteration of machine learning models on its data in order to assess the quality of its features and data preprocessing. Rallies usually have lower Cash Prize. Their might be multiple iterations of rallies for the same client that could lead in the end to a competition with a higher Cash Prize. # Mid+One Attacking together, make us stronger. Welcome to Mid+One! Dive into the world of martingales and market dynamics. Spot tiny shifts in high-frequency time-series, to predict where prices are heading. It's all about finding that elusive mid-price, one minute into the future. Ready to attack? With Mid+One Crunch has found another opportunity: Thousands of banks are consuming mid-market prices for their execution algorithms. The community meta-model will unlock a stream of ongoing rewards with a potential to serve a multi-billion dollar market. ## TL;DR {% hint style="info" %} * Detect small exceptions to the martingale property of a time-series. * Determine when a time-series will rise or fall. * A "buy and hold" strategy is applied for each prediction over the next 30 time steps. * The goal is to maximize profit after accounting for the transaction costs. * Only one attacker can be selected for OOS and Reward * [Quickstarter notebook](https://colab.research.google.com/github/crunchdao/quickstarters/blob/master/competitions/mid-one/quickstarters/mean_reversion_attacker/mean_reversion_attacker.ipynb). {% endhint %} ## Problem Statement You will build algorithms that takes one data point at a time and decides whether the average future value 30 steps in advance will be higher or lower than the present value. You only have the time-series, nothing else. Your prediction must be determined only by the past history of the time-series and by patterns you detect therein. Unlike most forecasting tasks, however, you goal is not to provide a precise prediction 30 steps in the future. Instead you should decide between three possibilities: 1. The time-series will go up, on average, by at least `EPSILON`; 2. The time-series will go down, on average, by at least `EPSILON`; 3. Or the average value of the time series will fall between `-EPSILON` and `EPSILON`. {% hint style="info" %} `EPSILON` is the Transaction cost value and is set at `0.0025` {% endhint %} ## Evaluation For every non-zero prediction, the system initiates a "buy and hold" for 30 data points. If the prediction is positive we go buy and hold. If the prediction is negative we go short and hold. However, a fixed transaction cost (`EPSILON`) is applied to the profit in both case. #### Example If the value rises by `0.50` over the next 30 periods; the profit will be `0.50` and the net profit would be `0.4975`. Similarly, should the price fall by `0.20` then the net profit would be `-0.2025`. ### Only one Model on the Leaderboard In the second Rally, you have to choose which model will appear on the leaderboard. You can still play with 4 different models. {% hint style="info" %} [Learn more how to select your model...](/competitions/leaderboard#only-one-model-on-the-leaderboard) {% endhint %} ### Time constraints {% hint style="danger" %} Your tick and predict must run in less then 20ms! {% endhint %} In Mid+One, delivering value to the customer quickly is crucial. Crunch aims to tackle increasingly lower frequencies. As a result, in the infer function, your code must not take more than 20ms from receiving the message to returning the result. If your average inference time exceeds this limit, your position on the leaderboard will be marked with an "Out-Of-Range" badge. ### Out-of-Range badge Submissions that are too slow but still achieve great results will be rewarded for the Rally. However, they cannot be deployed in Production or used by Financial Institutions. If you're out of range, there are plenty of ways to optimize your code to meet the 20ms threshold. {% hint style="info" %} Keep in mind that gains in Production will be much higher than during the Rally. OPTIMIZE! {% endhint %} ## Phases and Format ## Timeline Mid+One is going to evolve into a live Crunch. We went through a first 2 months test phase called "Rally" in order to ensure both problem statement, data and models integrity. * **Friday Oct 18, 2024, 09:00 AM CET** - First rally open * **Wednesday Dec 18 , 2024, 09:00 AM CET** - First Out-of-Sample scoring * **Wednesday Jan 8, 2025, 09:00 AM CET** - Submission re-open - Second Rally * **Sunday Feb 16, 2025, 11:59 PM CET** - Out-of-Sample - Second Rally * Live is soon to be announced {% hint style="info" %} **Live** refers to "in Production" mode where Crunch and the submitted models actively serve real-world end customers. {% endhint %} ### Submission Phase During the [Submission Phase](/other/glossary#submission-phase) the Crunchers are required to submit valid Notebooks or Python files. This submission need to "run" successfully on the Crunch hub in order to receive to be triggered in [Out-of-Sample Phase](/other/glossary#out-of-sample-phase) and receive live data. {% hint style="info" %} [Get started quickly with a Quickstarter!](https://colab.research.google.com/github/crunchdao/quickstarters/blob/master/competitions/mid-one/quickstarters/mean_reversion_attacker/mean_reversion_attacker.ipynb) {% endhint %} ### **Out-of-Sample Phase** Once the [Out-of-Sample Phase](/other/glossary#out-of-sample-phase) start, new data will be run through the models submitted. ## Data In Mid+One, participants are facing univariate time-series called [Streams](/competitions/participate/data#stream). Crunch's Streams are iterator objects that allow you to traverse all elements of a time-series, one at a time. ```python # Print the first x_train time-serie's content for message in x_train[0]: print(message) # Would print # ({x: 10303.346153849048}) # ({x: 10303.500000002896}) # ({x: 10303.461538464431}) # ({x: 10303.461538464431}) ... # ({x: 10303.269230772126}) # ({x: 10303.384615387501}) # ({x: 10303.307692310584}) ``` ## Building an Attacker with the Mid+One package This package is intended to make life simpler for those participating in Mid+One. ```bash $ pip install --upgrade midone ``` {% hint style="info" %} [Find the Mid+One package's code here.](https://github.com/microprediction/midone) {% endhint %} ## Some Concepts * `Attacker` is a Python class that consume a univariate sequence of numerical data points (such as stock prices, bond prices, or any time series) `x1`, `x2`, …`xt` and attempts to predict its future movement. * `Tick` is a method from the `Attacker` class that allows the consumption of incoming data points. * `Predict` is a method from the `Attacker` class that take a decision base on previous data points. * `Tick&Predict` is a method from the `Attacker` class that do `Tick` and `Predict` in a single function call. * `Accounting` handle tracking and logging the profit and loss (PnL) for decisions made by an `Attacker`. ### Example Attackers {% hint style="info" %} Read more in [attacker.md](https://github.com/microprediction/midone/blob/main/midone/attackers/attacker.md). {% endhint %}
NotebookDescription
Mean reversionA minimalist contest entry notebook
Mean reversion attackerIllustrates use of the Attacker class
Momentum attackerIllustrates use of running calculations
Regression AttackerIllustrates running regression pattern
### Attackers FAQ Some common questions have already been answered in the [FAQ.md](https://github.com/microprediction/midone/blob/main/midone/attackers/FAQ.md). ## Prizes * In the first Rally, the top ten performers judged by profit and loss will share $10,000 in proportion to their profit in the out of sample period. (done) * In the second Rally, the top 50 performers with positive profit and loss will share $10,000 in proportion to their profit in the out of sample period. {% hint style="info" %} [An example is available here.](https://github.com/microprediction/midone/blob/main/PRIZES.md) {% endhint %} # DataCrunch Rally ## Overview DataCrunch uses the quantitative research of the CrunchDAO to manage its systematic market-neutral portfolio. DataCrunch built a dataset covering thousands of publicly traded U.S companies. The long-term strategic goal of the fund is capital appreciation by capturing idiosyncratic return at low volatility. In order to achieve this goal, DataCrunch needs the community to assess the relative performance of all assets in a subset of the [Russell 3000](https://www.investopedia.com/terms/r/russell_3000.asp) universe. In other words, DataCrunch is expecting your model to rank the constituent of its investment universe. This rally is a new iteration on DataCrunch's dataset called master-v3. ## Prize The total cash prize for the rally is 10,000 $USDC, equivalent to 3,125 $CRUNCH. Winners will be able to choose their prize format, $CRUNCH or $USDC. ## Rally Phases And Format The Rally is formatted in two phases: * The `Submission Phase:` competitors will have 1 month to build, test and submit their model. Once they are satisfied with it, they will need to select it in order to compete in the `Out-Of-Sample` phase. * Dates: March, 15th - April, 29th 11:59:59PM CET 2024. * The `Out-Of-Sample Phase:` the models will be run on the Out-Of-Sample data for four weeks, without any possibility to modify the submission. The scores will be released each week on Fridays. * Final Leaderboard Date: May, 24th 2024. ## Data Each row of the dataset describes a stock at a certain date. The dataset is compose of three files, `X_train y_train and X_test`. ### X\_train * `Moons`: A sequentially increasing integer representing a date. Time between subsequent dates is constant, denoting a weekly fixed frequency at which the data is sampled. * `id`: A unique identifier representing a stock at a given Moon. Note that the same asset has a different `id` in different Moons. * `Feature_Industry`: the industry to which a `stock` belongs at a given `moon`. To ensure market neutrality, DataCrunch limits exposure to industry bets by cross-sectionally neutralizing model output before scoring. Therefore, it is important that your model does not bet on any particular industry but provides accurate performance estimates across industries. * (`Gordon_Feature_1`, …, `Dolly_Feature_30`): Anonymised `features` that describe the state of assets on a given date. They are grouped into several families, or ways of assessing the relative performance of each stock on a given month. Note: All features have the string "Feature" in their name, followed by a code name for the feature family. ### y\_train * `Moons`: Same as in `X_train`. * `id`: Same as in `X_train`. * (`target_b_idiosyncratic`, …, `target_b_4f_neutral`): the targets that may help you build your models. Target\_w, r, g, b refer to 7, 28, 63, 91 days compounding of returns. * In particular, `target_w` is a quantised version of the target that you are scored against. Its value is quantised in 7 bins, following the time-dependent, fat-tailed geometry of returns. ### X\_test Same structure as `X_train` but comprises only 5 Moons. This file is used to simulate the submission process locally via `crunch.test()` (within the code), or `crunch test` (via the cli). The aim is to help participants debug their code and have successful submissions. A successful local test usually means no errors during execution on the submission platform. ## The Performance Metric The infer function from your code will return your predictions. The latter will be processed in order to compute your performance for each cross-section. We call this performance metric `alpha_score`. Your inference is [processed](https://github.com/crunchdao/crunch-cli/blob/main/crunch/vendor/datacrunch.py#L18) as follows: 1. **Gaussianisation of the prediction:** This step ensures the standardization of the inferences. Different model and method may give inferences that are calibrated differently. In order to compare them, this step is recentering, rescaling and reshaping all the inferences. 2. **Orthogonalisation:** linear neutralization against industries and common sources of returns. This step makes sure that the information provided by your inference is not too obvious (industry) or already known by DataCrunch; in other words, DataCrunch makes sure that you contribute incrementally to their modeling capabilities. 3. **Mean-zero:** in addition to the neutrality of the prediction, setting it to mean zero informs the score about the dollar neutrality constraint of the portfolio at rebalancing. 4. **L1 normalisation:** By keeping the sum of the absolute value constant and mean zero from the previous step, we ensure that your inference is as close as possible to a dollar-neutral, constant size, and tradable portfolio for DataCrunch. 5. **Dot Product:** dot product against the un-quantised version of `target_w`. This product is a proxy for the weekly returns of the simulated portfolio weights multiplied by the return of each individual position. ### Final Score Your final score will be the cumulative product over time of the above simulated weekly returns (i.e., `alpha_score`) over the the Out-Of-Sample period. The use of a non-local metric is key in being able to estimate the covariance structure of the various ML-based alphas, beside reflecting the compounding nature of financial returns. ### Interacting with the performance metric The `alpha score` can be called locally and in the cloud environment with the function `crunch.alpha_score()`. This should be useful for you to cross-validate your model or even use it in your fitness function. In this way, data privacy is preserved and, at the same time, you are provided with the necessary informations to build performant models. ### Reward Scheme The Cash Prize will be shared as follow: ```python import numpy as np payout_pool = 3125 # $CRUNCH player_score = np.cumprod(alpha_score + 1) - 1 player_payout_factor = player_score if player_score > 0 else 0 player_share = player_payout_factor / sum(players_payout_factor) player_payout = player_share * payout_pool ``` ## Computing Resources Competitors will be allocated a specified quantity of resources within the cloud environment for the execution of their code automatically. During the submission phase, they are entitled to 10 hours of computing time per week, and for the scoring phase, this allocation increases to 20 hours per week. ## IP Sharing Only the predictions of the models will be used by DataCrunch, and only for research purposes. ## Quickstarter Notebook A Quickstarter notebook, together with an Exploratory Data Analysis one, is available below so you can get familiar with what is expected from you and how to use the `crunch.alpha_score()` function. {% embed url="" %} ## # X-Alpha Rally CrunchDAO & X Alpha offer this first of a series of $10,000 Bounty to build the most Powerful, Robust and Unbiased AI-Driven VC algorithm. ## Overview The evolving landscape of [Venture Capital](https://en.wikipedia.org/wiki/Venture_capital) is marked by the automation of both data and investment funds, including Micro VC and solo general partners. This trend is leading to a comprehensive decentralization within the venture class. Against this backdrop, investors and Limited Partners are increasingly in need of a systematic approach to maximize returns across diverse asset classes. In response to this need, the CrunchDAO is crunching the data of more than 2 million startups and 28 million founders, in order to discover the hidden patterns and relationships that will fuel the next wave of venture capitalism. ## Problem Statement ### **Cross-sectional Approach** The proposed approach allows for a comprehensive analysis of startups by simultaneously examining various data points and trends. This method contrasts with traditional models by integrating diverse data sets and employing advanced statistical techniques to discern both linear and non-linear relationships. Such a multifaceted view enables more accurate predictions and therefore effective capital allocation, incorporating quantitative risk management strategies not commonly used in the Venture Capital sector. ### **Supervised Classification Approach** Startups have been categorized, enabling participants to develop supervised learning algorithms. Startups labeled as 1 are expected to achieve higher valuations, while those labeled as 0 are not anticipated to experience significant valuation growth. ## Evaluation ### Objective The goal of the participant is to perform binary classification \[0, 1] on a universe of startups. This requires submitting the exact value of 0 or 1 for each investment opportunity (row of the dataset). Startups labeled as 1 are expected to achieve higher valuations, while those labeled as 0 are not anticipated to experience significant valuation growth. The ensembling of all the submissions will allow identifying which investments are likely to perform better than others by ranking them according to the overall community consensus on each opportunity. ### **Scoring Metric** The [F1 score](https://en.wikipedia.org/wiki/F-score) will be used in order to assess the models performance effectively. This metric balances the precision (true positives identified by the algorithm) and recall (accounting for missed opportunities). For the algorithm to demonstrate its effectiveness, it must accurately identify investment opportunities while minimizing false negatives and false positives. The F1 score will provide a comprehensive view of the algorithm's accuracy and reliability. $$ F1=\frac{2∗Precision∗Recall}{Precision+Recall} $$ Where: $$ Precision=\frac{TP}{TP+FP} $$ $$ Recall=\frac{TP}{TP+FN} $$ ## **Competition Phases and Format** In the first phase, participants are required to submit either a Python notebook (.ipynb) or Python script (.py) file. This file should contain the necessary code to build, load, or update their models trained on the data. The code will be executed by the CrunchDAO on the Out-Of-Sample data. Participants can either submit static models, trained only once on the initial training set, or dynamic models that update or retrain themselves on the unseen data, as explained further in the documentation. ## Timeline **Submission Phase:** * **December 8, 2023, 06:00 PM CET** - Start of the competition. * **January 8, 2023, 05:59 PM CET** - Submission deadline. You must accept the competition rules before this date. **Out-of-Sample Phase:** After the final submission deadline, there will be one update to the leaderboard that will reflect your score on the OOS data. * **January 9, 2023, 06:01 PM CET** - Out-of-Sample scoring begins. * **January 12, 2023, 06:00 PM CET** - Final Leaderboard release. ## Data Each row of the dataset describes an investment vehicle at a certain date. Here follows a concise description of the columns of the files: `X_train`: * `date`: A sequentially increasing integer representing a date. Time between subsequent dates is a constant, denoting an unknown but fixed frequency at which the data is sampled. The initial training dataset is composed of 268 dates. * `id`: A unique identifier representing the investment vehicle at a given date. Note that the same asset has a different `id` at each date. * `feat_1, ..., feat_n`: Anonymized features describing an investment vehicle at a given date. `X_test`: * Same structure as `X_train` but comprises only a few dates. This file is used to simulate the submission process locally via `crunch.test()`, or `crunch test` (if using the CLI). A successful local test usually means no errors during execution on the submission platform. The dataset is obfuscated. ## Prize The winners will be determined at the end of the [Out-of-Sample](/other/glossary#out-of-sample-phase) period, based on the metric described in the [#evaluation](#evaluation "mention") section. | Winners’ rank | Prize value | | ------------- | ----------- | | 1st Place | $6,000 USD | | 2nd Place | $3,000 USD | | 3rd Place | $1,000 USD | ## Original Documentation {% embed url="" %} # Participate To get started and submit your first model, you will need to pass through the following steps. ## Register Creating an account on the CrunchDAO platform will allow you to be identified and get access to the competition dataset. Follow the link below to join the competition. {% embed url="" %} ## Submit Two distinct formats of submission are accepted for the competitions: * **Jupyter Notebook** (`.ipynb`), which is a self-contained version of the code * **Python Script** (`.py`), which allows more flexibility and to split the code into multiple files {% hint style="info" %} All the work you submit remains your exclusive property. The Crunch Foundation guarantees the privacy of both client data and competitors' code. {% endhint %} ### Jupyter Notebook Notebook users can use the Quickstarters provided by CrunchDAO to quickly experiment with a working solution that users can tinker with. #### Setting the Environment Before trying to execute any cell, users must set up their environment by copying the command available on the competition page:

The "Submit a Notebook" tab from the "Submit" page of a competition

Run the commands to set up your environment and download the data to be ready to go: {% code title="Python Notebook Cell" %} ```python # Upgrade the Crunch-CLI to the latest version %pip install crunch-cli --upgrade # Authenticates yourself, it will downloads your last submission and the data !crunch setup --notebook --token ``` {% endcode %} {% hint style="info" %} [Learn more about how setup tokens work and if it is safe to leak them.](#setup-tokens) {% endhint %} Users can now load the data locally: {% code title="Python Notebook Cell" %} ```python # Load the notebook, run me once import crunch crunch = crunch.load_notebook() # Load the data, re-run me if you corrupt the dataframes X_train, y_train, X_test = crunch.load_data() ``` {% endcode %} #### Local Testing When users are satisfied with their work, they can easily test their implementation: {% code title="Python Notebook Cell" %} ```python # Run a local test crunch.test() ``` {% endcode %} #### Submitting your Notebook After testing the code, users need to have access to the `.ipynb` file. * If you are on Google Colab: `File` > `Download` > `Download .ipynb` * If you are on Kaggle: `File` > `Download Notebook` * If you are on Jupyter Lab: `File` > `Download` Then submit on the **Submit a Notebook** page:
Some model files can also be uploaded along with the notebook, which will be stored in the `resources/` directory. [Read more about the file selection dialog.](/competitions/participate#file-selection-dialog) #### Automatic line commenting [Learn why and how your code is commented in your notebook when you submit it.](/competitions/participate/notebook-processor#automatic-line-commenting) #### Global variables [Learn how to use global variables in your notebook.](/competitions/participate/notebook-processor#global-variables) #### Specifying package versions [Learn how to specify package versions (requirements.txt) directly within your notebook.](/competitions/participate/notebook-processor#specifying-package-versions) #### Embed Files [Learn how to submit additional files with your notebook.](/competitions/participate/notebook-processor#embed-files) ### Python Script Script users can use the Quickstarters provided by CrunchDAO to know what the structure should be. A mandatory main.py is required to have both functions (`train` and `infer`) in order for your code to run properly. #### Setting the Environment Before starting to work, users must setup their environment which will be similar to a git repository.

The "Submit via CLI" tab from the "Submit" page of a competition

Run the commands to set up your environment and download the data to be ready to go: {% code title="Terminal" %} ```bash # Upgrade the Crunch-CLI to the latest version $ pip install crunch-cli --upgrade # Authenticates yourself, it will downloads your last submission and the data $ crunch setup --token [directory] # Change the directory to the configured environment $ cd ``` {% endcode %} [Read more about how setup tokens work and why it is safe to (accidentally) "leak" them.](#setup-tokens) #### Directory Layouts {% code title="File Explorer" %} ```bash # Example of a folder structure. # The data files may change depending on the competition. . ├── data/ │   ├── X_test.parquet │   ├── X_train.parquet │   └── y_train.parquet ├── main.py ├── requirements.txt └── resources/    └── model.joblib ``` {% endcode %}
File / DirectoryReason
data/Directory containing the data of the competition, should never be modified by the user. Always kept up to date by the CLI.
main.pyCode entry point. Must contain the train() and infer() function. Can import other files if necessary. Learn more...
requirements.txtList of packages used by your code. They are installed before your code is invoked.
resources/Directory where your model should be stored. The content is persisted across runs during the transition between the Submission and Out-of-Sample phases.
#### Local Testing When users are satisfied with their work, they can easily test their implementation: {% code title="Terminal" %} ```bash # Run a local test using a shell command $ crunch test ``` {% endcode %} #### Pushing your Code After the code has been tested, the submission needs to be uploaded to the server. The message is optional and is just a label for users to know what they did. {% code title="Terminal" %} ```bash $ crunch push --message "hello world" ``` {% endcode %} {% hint style="info" %} Remember to include all your dependencies in a `requirements.txt` file. {% endhint %} #### Package version freezes Before submitting, the CLI does a `pip freeze` in the background to find out what version you are using locally. These versions are then used to freeze the `requirements.txt` on the server. This is to ensure that if your code works locally with the exact same versions, it should *theoretically* work the same on the server. However, this behavior may result in your packages not being installed because: * you are using custom/externally installed package versions, * you have installed some packages by force, but PyPI considers them incompatible, * the architecture is different and the package is platform-specific. If you want to disable this behavior, use the `--no-pip-freeze` flag. {% code title="Terminal" %} ```bash $ crunch push --no-pip-freeze --message "hello world" ``` {% endcode %} {% hint style="info" %} Your original requirements will always be preserved.\ Don't hesitate to reach out to us on [Discord](https://discord.gg/veAtzsYn3M) or the [Forum](https://forum.crunchdao.com/) for help. {% endhint %} ### Hybrid For some complex setups, users may need to use the CLI to submit a Jupyter Notebook. This can happen if they want to submit with a large pre-trained model, or they want to include non-PyPI packages. It will be very similar to the Python Script setup: * [#setting-the-environment-1](#setting-the-environment-1 "mention"), like for a Python Script. * Remove the `main.py`. * Move your notebook to the project directory and name it `main.ipynb`. {% hint style="info" %} The `main` name can be changed by using the `--main-file .py` option. (keep the `.py` at the end) {% endhint %} If done correctly, before each crunch push, the CLI will first convert the notebook to a script file before sending it. {% hint style="warning" %} [Package version specifiers](#specifying-package-versions) will not work.\ The `requirements.txt` file must be updated manually. {% endhint %} {% hint style="info" %} [Package version freezes](/competitions/participate#package-version-freezing) are still being done in the background. {% endhint %} ### Files If you do not want to use the CLI and did not use a notebook to write your code, you can submit files directly. This is an advanced feature that requires preparation. The directory layout must be the same as [the CLI directory layout](/competitions/participate#directory-layouts). #### File Selection Dialog To add files you can: * select multiple files by clicking on the "Add file(s)" button * or select the contents of an entire directory by clicking on the "Add directory" button

Files selection dialog

{% hint style="info" %} If no files have been selected yet and you add a directory, that directory will be used as the root. {% endhint %} {% hint style="info" %} Due to a limitation of web browsers, it is not possible to select a directory via the "Add file(s)" button or add multiple directories via the "Add directory" button. To achieve this, either: * add multiple directories * or place your submission in a directory and add the directory once. {% endhint %} Once added, files can be disabled if you add too many, or renamed if the name is incorrect.

A model is selected

{% hint style="warning" %} You need to submit the `requirements.txt` yourself. {% endhint %} ### Setup Tokens The site generates new tokens every minute, and each token can only be used once within a 3-minute timeframe. This prevents any problems if your token is accidentally shared, as it will likely have already been used or expired. Even the team shares their expired tokens in Quickstarters. This token allows the CLI to download the data and submit your submission on your behalf. ## Run #### Checking your submission

A successful submission.

The system parses your work to retrieve the code of the interface functions (`train()` and `infer()`) and their dependencies. By clicking on the right arrow, you can access the contents of your submission.

The view of a submission once properly uploaded

#### Running in the Cloud Once you've submitted, it's time to make sure your model can run in the cloud environment. Click on a submission and then click the **Run in the Cloud** button.

Click Run in the Cloud to start your run

Your code is fed a standard epoch of data and the system simulates an inference.

If your submission ran properly, you'll see the status as successful.

{% hint style="info" %} A successful run means that the system will be able to call your code on new data to produce the inferences for that customer. {% endhint %} #### Debugging with the logs If your run crashes or you want to better understand how your code behaved, you can review the logs.

How to check your execution logs

Logs of a run

{% hint style="warning" %} Due to abuse, only the first 1,500 lines of a user's code logs will be displayed. {% endhint %} ## Select You will be given a few days after the end of the competition to select the run which will be used for running on the [Out-of-Sample Phase](/other/glossary#out-of-sample-phase). This time is used for participants that started a run at the very last second and need to wait for the results before making their selection. If you do not select anything, the last successful run will be selected. Some competitions (like [DataCrunch 2](/competitions/competitions/datacrunch-2)) do not offer this grace period, and even terminate still running runs when the [Submission Phase](/other/glossary#submission-phase) ends, because they are too time sensitive to afford to wait for all runs to finish. ### Why a Run and not a Submission? You can submit a pre-trained model with your submission, but this does not guarantee that your code will run properly in the cloud environment. You must [run it at least once to confirm its validity](/competitions/participate/resources-limit#runs). Runs also carry the new version of your model, allowing you to train it using a larger dataset that is not accessible locally. ```mermaid graph LR Submission -- No model --> Run1[Run 1] Run1 -- Trained model --> Run2 Run2 -- Re-trained model --> Run3 Submission --> Run2[Run 2] Submission --> Run3[Run 3] ``` This can be seen as a "chain" of runs, where they are all linked together by using an iterative version of your model to continue the work of the previous one. It resets if you decide to create a run during the submission phase and select it. ## Repeat Once you have had a successful run, you can start trying to improve your model up until the end of the competition. A new submission is required for each new attempt, as previous submissions cannot be altered or deleted for reasons of reproducibility. You only need to retry setting up your environment in two cases: * If you are reopening your notebook and the runtime has been reset, * If you have changed computers. In most cases, running a local test will ensure that you have the latest version of the data. # Notebook Processor When a notebook is submitted, it is first processed to extract the requirements and concatenate all the cells into a single Python file. This processing involves a lot and sometimes comments too much code. You can provide hints to help the processor better understand your intentions. The source code for the tool is available on GitHub: {% embed url="" %} ## Automatic line commenting The notebook is automatically converted into a Python script that only includes the functions, imports, and classes. Everything else is commented out to prevent side effects when your code is loaded into the cloud environment. (e.g. when you're exploring the data, debugging your algorithm, or doing visualizating using Matplotlib, etc.) You can prevent this behavior by using special comments to tell the system to keep part of your code: * To start a section that you want to keep, write: `@crunch/keep:on` * To end the section, write: `@crunch/keep:off` {% code title="Python Notebook Cell (before)" %} ```python # @crunch/keep:on # keep global initialization device = torch.device("cuda" if torch.cuda.is_available() else "cpu") # keep constants TRAIN_DEPTH = 42 IMPORTANT_FEATURES = [ "a", "b", "c" ] # @crunch/keep:off # this will be ignored x, y = crunch.load_data() def train(...): ... ``` {% endcode %} The result will be: {% code title="Python Notebook Cell (after)" %} ```python device = torch.device("cuda" if torch.cuda.is_available() else "cpu") TRAIN_DEPTH = 42 IMPORTANT_FEATURES = [ "a", "b", "c" ] #x, y = crunch.load_data() def train(...): ... ``` {% endcode %} The command does not affect comments, functions, classes, or imports. {% hint style="info" %} You can put a `@crunch/keep:on` at the top of the cell and never close it to keep everything. {% endhint %} ### Global variables If you want to use global variables in your notebook, put them in a class, this will improves the readability of your code: {% code title="Python Notebook Cell" %} ```python class Constants: TRAIN_DEPTH = 42 IMPORTANT_FEATURES = [ "a", "b", "c" ] def infer(): print(Constants.TRAIN_DEPTH) # 42 ``` {% endcode %} {% hint style="info" %} This method was introduced before the recommended `@crunch/keep:on` command. {% endhint %} ## Specifying package versions Since submitting a notebook does not include a `requirements.txt`, users can instead specify the version of a package using import-level [requirement specifiers](https://pip.pypa.io/en/stable/reference/requirement-specifiers/#examples) in a comment on the same line. {% code title="Python Notebook Cell" %} ```python # Valid statements import pandas # == 1.3 import sklearn # >= 1.2, < 2.0 import tqdm # [foo, bar] import scikit # ~= 1.4.2 from requests import Session # == 1.5 ``` {% endcode %} {% hint style="warning" %} Importing at a level other than the very top level will not work. Wrapping it in a `try-except` or `if-else` statement will generate a warning. The import will be ignored. Imports in functions are also ignored but do not generate warnings. {% endhint %} ### Inconsistent versions Specifying multiple times will cause the submission to be rejected if they are different. {% code title="Python Notebook Cell" %} ```python # Inconsistant versions will be rejected import pandas # == 1.3 import pandas # == 1.5 ``` {% endcode %} ### Standard libraries Specifying versions on standard libraries does nothing (but they will still be rejected if there is an inconsistent version). {% code title="Python Notebook Cell" %} ```python # Will be ignored import os # == 1.3 import sys # == 1.5 ``` {% endcode %} ### Optional dependencies If an optional dependency is required for the code to work properly, an import statement must be added, even if the code does not use it directly. {% code title="Python Notebook Cell" %} ```python import castle.algorithms # Keep me, I am needed by castle import torch ``` {% endcode %} ### Name conflicts It is possible for multiple import names to resolve to different libraries on PyPI. If this happens, you must specify which one you want. If you do not want a specific version, you can use `@latest`, as without this, we cannot distinguish between commented code and version specifiers. {% code title="Python Notebook Cell" %} ```python # Prefer https://pypi.org/project/EMD-signal/ import pyemd # EMD-signal @latest # Prefer https://pypi.org/project/pyemd/ import pyemd # pyemd @latest ``` {% endcode %} ## Embed Files Additional files can be embedded in markdown cells to be submitted with the Notebook. In order for the system to recognize a cell as an Embed File, the following syntax must be followed: {% code title="Markdown Notebook Cell" %} ```markdown --- file: .md --- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean rutrum condimentum ornare. ``` {% endcode %} Submitting multiple cells with the same file name will be rejected. While the focus is on Markdown files, any text file will be accepted. Including but not limited to: `.txt`, `.yaml`, `.json`, ... # Code Interface ## Requirement Your submission needs to provide at least three components: `import`s, `train()`, and `infer()`. 1. **`import`s**: As with any script, if your solution has dependencies on external packages be sure to import them. The system will automatically install your dependencies. Make sure that you only use packages that are [whitelisted](https://hub.crunchdao.com/competitions/datacrunch/submit/libraries). 2. **`train()`**: In the training function, users build and train the model to make inferences on the test data. The model must be stored in the `resources/` directory. 3. **`infer()`**: In the inference function, the trained model is loaded and used to make inferences on a sample of data that matches the characteristics of the training test. {% hint style="info" %} Some competitions may require more or fewer functions. Always read the competition's documentation. {% endhint %} ### Dynamic Parameters If required, parameters can also be queried by name: * If the name does not exist, `None` is used. * If a default value is specified, the value is retained (useful for local testing). * Typing is always ignored, so make sure it is correct. They can be used in both the `train()` and the `infer()` functions: ```python def train( X_train: pandas.DataFrame, y_train: pandas.DataFrame, has_gpu: bool, embargo: int, my_custom_value=42, # user specified ) -> None def infer( X_test: pandas.DataFrame, model_directory_path: str, ) -> pandas.DataFrame ``` ## Containers Some competitions will provide special objects as parameters that have unique behaviors of which you should be aware of. #### Iterable vs Iterator An iterator can be iterated many times, whereas an iterator can only be consumed once. ([learn more](https://stackoverflow.com/a/18809506/7292958)) The difference is subtle, but really important: * The `train()` function is called once, but can **consume the streams as many times as necessary** * The `infer()` function is called **only once per stream**, and there is no going back {% hint style="info" %} This specifically target the [Structural Break](/competitions/competitions/adia-lab-structural-break-challenge) competition. {% endhint %} # Data In all Crunchs, the crunchers can access our datasets as follow: ## Regular competitions ### The setup command The easiest way is to use the command provided in the `Submit` tab of each competition. This command ensures that you have the latest data and keeps it updated in the event of continuous competitions. ### Manual download If you prefer not to use the CLI, you can access the file via the `Resources` then `Datasets` tab. Those links only work as long as you have accepted the rules. This page allows you to download files after a competition has ended. ### Load the data Most competition data can be loaded using the `crunch_tools.load_data()` function. However, some data is too complex or large to be loaded automatically. If this is the case, refer to the Quickstarter, which provides a working example you can use as a base. The shape of the data is always described in the competition overview. Feel free to ask questions on the [Discord server](https://discord.com/invite/veAtzsYn3M) or the [forum](https://forum.crunchdao.com/). ## Real-time competitions Some competitions do not provide data directly via the platform, but instead use a dedicated mechanism. Always read the competition's overview to learn how to do so. # Whitelisted Libraries To prevent users from using malicious packages on the competition's infrastructure, a series of Python packages has been whitelisted. The whitelist is also used when converting notebooks because some libraries have different names when imported than on PyPI. For example, `scikit-learn` is `sklearn`. You can search the whitelisted library in the **Resources > Whitelisted Libraries** section of each competition. ## Requesting a package You can request to whitelist packages via the "**Request whitelisting**" button in the "**Resources > Whitelisted Libraries**" section of each competition.

Request to whitelist a library form

Administrators need some information to find the package on PyPI and approve it. * The **Name** is the package name on PyPI.\ e.g.: `scikit-learn` * The **Alias** is the name used in the Python code to access the package.\ e.g.: `sklearn` * The **GPU Requirement** is the hardware requirement to use the package: * If you are unsure, select **I don't know**. * If the package does not require a GPU at all, select **Useless for this library**. * If a GPU can be used but is not required, select **Can be useful for this library**. * If the package requires a GPU, select **Required for this library**. * You can leave **Additional Details** if the package requires special attention, so the administrators can better understand why. ### Requirements However, there are a number of requirements (pun intended) that must be met: * The package must be available on [PyPI](https://pypi.org/). * The package must have a good readme/documentation. * The package must not be new (more than 6 months old). * The package must have many downloads (e.g. [pandas](https://pypistats.org/packages/pandas)). * The package must have [verified details](https://docs.pypi.org/project_metadata/) (by PyPI). {% hint style="info" %} We reserve the right to refuse a package if it looks suspicious. {% endhint %} ### Package not on PyPI If the package is only available on GitHub, you will need to download it and include it in your submission. Only PyPI packages are allowed. # Resources Limit ## Submissions You can only submit during the [Submission Phase](/other/glossary#submission-phase). You can only submit up to 5 times per day. Some competition may allow more. You can see your current submission for the day at the bottom left. A submission cannot exceed 5GB and his `resources/` directory cannot exceed 10GB. {% hint style="info" %} We can **exceptionally** allow you more as we understand that debugging why your code works locally but not in the cloud environment can be very difficult. For this, [please contact us on Discord](/competitions/faqs/contact-us#help-with-the-hub-competition). {% endhint %} ## Models Also known as the `resources/` directory, it allows you to carry state information over multiple runs, provided the total size is under 10 GB. There is no limit on the number of files themselves, but if we detect abuse, we may introduce one that could break existing models. All types of file are permitted, including but not limited to: * persisted models (.joblib), * model weights (.pkl), * configuration files, * ... Compared to including them in your submission, those files can be modified and persisted across multiple runs. ## Runs For a run to be considered valid, it must: * not crash due to a bug in your code or because you have run out of RAM/disk space, * complete the work under the time constraints, * produce a prediction. Predictions are checked once the run is over. However, if a prediction is deemed invalid, the run itself will not be invalidated. ### **Resources** You can view the runtime specifications before creating a run:

Runtime option selector on the Run creation page.

{% hint style="info" %} The "Authorized Quota" is global; it is not allocated to each individual runtime. Some consume the quota faster, some slower, but almost always at the normal speed. This rate is based on how much they cost and how powerful they are. {% endhint %} #### Providers We are using multiple compute provider under the hood:
NameRuntime TypeContext
AWS FargateCPUGeneral computing.
AWS BatchGPUGeneral computing.
OpenGPUGPUPowerful GPUs.
GCP BatchCPU + GPUGeneral computing.
Using one provider instead of another should not affect your code as we make sure to always provide the same driver versions. If you think there is a bug, please [contact us on Discord](/competitions/faqs/contact-us#help-with-the-hub-competition). #### Networking Access to the internet in a cloud environment is not permitted. This is achieved by [only allowing local connections](https://linux.die.net/man/7/unix#:~:text=The%20AF_UNIX%20\(also%20known%20as,as%20being%20of%20type%20socket\).), which allows your favorite parallelization libraries to still take advantage of it. ### Quota The allocated quota are different per competition, you should the overview section to know how much is granted to every participants. You can also visualize your quota usage under the **Submissions & Runs** tab. Failed runs are only taken into account when computing the quota at a discounted rate: * Runs that reach a timeout will count for 5% of their duration. * Runs that are terminated will count for 5% of their duration. * All other crashes will count for 30% of their duration. #### Resets The quota resets every week during the [Submission Phase](/other/glossary#submission-phase). It only resets once at the beginning of the [Out-of-Sample Phase](/other/glossary#out-of-sample-phase). Most competitions are unaffected by this, as they run the models on the entire dataset at once. However, some competitions run models on small sections of the dataset over several weeks. #### Extras Whenever you create a run, the timeout is set 30 minutes after your quota. This allows you to still have a chance of completing the run if you only have a few minutes left. This also covers the time necessary to set up the environment, such as downloading the data and installing the libraries. This is valuable time during which your model is not running, but which is necessary for the system. But those minutes do not count towards your quota! If you exceed your quota -for example, if you use 15 hours and 30 seconds but your quota is 15 hours- you will not be able to create a new run. ### Security Your model runs with the lowest possible level of privilege. Most of the time, the data is fed directly to your function as an argument, as some pre-processing operations (such as filtering) are sometimes needed. Any users attempting to bypass the restriction mechanism will be **disqualified**! ### `crunch.load_notebook()` Using `crunch.load_notebook()` in the cloud environment isn't supported and might raise an error in the future. CLI users accidentally use it when trying to convert notebooks into regular Python script files. For these users, two alternatives are offered: 1. Use the dedicated `crunch test` command, which performs the testing command automatically, eliminating the need to write code. 2. Use a dedicated script. By creating a file named [`local_test.py`](#user-content-fn-1)[^1], you can import the necessary functions and run the testing function yourself. It also allows for automated pipelines to manipulate the output for additional checks (if desired). {% code title="local\_test.py" expandable="true" %} ```python import crunch from main import train from main import infer crunch_tools = crunch.load_notebook() crunch_tools.test() ... # read the prediction, do post-run analyses, ... ``` {% endcode %} {% hint style="info" %} Only competitions in the old format (like DataCrunch's, ADIA Lab's, or EWSC at Broad Institute's) are compatible. For competitions that require Web3 (such as Synth or Numinous), please read their respective documentation. As an adapted code of the second method will work. {% endhint %} ## Predictions Your prediction must not exceed a certain size. This limit varies depending on the competition, but is usually large enough to accommodate everyone. However, when participants are responsible for writing the prediction files themselves, they must also ensure that they name their files properly and use the correct flags to persist them in the correct format. The most common mistake is including the default `pandas.DataFrame` index when [saving a CSV file](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_csv.html#:~:text=indexbool%2C%20default%20True). Participants cannot download prediction files. If a check fails, the error message should include enough details to help you debug the situation (extra or missing columns, `NaN` or infinite values, ...). If you need further assistance, please, [contact us on Discord](/competitions/faqs/contact-us#help-with-the-hub-competition). [^1]: The file name does not matter. # R Usage Experimental polyglot support. {% hint style="warning" %} This feature is still experimental. If you encounter any issues, please contact us on [Discord](https://discord.com/invite/veAtzsYn3M) or in the [Forum](https://forum.crunchdao.com/). {% endhint %} R is now partially supported in the runner. We recommend using [rpy2](https://rpy2.github.io/) ([link to documentation](https://rpy2.github.io/doc/latest/html/introduction.html)) which embed a R runtime inside Python so it can run in the same process. Spawning a new process is still possible, but not recommended, as it will **considerably slow down your code** for big loops. ## Submit The R runtime will only be installed if a `requirements.r.txt` file is present in the root directory of your submitted files. As with Python packages, R packages are subject to a whitelist and can be requested using the same button. In the dialog, chose "R" in the "Programming Language" select. ### Notebooks #### Importing a package For notebook users, the packages are automatically extracted from the `importr("")` calls, which is provided by [rpy2](https://rpy2.github.io/). {% code title="Python Notebook Cell" %} ```python # Import the `importr` function from rpy2.robjects.packages import importr # Import the "base" R package base = importr("base") ``` {% endcode %} The following format must be followed: * The import must be declared at the root level. * The result must be assigned to a variable; the variable's name will not matter. * The function name must be `importr`, and it must be imported as shown in the example above. * The first argument must be a string constant, variables or other will be ignored. * The other arguments are ignored; this allows for [custom import mapping](https://rpy2.github.io/doc/latest/html/robjects_rpackages.html#importing-r-packages) if necessary. The line will not be commented, [read more about line commenting here](/competitions/participate/notebook-processor#automatic-line-commenting). #### Installing a package If the library is not available in your environment, you can install it using either `apt-get` (the recommended method) or the `install.packages()` from the `utils` package. {% code title="Python Notebook Cell" %} ```python # Use `apt-get` as a regular command !apt-get install r-cran-lme4 ``` {% endcode %} {% code title="Python Notebook Cell" %} ```python # Use the `utils` package, but it must be imported first from rpy2.robjects.packages import importr utils = importr("utils") # Call `install.packages(...)`, this will be commented on submit utils.install_packages("lme4") ``` {% endcode %} #### Declaring a R function [Any R code can be run using the `robjects.r(code)` function.](https://rpy2.github.io/doc/v2.9.x/html/robjects_rinstance.html) The following code can be used to [define a function written entirely in R](https://rpy2.github.io/doc/v2.9.x/html/robjects_rinstance.html#evaluating-a-string-as-r-code): {% code title="Python Notebook Cell" %} ```python from rpy2.robjects import r # Declare the function in R and have it return f = r(""" f <- function(r, verbose=FALSE) { if (verbose) { cat("I am calling f().\n") } 2 * pi * r } # Don't forget to return it f """) # Call the function circumference = f(3) ``` {% endcode %} {% hint style="warning" %} Even if the import must be at the root level, the call to `robjects.r(code)` must be made within a Python function so that it is not commented out. We recommend declaring them in a dedicated cell and using the `# @crunch/keep:on` command to prevent them from being commented out. [Read more about line commenting here](/competitions/participate/notebook-processor#automatic-line-commenting). {% endhint %} ### Files #### Manage requirements For CLI or files users, the packages must be provided in a `requirements.r.txt` file. The format is the same as a regular `requirements.txt` file, except all specs, extras, and comments will be ignored (for now). Technically, notebook users can also do it [by using embedded files](/competitions/participate/notebook-processor#embed-files). #### Executing a file A R script can be run using `robject.r.source(path)`, and the variable will be accessible via the `robject.r[key]` accessor. {% code title="Local File: `hello.R`" %} ```r cat("Hello world from a R file") x <- 42 ``` {% endcode %} {% code title="Local File: `main.py`" %} ```python # Import the R instance from rpy2.robjects import r # Load and execute the R file r.source('hello.R') #> "Hello world from a R file" # Get a value from the scope print(r["x"]) #> "42" ``` {% endcode %} ## Running When a [Run](/other/glossary#run) is started, before using `pip` to install your requirements, R and your requested packages are first installed using `apt-get install r-cran-`. This has the benefit of installing them faster, but it greatly limits the number of choices to only the most downloaded ones (for now). ## Recommendations ### Redirect outputs R's output going through [`rpy2` callback](https://rpy2.github.io/doc/latest/html/callbacks.html#console-i-o) and can be a bit hard to read, a solution is to redirect the outputs: {% code title="Notebook Cell" %} ```python # @crunch/keep:on from rpy2 import rinterface rpy2.rinterface_lib.callbacks.consolewrite_print = lambda x: sys.stdout.write(x) rpy2.rinterface_lib.callbacks.consolewrite_warnerror = lambda x: sys.stderr.write(x) rpy2.rinterface_lib.callbacks.consoleflush = lambda: sys.stdout.flush(); sys.stderr.flush() ``` {% endcode %} [The comment at the very top is important because it ensures that your code is not commented out.](/competitions/participate/notebook-processor#automatic-line-commenting) ### Pandas Conversion [rpy2 supports DataFrame conversion](https://rpy2.github.io/doc/latest/html/pandas.html), but you must use the `with` syntax to enable/disable conversion at the desired location. An easier way is to use the following helpers: {% code title="Notebook Cell" %} ```python import pandas as pd import rpy2.robjects as ro from rpy2.robjects import pandas2ri def convert_df_pandas_to_r(pandas_df: pd.DataFrame) -> "FloatMatrix": """ Convert a pandas's DataFrame to an equivalent R's FloatMatrix. """ with (ro.default_converter + pandas2ri.converter).context(): return ro.conversion.get_conversion().py2rpy(pandas_df) def convert_df_r_to_pandas(r_df: "FloatMatrix") -> pd.DataFrame: """ Convert a R's FloatMatrix to an equivalent pandas's DataFrame. """ with (ro.default_converter + pandas2ri.converter).context(): return ro.conversion.get_conversion().rpy2py(r_df) ``` {% endcode %} # Teams A TLDR about the team feature. ## Overview Teams can be composed of one person, and up to 5 (some competition may allow more). A team can be [created and managed](/competitions/teams/managing#creating-your-team) by a participant. Any decision must be [voted on in a referendum](/competitions/teams/referendums). Team can [invite someone to a team](/competitions/teams/referendums#inviting-a-user), the invitee must then accept the invitation. If a team is looking for teammates, [anyone can ask to join](/competitions/teams/referendums#accepting-a-user). ## Participate You cannot submit as a team directly, each member of a team can only participate individually and [each member will have a ranking on the leaderboard\*](#user-content-fn-1)[^1]. [Rewards will be shared equally among all members.](/competitions/teams/rewards) ## Leaders Some competitions require a leader to represent the team on the leaderboard. By default, this is the user who created the team, [but it can be changed by a vote](/competitions/teams/referendums#promoting-a-leader). All members can submit on the platform, but only the leader will be able to be ranked on the leaderboard and so, be eligible for rewards. The leader can only be changed during the [Submission Phase](/other/glossary#submission-phase), and cannot be changed after that, even if: * The selected [run](/other/glossary#run) is not valid for the [Out-of-Sample](/other/glossary#out-of-sample-phase). * Another member has a better score. *** {% hint style="info" %} To learn more about the team feature, please read the following pages. {% endhint %} [^1]: Only if the competition does not require a team leader. [See Leaders below.](/competitions/teams#leaders) # Managing ## Modifications Modifications can only be done while in a [Submission Phase](/other/glossary#submission-phase). That includes: * [creating a new team](#creating-your-team) * [editing your team](#editing-your-team) * [creating a new referendum](/competitions/teams/referendums) * [voting a referendum](/competitions/teams/referendums#voting) * [accepting an invitation](#team-invitations) ## Creating your team If you see a TEAMS tab in the navigation bar, that means that you can submit as a team.

Team creation form

* The team name must be unique. The format must be a slug: all lowercase, with letters, numbers, and hyphens. * The team description is just an optional text that will be displayed on your team overview. * Looking for teammates allows users to find teammates. Teams can also advertise that they are looking for a teammate. By default, teams do not advertise for new members. ## Team invitations If a team invite you, you can accept or deny the invitation.

Team invitations

You are free to never accept or deny any invitation. {% hint style="info" %} An invitation cannot be canceled by a team. {% endhint %} ## Editing your team Team edition can be done by any member at anytime (only during the [Submission Phase](/other/glossary#submission-phase)).

Team edition form

The fields are the same as for creating a new team. ### Looking for Teammates If a team is looking for teammates, users who are looking for one can ask to join via the "Ask to join" button.
The team receives an [Accepting a user](/competitions/teams/referendums#accepting-a-user) referendum, which the members must accept. # Referendums To ensure that there is no controlling entity in the team, all decisions affecting members must be voted on. 🗳️ ## Unanimity consensus All votes shall be unanimous. At the first `AGAINST`, the referendum is considered as `REJECTED`. Once all members have voted `FOR`, the referendum is considered as `APPROVED`. ## Expiration Polls only have a window of 3 days before they are considered `EXPIRED`.\ \ After that, no new votes will be accepted. ## Referendum Types ### Inviting a user You can invite other users to join your team. Once the decision has been made, the target user will receive an invitation.

Invite a user form

#### Conditions * The target user must not already be invited * The target user must not be in a team * The new team size must be under 5 {% hint style="warning" %} An invitation cannot be cancelled by the team. {% endhint %} ### Kicking a user A user can only be kicked from the team if he or she accepts the kick. This is the only way someone can leave a team. (apart from [disbanding](#disbanding-a-team)).

Kick a member form

#### Conditions * The target user must be a member of the team * The target user must not be the last member of the team (use a [disband](#disbanding-a-team) instead) ### Promoting a leader Only the team leader will have a rank on the leaderboard. Other members can still participate and appear on the leaderboard; they will just not be ranked.

Promote a leader form

#### Conditions * The target user must not be the team leader * The target user must be a member of the team {% hint style="info" %} Not all competitions are using leaders to represent teams. {% endhint %} ### Accepting a user Teams that advertise for new team members can receive requests for people to join their team. When a user makes a request, it triggers a referendum.

View of a user searching for a team

Request to join form

#### Conditions * The target user must not be already invited * The target user must not be in a team * The new team size must be under 5 {% hint style="info" %} Some competitions allow for larger teams. {% endhint %} ### Disbanding a team Teams can be deleted by disbanding them.

Disband a team form

{% hint style="danger" %} **The deletion of a team cannot be undone.** {% endhint %} ## Voting

Voting to invite happy-mike

## Post-vote failure Some referendum can fail to be deployed if some conditions are not true anymore, a few examples includes: * The team is full * An invited user joined another team * An invited user rejected the invitation Depending on the reason, teams can re-create another referendum. {% hint style="info" %} If a team of one member try to create a referendum, it will be automatically accepted. {% endhint %} # Leaderboard ## Competition-level ### Leaderboard The team name will appear next to the usernames on the leaderboard. Members keep their individual submission rank.

Team liverpool

{% hint style="info" %} If the team has a leader, only the leader will have a rank. {% endhint %} ### Statistics The overview now displays your statistics vs your team statistics.

Some dummy competition statistics

* Your personal score * Your team score * Your team rank on the competition-level leaderboard ## Team-level ### Leaderboard Teams also have a dedicated leaderboard.

A dummy team leaderboard

### Statistics Anyone can see a team's statistics.

Some dummy team statistics

* The team score * The team rank on the competition-level leaderboard # Rewards How are rewards distributed within a team? ## Strict Rules The team is rewarded based on the best reward of all its members' models, **divided equally** among the team members. {% hint style="warning" %} No exceptions will be made. If a member is unreachable or refuses his share, **it will be considered as lost and will not be rewarded**. Choose your teammates wisely. {% endhint %} {% hint style="success" %} i.e., in a team of three, each member receives 1/3 of the total reward. {% endhint %} ## Disqualification If a team is suspected of cheating, CrunchDAO may decide to disqualify it. # Leaderboard The only requirement to appear on the leaderboard is to have a successful [Run](/other/glossary#run). Being on the leaderboard makes you eligible for rewards based on the rewards system of the crunch you are participating in. ## Submission Phase During the [Submission Phase](/other/glossary#submission-phase), the leaderboard is updated when a [Run](/other/glossary#run) is scored. If there is no public leaderboard, the leaderboard will be updated when a [Prediction](/other/glossary#prediction) is validated to make sure it is ready for scoring. ## Out-of-Sample Phase During the [Out-of-Sample Phase](/other/glossary#out-of-sample-phase), the leaderboard is only updated once when all the [Run](/other/glossary#run) that are running on unseen data are scored. ## Columns Only a subset of columns will be visible, depending on the current state of the crunch.
NameDescription
RankRank of the participant / model
ParticipantName of the participant / model
ParticipationDistinguish if a participant is ready for the private leaderboard
(only if there is no public leaderboard)
TeamThe participant's team name
(only if the competition allows teams)
Best metric (unit)The best submission score for the metric metric
(reset between each rounds)
Last metric (unit)The last submission score for the metric metric
MeanThe weighted average of the metrics
(only if there are multiple)
metric (unit)The score value of the metric metric
Run SuccessThe number of successful runs / the number of failed runs
Previous LB ChnRank change from the previous round
Public LB ChnRank change from the public leaderboard
Weekly ChangeRank change from the previous week
User Hist. RewardsThe amount of rewards ever received by the participant
Proj. RewardsThe amount of rewards the participant is expected to receive at the end of the month

A small view of the leaderboard, including rank changes and a metric.

### Badges
IconDescription
The participant has won the crunch.
The participant has earned a certificate.
The participant has received a prize.
The participant only submitted a Quickstarter and is disqualified from receiving any rewards.
The prediction is too close to another
(grouped by same user or same team, if correlation > 95%*).
The code is not deterministic and is not eligible for any rewards.
It is tested by running the infer function twice (on the same data) and comparing if the predictions are the same.
The score is not within the allowed range.
Hover over to see which limit(s) the model did not fit.
Only the team leader is considered for ranking on the leaderboard.
## Participant If the crunch allows more than one model, the model name is displayed right after the participant name. Otherwise, only the participant name is displayed. ### Only one Model on the Leaderboard Some competitions (like Mid+One) allow more than one model, but only ONE can be displayed on the leaderboard at any given time. The selection can be made during the [Submission Phase](/other/glossary#submission-phase). After that, the "Use this one" button will no longer be available.

The model will be displayed on the leaderboard and will be eligible for rewards.

The model will NOT be displayed on the leaderboard and will NOT be eligible for rewards.

## Ties Some competitions (such as Broad #2) allow ties when models achieve the same scores, meaning they receive the same rank. The rank assigned to all tied models is the first rank of the tie. The reward is distributed equally among the tied models, calculated as the sum of the rewards for the tied models divided by the number of models in the tie. #### Example For a prize pool that rewards the top 10 models: * If the 1st, 2nd, and 3rd place models are tied, they will all be ranked 1st.\ The next model will be ranked 4th.\ Their total reward will be: (1st place prize + 2nd place prize + 3rd place prize) / 3. * If the 8th, 9th, 10th, 11th and 12th place models are tied, they will all be ranked 8th.\ The next model will be ranked 13th.\ Their total reward will be: (8th place prize + 9th place prize + 10th place prize) / 5. # Duplicate Predictions How is the duplicate badge triggered. Some competitions have a duplicate detection feature that flags all models that have a prediction correlation above 95%[^1] and makes them ineligible for prizes. The correlation is computed using [`pandas.DataFrame.corr(method="spearman")`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.corr.html). ### Grouping The prediction are first grouped between: * user, to avoid duplication between multiple models * team, to avoid duplicate between all models of every team member The correlation function is then called for each prediction against each other. ### Keeping Predictions are then grouped into correlated pairs to isolate them. The best[^2] model in a pair is always kept. Other models are treated as copies. #### Example * If A & B are considered duplicates, and C & D are also considered duplicates, then A and C will be retained and B and D will have the duplicate badge. * However, if B & C are also considered duplicates, then only A will be retained. B, C, D will have the duplicate badge. * In some cases, even if A and D are not considered to be duplicates, because of the link from A to B to C to D, the duplicate badge is still displayed. [^1]: This threshold can be changed at any time during the competition and at the discretion of the organizers. [^2]: Old competition used the first model created. # FAQs Frequently Asked Questions ## Can I train a model locally? Yes, but you should still include the code in case we need to rerun your work. We understand that training some models requires significant resources and computing time, which cannot easily fit within the weekly quota. Some models even require a network connection that is not available in the cloud environment. For these models, we recommend training locally but including the training code in a separate file or commenting it out. If you do not submit your training code, you [might miss the opportunity](#user-content-fn-1)[^1] to retrain with more data during the [Out-of-Sample Phase](/other/glossary#out-of-sample-phase), as we have done in past competitions. ## `SCIPY_ARRAY_API` As this flag introduces changes that could be changed in the future and was only intended as a [temporary measure](https://docs.scipy.org/doc/scipy/dev/api-dev/array_api.html), you need to enable it manually. To achieve this, insert this cell before the one that imports SciPy for the first time. {% code title="Python Notebook Cell" expandable="true" %} ```python import os os.environ["SCIPY_ARRAY_API"] = "1" ``` {% endcode %} ## How can I get the output of a Run? Files generated in the cloud environment (e.g., trained models) cannot be extracted or downloaded locally. This ensures that data accessible only there cannot be exfiltrated. ## I get a `NameError` on my constant If you receive the following error: 
NameError: name ‘MY_CONSTANT’ is not defined
This happened because your constant was commented out during the submission process. You can avoid this by doing one of the following: * Use the `# @crunch/keep:on` and `# @crunch/keep:off` commands to wrap the declaration of your constant: {% code title="Python Notebook Cell" %} ```python # @crunch/keep:on MY_CONSTANT = true # @crunch/keep:off if MY_CONSTANT: ... ``` {% endcode %} * Move your constants into a dedicated cell, then place `# @crunch/keep:on` at the very top: {% code title="Python Notebook Cell" %} ```python # @crunch/keep:on MY_CONSTANT = true ``` {% endcode %} {% code title="Python Notebook Cell" %} ```python if MY_CONSTANT: ... ``` {% endcode %} * Declare them in a class: {% code title="Python Notebook Cell" %} ```python class Constants: MY_CONSTANT = true if Constants.MY_CONSTANT: ... ``` {% endcode %} For more details, see [the Notebook Processor page](/competitions/participate/notebook-processor#automatic-line-commenting). [^1]: They can be decided late in the competition. # Prize Winners ## I just received a prize, what should I do? Congratulations on the good ranking! 🎉 Your prize will be transferred to your Solana wallet, which is managed by [Privy.io](https://privy.io/), our authentication provider. You are the only one who can access it, and CrunchDAO will cover all transfer fees when you need to move your prize elsewhere. All transactions are now processed through [the Solana network](https://solana.com/). **This is a significant change because wallets are incompatible with the Ethereum network**. We had to leave Ethereum due to its high barriers to entry and high fees. ### KYC If you took part in a major competition such as the Structural Break, you will receive an email with instructions on how to complete the KYC process. This is necessary in order for you to receive the funds. It helps us combat individuals with multiple accounts who try to exploit our system. Refusing to comply is the same as refusing your prize. ### Managed Wallets CrunchDAO uses [Privy.io](https://www.privy.io/) to provide users with a wallet that is ready to use. Users can receive prizes in their wallet and transfer them out later when they are ready. You manage this wallet [via the Hub](https://hub.crunchdao.com/account/wallet), where you can: * Check your current balance and transfer tokens elsewhere. * View your transaction history.\ This logs all your interactions with real-time competitions. * Export the private key.\ The first time you do it, you will receive an email indicating that the wallet is now considered unsafe. If you did not take this action, please [contact us](/competitions/faqs/contact-us#help-with-the-crytocurrencies) as soon as possible! {% hint style="info" icon="lock" %} To secure this wallet, we strongly suggest that you [enable 2FA on your account](https://hub.crunchdao.com/account/security). This step is necessary for exporting the private key. {% endhint %} #### How to withdraw the funds? To withdraw funds from your Crunch account, click the Send button for the currency you just received.
Then, you will be able to specify the destination address to which they should be sent. The last step is to ensure that you have accepted all of the disclaimers. **Once a transaction has been sent, ****it cannot be reversed!**
{% hint style="warning" %} If you're unsure about what you're doing, **we highly recommend performing a test transaction of 1 unit.** {% endhint %} ### Centralized Exchange Wallets To withdraw funds more quickly into a fiat currency, select the correct network and click the **`Receive USDC`** button on your exchange. This will ensure the funds appear directly in your account, so you won't need to cover the transfer fees. {% hint style="warning" %} Make absolutely sure that you **can convert the 1 UDSC test back to fiat first**, as some exchanges have a minimum amount policy (e.g., [Kraken](https://support.kraken.com/articles/360000292886-cryptocurrency-deposit-fees-and-minimums)). Once we send the full amount, you are responsible for the tokens. {% endhint %} {% hint style="info" %} If your exchange asks you about the origin of the tokens, tell them the following: * Company Name: Crunch Lab, Inc. * Exchange Name: Bitstamp. If they request more information, don't hesitate to [contact us](/competitions/faqs/contact-us#help-with-the-crytocurrencies). {% endhint %} ## Why cryptocurrencies? To eliminate the need for international transfers, all prizes are paid in cryptocurrencies. This rule will not be waived under any circumstances, including requests for a bank transfer or equivalent. This also ensures that transactions can be traced on the blockchain. ## Be on alert for Scams! To keep your assets safe, always verify the source of any communication that claims to represent us. Verify email addresses and social media profiles. Remember that we will never ask for your private keys or passwords. There is a 100% chance that the following is a scam: * If you need to complete any KYC procedures or validation for your wallet! **That is a scam!** * If you need to have any minimal amount to receive our prize. **That is a scam!** * If we promise an airdrop in private/direct messages. **That is a scam!** If you have any doubts, [contact us directly](/competitions/faqs/contact-us#help-with-the-crytocurrencies)! # Contact us There are multiple ways to contact us. Depending on the subject, please use the most efficient method for the fastest response. ## Help with the Hub/Competition If you need help with a submission, run, or quota, or if you find a bug on the platform, we suggest trying the following: * [The Discord server](https://discord.gg/veAtzsYn3M) * Please use the correct competition channel whenever possible. * Or create a ticket by using the `#ticket-new` channel. * [The Community Forum](https://forum.crunchdao.com/) * Please use the correct competition subcategory whenever possible. ## Help with the KYC process To avoid leaking any personal information, please try the following: * [Create a ticket on via the `#ticket-new` channel on the Discord server](https://discord.gg/veAtzsYn3M) * [Email us at `crew@crunchdao.com`](mailto:crew@crunchdao.com) ## Help with the crytocurrencies If the exchange requests more information, please try the following: * [Create a ticket on via the `#ticket-new` channel on the Discord server](https://discord.gg/veAtzsYn3M) * [Email us at `crew@crunchdao.com`](mailto:crew@crunchdao.com) * 🚨 **Stay Safe From Scammers** 🚨\ Your security is our top priority. Please remember: * We will **never** ask for your seed phrase, private keys, or wallet password. * We will **never** DM you first. Always verify before responding. * Beware of fake accounts, airdrop promises, or “support” messages. * Only trust information shared through our **official channels**. 👉 If something feels suspicious, assume it’s a scam and double-check with the team ## Help with Legal If you have a legal inquiry, try the following: * [Email us at jean.herelle@crunchdao.com](mailto:jean.herelle@crunchdao.com) # Known Issues A list of known problem and how to solve them. ## Terminated Run never finishes Depending on the timing, a run may become unresponsive to further action and get stuck in an infinite "running" state. If that happens, please contact an administrator on [Discord](https://discord.com/invite/veAtzsYn3M) or the [Forum](https://forum.crunchdao.com/). ## `SCIPY_ARRAY_API` before importing `sklearn` or `scipy` Since the [SciPy array API](https://docs.scipy.org/doc/scipy/dev/api-dev/array_api.html) is experimental, models that need it must enable it. In a **dedicated cell** at the **very top**—even **before the other imports**—and **without changing anything**, write the following: {% code title="Notebook Cell (Python)" %} ```python # @crunch/keep:on import os os.environ['SCIPY_ARRAY_API'] = '1' ``` {% endcode %} ## `NaN` in the Cloud, but none locally The `index` might not always be the same in the Cloud and locally as the data that is shared **is** different. Depending on your code, this could cause some issues when trying to do operation between `pandas` objects that do not share the same `index`. {% code title="Notebook Cell (Python)" %} ```python # will use a range index, from 0 to len(X_test) final_ensemble = pd.Series( [0] * len(X_test), dtype='float' ) # pandas objects do not share the same index, it will likely result in only nans final_ensemble += (X_test.loc[:, 'some_colomn'] * 2) ``` {% endcode %} ### Use the same index {% code title="Notebook Cell (Python)" %} ```python final_ensemble = pd.Series( [0] * len(X_test), index=X_test.index, dtype='float', ) ``` {% endcode %} ### Reset the index {% code title="Notebook Cell (Python)" %} ```python X_test.reset_index(inplace=True) # then do your operations final_ensemble += (X_test.loc[:, 'some_colomn'] * 2) ``` {% endcode %} ## Other If your problem is not listed, don't hesitate to reach the team! # Tournament API How to use the API to interact with the website. ## Endpoints Most endpoints are documented in the Swagger UI, which is available here: {% embed url="" %} ## Python Client The crunch-cli offer an API client to easily use do action on the platform: {% embed url="" %} ### Usage The following code export the all the leaderboards from the ADIA Lab Structural Break competition: {% code title="Python Code" %} ```python import crunch client = crunch.api.Client.from_env() competition = client.competitions.get("structural-break") round_ = competition.rounds.get(1) phase = round_.phases.submission for crunch_ in phase.crunches: if not crunch_.published: continue df = competition.leaderboards.get_default(crunch=crunch_).as_dataframe() df.to_csv(f"{crunch_.number}.csv", index=False) ``` {% endcode %} {% hint style="info" %} The `Client.from_env()` function will search for the `CRUNCH_API_KEY` environment variable. {% endhint %} ### Features * Fluent syntax * Frequently used routines * Typing * Maintained by us and always up-to-date * Error classes, for easier try-except ## Authentication There are multiple ways to authenticate:
Token TypeHeaderQuery Parameter
API KeyAuthorization: API-Key <token> ?apiKey=<token>
Access TokenAuthorization: Bearer <token> ?accessToken=<token>
You can generate an API-Key in the [API Management section of your account](https://hub.crunchdao.com/account/api). ## Error Any message that does not return a `2XX` or `3XX` error code is considered an error. All errors are formatted in the following way: {% code title="API Error Response" %} ```json { // A unique code, always in UPPER AND SNAKE_CASE "code": "SUBMISSION_NOT_FOUND", // A message, that often, contains more details "message": "submission not found", // More properties that provide additional context "submissionNumber": 123, "projectName": "my-model" } ``` {% endcode %} # Your Wallet (MetaMask) ## Installation To interact with the $CRUNCH token, the easiest way is to use [MetaMask](https://metamask.io/). To install your wallet simply follow the instruction from their [website](https://metamask.io/). ## Interaction ### CrunchDAO Tournament Click on the button `CONNECT WITH METAMASK` available on your [profile](https://tournament.datacrunch.com/profile). ![](/files/-MlFnuCYTS5qmNzpCNdy) You will be prompted to accept the connection. You must click `Next`. Then `Connect`. Now you should be able to see your $CRUNCH balance. (clicking on it will refresh it) ![](/files/-MlFoyiP7qUbe-L-yrCQ) # Adding the Token Since the $CRUNCH Token is an ERC20 Token, it must be added manually to MetaMask. ### Through the website A button will allow you to add the $CRUNCH Token. Go to your [profile](https://tournament.datacrunch.com/profile), and click `ADD TOKEN TO METAMASK`. ![](/files/-MlFru-MFbKVZSiCWuE0) A confirmation window will pop-up, just click `Add Token`. Done! You successfully added the $CRUNCH Token to your MetaMask wallet. ### Manually You must open MetaMask, and then click on the `Add Token` button. ![](/files/-MlFpbNjL9PvFOJircWP) Then select `Custom Token`. ![](/files/-MlFprnRmP8-FA8P_dNL) And paste the token's address: [0x74451D2240Ef9e86b3cEA815378aF61566B81856](https://etherscan.io/address/0x74451d2240ef9e86b3cea815378af61566b81856) You should be able to see `CRUNCH` as the symbol. ![](/files/-MlFqqbYkJfK6rGWco8a) Then click `Next`. ![](/files/-MlFrEGQyqi91Fljasti) A confirmation screen will appear, click `Add Token`. ![](/files/-MlFrMtt7W2e1MnbHl8Z) Congratulations! You successfully added the $CRUNCH Token to your MetaMask. ![](/files/-MlFra5xEoyqtB6Wb3wB) # Release Map ## 2022-07-25 ### CrunchMultiVestingV2 * Deployed Multi Vesting V2 smart contract. [(code)](https://github.com/crunchdao/contracts/blob/master/contracts/CrunchMultiVestingV2.sol) [(address)](https://etherscan.io/address/0xf3b262b8623aa8eaf302bd46a393179df0ed13c5) ## 2022-02-18 ### CrunchMultiVesting * Deployed Multi Vesting smart contract. [(code)](https://github.com/datacrunch-com/datacrunch-contracts/blob/master/contracts/CrunchMultiVesting.sol) [(address)](https://etherscan.io/address/0xe469f12f4746b5ae105a1b888bff5a1b9e27fee5) ## 2022-01-24 ### CrunchSelling * Deployed Selling smart contract. [(code)](https://github.com/datacrunch-com/datacrunch-contracts/blob/master/contracts/CrunchSelling.sol) [(address)](https://etherscan.io/address/0x22525935cb0f5c27ae025fe5a403bc7a0eb9c857) ## 2021-10-19 ### CrunchTimelock * Deployed Timelock smart contracts. [(code)](https://github.com/datacrunch-com/datacrunch/blob/master/contracts/CrunchTimelock.sol) [(factory address)](https://etherscan.io/address/0xB1C77D4d05e16913b19822dF99eacAFFe3387C1a) ## 2021-09-22 ### CrunchStaking * Deployed the Staking smart contract. [(code)](https://github.com/datacrunch-com/datacrunch/blob/master/contracts/CrunchStaking.sol) [(address)](https://etherscan.io/address/0xFb99073EAA547dC965Ad75420128093A01128EC2) ## 2021-09-20 ### CrunchReward * Deployed the Reward smart contract. [(code)](https://github.com/datacrunch-com/datacrunch/blob/master/contracts/CrunchReward.sol) [(address)](https://etherscan.io/address/0xa3B20d15649B03f38Ab71d64f0f5Fcb3ac48c3f4) ## 2021-09-13 ### CrunchVesting * Deployed DataCrunch's Team Vesting smart contracts. [(code)](https://github.com/datacrunch-com/datacrunch/blob/master/contracts/CrunchVesting.sol) [(factory address)](https://etherscan.io/address/0xb47fa11d32d1005babc52e876bcfa6bfa19e490b) ## 2021-09-05 ### CrunchAirdrop * Deployed the airdrop smart contract. [(code)](https://github.com/datacrunch-com/datacrunch/blob/master/contracts/CrunchAirdrop.sol) [(address)](https://etherscan.io/address/0xed28c62F1df817C23adbE577f406C1386cb61e8F) ### CrunchToken * Deployed the CRUNCH Token. [(code)](https://github.com/datacrunch-com/datacrunch/blob/master/contracts/CrunchToken.sol) [(address)](https://etherscan.io/token/0x74451d2240ef9e86b3cea815378af61566b81856) # Glossary Technical definitions of platform concepts. ## Competition A structured, (often) time-bound, incentive-driven challenge where participants (typically data scientists) compete to solve a specific problem or achieve a defined objective using data, algorithms, or machine learning models. ## Crunch A period during which the leaderboard can be updated and compute quotas can be refreshed. This period usually lasts for a week, but it can be arbitrary. {% hint style="info" %} Crunch is the technical term. It is also used by marketers to refer to a [Competition](#competition). {% endhint %} ## Submission Phase Part of the competition while submission is allowed. The data does not change and the quota is regularly reset. It is also known as the public leaderboard. ## Out-of-Sample Phase Part of the competition while scoring is happening on an out-of-sample phase. The data does change regularly, and no modification is allowed to your code or selection. It is also known as the private leaderboard. ## Submission Similar to a Git commit, a submission represents a frozen version of a user's code. Once on the platform, no file can be updated and the message cannot be changed. ## Run Similar to GitHub Actions, a run is when a user's code is executed in the cloud environment. The environment is heavily restricted. The user's code must follow the [Code Interface](/competitions/participate/code-interface) in order to run properly. ## Prediction A prediction is the output of a [Run](#run). # Credits ## Profile Pictures The default avatars come from NASA's [Astronomy Picture of the Day Archive](https://apod.nasa.gov/apod/archivepix.html). All of the credit goes to the original artists of the pictures.