NeurIPS 2026 Competition

AIMO Interpretability Challenge

Can your method tell when AI genuinely solves the problem and when it is faking it?
Distinguish robust from spurious reasoning in frontier AI.

Starting mid August 2026 -- Join our discord or register for updates!
💰 $12,500 in prizes 📐 Unseen olympiad problems 🧠 Frontier reasoning models from AIMO 3
▷ Join the Challenge 📄 NeurIPS 2026 Proposal How to participate
Background

Why Interpretability for AIMO?

AIMO (AI Mathematical Olympiad) is an annual competition of AI systems in solving unseen frontier mathematical problems. Thanks to its importance and large prizes ($2.2M USD this year), AIMO attracts huge community attention — last year, the competition was covered by global media The Wall Street Journal and Bloomberg, and participated in by over 4,000+ teams.

The ambition that we have in the Fields Model Initiative is to turn this tremendous engineering effort into scientific knowledge — laying robust stepping stones that will accelerate progress in mathematical reasoning and in AI as a whole.

We believe that interpretability research can play a key role in helping us achieve this goal — allowing us to understand the mechanisms of reasoning in SOTA AI systems and providing actionable knowledge for their future, open advancements.

More broadly, the competition connects interpretability and generalization research around one of the most important questions in AI today: to what extent is the decision-making of frontier AI models generalizable and thus, reliable?

The Challenge

Robust or Spurious?

The AIMO Interpretability Challenge requires participants to submit a system that decides, for a single problem–model pair, whether that model answers that problem robustly. The full problem formulation, data collection, and evaluation protocol are described in our competition proposal paper.

One problem, one model, one verdict An unseen olympiad problem and one AIMO 3 model are passed to the participant's system, which returns a single verdict: robust or spurious. PROBLEM Olympiad problem unseen, from AIMO 3 MODEL One AIMO 3 model weights · activations · CoT Your system white-box or black-box Is this model robust on this problem? ✓ Robust ⚠ Spurious one binary verdict
One problem, one model, one verdict An unseen olympiad problem and one AIMO 3 model are passed to the participant's system, which returns a single verdict: robust or spurious. PROBLEM Olympiad problem unseen, from AIMO 3 MODEL One AIMO 3 model weights · activations · CoT Your system white-box or black-box Is this model robust on this problem? ✓ Robust ⚠ Spurious one binary verdict

What do we mean by robust or spurious?

🛡️ Robust

The model's answer holds up under perturbations that leave the mathematical content of the problem unchanged.

⚠️ Spurious

Counterfactual evaluation with perturbations reveals that the model's answer relies on at least one spurious pattern.

Competition submissions take the form of Codabench submission bundles that comply with the unified interface defined in the getting-started repository, and are evaluated on our servers without internet access. Methods are ranked by their accuracy in classifying whether a given model answers a given problem robustly. The provided validation set covers all types of models contained in the test set.

Submission Types

🏆

Main Track

Includes the full scale of top-performing models from AIMO 3 — no restrictions on model size.

🔬

Small Models Track

Subsets the evaluation to the best-performing models below the 10-billion-parameter scale, providing a comparable setup for compute-heavy methods such as Sparse Autoencoders or Transcoders.

Prizes

🏆 Main Track

  • 🥇 1st place $3,000
  • 🥈 2nd place $1,000
  • 🥉 3rd place $600

🔬 Small Models Track

  • 🥇 1st place $2,000
  • 🥈 2nd place $600
  • 🥉 3rd place $300

Ties in the ranking are broken by efficiency: preference is given to the submission with the faster average runtime over three repeated trials.

✍ Writeup Prizes

$5,000 will be awarded for selected technical reports, regardless of their ranking, judged on their scientific contribution. We are particularly interested in work on generalization of interpretability methods, actionable interpretability, negative results or efficient methods.

Competition Rules

Submissions must follow these rules, designed to prevent overfitting and keep the competition fair:

  1. Data: teams may not use extra labeled data for the same perturbation types as in our final validation set; however, unlabeled data and labeled data for other perturbations are allowed, within the bulk evaluation limit of 20min/sample. Any attempts for leakage of evaluation data will result in the disqualification.
  2. Signals: teams can use weights, activations, token probabilities, or any other model internal signals as inputs for their methods. While we encourage the development and submission of white-box methods, black-box statistics and behavioural analyses (eg based on models’ reasoning chains) are also permitted.
  3. Tracks: teams may submit to the Main or Small models track, but each submission will count only for the chosen submission track. Teams may choose to submit the same submission to both (Main and Small Models) tracks.
  4. Report: teams considered for awards must submit a technical report with details on their submission and its open-sourced implementation within two weeks of the final deadline (by November 15, 2026) for compliance checks.
  5. Reviewing: teams considered for awards commit to peer-review up to 3 technical reports of other teams during the reviewing period (November 15–30, 2026).

How to Participate

1️⃣

Register or Join our Discord for Updates

Fill in the registration form for email updates, or join our Discord channel.

2️⃣

Make your first submission

Clone the getting-started repository and submit one of its example solutions as-is to see your submission in the Codabench leaderboard.

3️⃣

Implement and package your new shiny method

Wrap your method in the Codabench interface specified in the getting-started repository, and draw on the reference baselines for worked implementations. No internet access is available at evaluation time.

4️⃣

Compete for the prizes

Iterate on the leaderboard until the final deadline on November 1, 2026 — with $12,500 in prizes across the Main and Small Models tracks.

Need compute support? Submit a brief proposal in the Fields Model Initiative to request access to compute for your participation.

Timeline

Key Dates

Early Aug: competition start & warmup phase
Competition start: release of train+validation data and baseline submission bundles, two-week warm-up phase
Aug 15 – Nov 1, 2026
Main competition phase
Nov 1, 2026
Final submission deadline — closing of the submission interface
Nov 1 – Nov 15, 2026
Participant technical report submission window
November 15 – November 30, 2026
Validation of results, technical reports review and rules compliance check, result analysis
December 11/12, 2026
Competition workshop at NeurIPS 2026
FAQ

Frequently Asked Questions

1. Who can participate?

The challenge is open to everyone — academic researchers, independent researchers, and industry practitioners alike. There are no restrictions on team size, affiliation, or career level. Students on all education levels are welcome!

2. Do I need to be a participant of AIMO to join?

No. The AIMO Interpretability Challenge is a separate competition. You will be given access to AIMO model submissions as part of the provided environment — you do not need to submit to AIMO yourself.

3. What is the difference between the Main Track and the Small Models Track?

The Main Track covers the full scale of top-performing models from AIMO 3 with no restriction on model size. The Small Models Track subsets the evaluation to the best-performing models below the 10-billion-parameter scale, offering a comparable setup for methods that are more compute-heavy to train — such as Sparse Autoencoders or Transcoders — where analysing very large models may be infeasible.

4. What runtime environment does my submission need?

Submissions must be packaged as Codabench submission bundle and must conform to the unified interface defined in the getting-started repository. Containers are evaluated on our servers without internet access. Detailed technical documentation and a starter kit are available in the getting-started repository.

5. Will there be tutorial material and a starter kit?

Yes. A starter kit with worked examples and full documentation is available in the getting-started repository, with baseline implementations in the baselines repository. The problem formulation and evaluation methodology are described in our competition proposal paper, available now on arXiv.

6. Can I participate in both tracks?

Yes, teams may submit to both the Main Track and the Small Models Track independently.

7. How is the winner determined?

Submissions are ranked by accuracy in classifying each problem–model instance as robust or spurious on the held-out test set. The Main Track and Small Models Track are ranked separately.

8. What if I have more questions?

Reach out to us at aimo-interp@gmail.com or open an issue in the getting-started repository. We aim to respond within 48 hours and will update this FAQ regularly.

Team

List of Organizers

The organizing team combines expertise in AI evaluation, interpretability, and data collection with top-tier olympiad-level mathematicians — a combination that lets us curate the high-quality robustness validation this challenge is built on. Several of us are actively involved in related initiatives, including AIMO, bringing first-hand experience running competitions at scale.

Michal ŠtefánikMŠ
Michal Štefánik
NII, Japan
Lead organizer, website, communication. Researcher in evaluation, interpretability, and robustness of language models.
Philipp MondorfPM
Philipp Mondorf
LMU Munich
Core organizer, baselines, leaderboard. PhD researcher in reasoning and generalization of language models.
Andreas WaldisAW
Andreas Waldis
Univ. of Tübingen
Core organizer, baselines, beta testing. PostDoc and junior group lead researching the reliability and robustness of language models.
Chuan YangCY
Chuan Yang
Fuzhou University
Math expertise, problems validation. Associate Professor of mathematics and optimization, contributing olympiad-level math expertise.
Róbert BelanecRB
Róbert Belanec
KInIT, Slovakia
Data curation, robustness labels. PhD researcher in efficient fine-tuning and generalization of language models.
Qianying LiuQL
Qianying Liu
NII, Japan
Math expertise, evaluation. Researcher in the multilingual reasoning and interpretability of language models.
Michal SpiegelMS
Michal Spiegel
Masaryk University
Data curation coordination. Researcher in mechanistic interpretability of reasoning and length generalization.
Josef KuchařJK
Josef Kuchař
Masaryk University
Infrastructure, Codabench. Software engineer focused on the development of machine learning systems.
Marek KadlčíkMK
Marek Kadlčík
Masaryk University
Symbolic chains, permutations. PhD researcher in the reasoning and generalization of neural models.
Adam Vawda-OomerjeeAV
Adam Vawda-Oomerjee
NII / UCL
Infrastructure, Codabench, beta testing. Researcher in reasoning, reinforcement learning, and world models.
Chaoran LiuCL
Chaoran Liu
NII, Japan
Infrastructure, participant support. Project Associate Professor researching architectures and training strategies of large language models.
Simon FriederSF
Simon Frieder
Oxford / AIMO
Competition design, AIMO liaison. AIMO Prize Manager and founder of the Benchmarks & Baselines non-profit, active leader in AI for Math community.
Barbara PlankBP
Barbara Plank
LMU Munich
Evaluation, data. Professor leading the MaiNLP lab and President of the Association for Computational Linguistics (ACL), focused on evaluation and reliability of language models.
Fazl BarezFB
Fazl Barez
Univ. of Oxford
Interpretability methods. Senior Researcher leading the Technical Safety & Governance lab, bringing expertise in interpretability.
Pontus StenetorpPS
Pontus Stenetorp
UCL / NII
Evaluation, adversarial data, promotion. Professor of NLP and Deputy Director of the UCL Centre for AI, bringing expertise in data and evaluation.
Contact

Get in Touch

For any questions about the challenge, compute access proposals, or technical issues, please reach out — we're happy to help.

✉️

Primary contacts: aimo-interp@gmail.com and the AIMO-Interp Discord.

We aim to respond within 48 hours. For technical questions, please also consider opening an issue or discussion on the corresponding GitHub repository. This FAQ is regularly updated.