Overview
FrontierCode is a benchmark released by Cognition on 2026-06-08, designed to evaluate model performance under the standards of high-quality production codebases. It no longer focuses solely on code correctness but measures code mergeability, quality, test quality, scope discipline, style, and adherence to codebase standards. The benchmark was co-designed by over 20 open-source maintainers, employing an innovative evaluation method consisting of unit tests, scoring criteria, and a new type of validator. Cognition positions FrontierCode as a benchmark upgrade from 'correctness' to 'quality'.
On 2026-10-12, Artificial Analysis launched FrontierCode 1.1 as a standalone leaderboard. This version was built by Cognition with maintainers of 36 open source repositories, who spent more than 40 hours per task authoring grading criteria covering correctness, test quality, scope discipline, style and adherence to codebase standards; the tasks are private to avoid contamination. Artificial Analysis runs the 100 tasks of FrontierCode 1.1 Main in Stirrup, its open source agent harness, capped at 250 turns per attempt, with three attempts per task. Claude Opus 5.5 (max with fallback) leads with an average score of 52% and a pass rate of 58%, followed by GPT-6.1 Sol (max) and Claude Fable 5.1 (max with fallback) at 49%. The leaderboard notes nine models from six labs sit between 43% and 52%, that MiMo-V2.6-Pro leads open weights models at 45%, and that 9 of the 100 tasks are passed by no model on any attempt. FrontierCode 1.1 is published as a standalone leaderboard and is not yet part of the Artificial Analysis Intelligence Index.
Key Features
- Code Mergeability Evaluation: The first benchmark to measure whether model code would be merged by maintainers, assessing end-to-end code quality
- Multi-Dimensional Quality Metrics: Covers correctness, test quality, scope discipline, style, and adherence to codebase standards
- Innovative Evaluation Method: Employs an evaluation method consisting of unit tests, scoring criteria, and a new type of validator
- Designed by Open-Source Maintainers: Co-designed by over 20 open-source maintainers to ensure evaluation standards closely reflect real production environments; FrontierCode 1.1 was built with maintainers of 36 open source repositories
- Standalone Leaderboard on Artificial Analysis: Launched on 2026-10-12 as a standalone leaderboard, running the 100 tasks of FrontierCode 1.1 Main in Stirrup, Artificial Analysis's open source agent harness, capped at 250 turns per attempt with three attempts per task
Use Cases
- Evaluate AI model code quality in real production codebases
- Compare model performance in terms of code mergeability
- Drive AI code generation from 'correct' to 'high-quality'
- Provide a reference standard for model code quality to open-source project maintainers
- Track model rankings on the FrontierCode 1.1 standalone leaderboard
Pros
- First benchmark focusing on code mergeability, with a unique positioning
- Designed by open-source maintainers, with evaluation standards close to real production environments
- Multi-dimensional quality metrics, surpassing traditional correctness evaluation
- Innovative evaluation method combining unit tests, scoring criteria, and validators
- FrontierCode 1.1 tasks are private to avoid contamination, with grading criteria authored over more than 40 hours per task
Pricing
FrontierCode is a public benchmark, free to use. Users can view leaderboards and evaluation results on the official website without subscription or payment.
Summary
FrontierCode is primarily aimed at AI model developers, researchers, and open-source maintainers, used to evaluate and compare model performance under the standards of high-quality production codebases. It upgrades from 'correctness' to 'quality', setting a new evaluation benchmark for AI code generation. Individual developers or teams can view leaderboards on the official website to understand model performance in dimensions such as code mergeability and test quality. As of 2026-10-12, FrontierCode 1.1 is available as a standalone leaderboard on Artificial Analysis, with Claude Opus 5.5 leading at a 52% average score.
Version History
- FrontierCode 1.1 launches as a standalone Artificial Analysis leaderboard, Claude Opus 5.5 leads at 52% (2026-10-12): On October 12, 2026, Artificial Analysis launched FrontierCode 1.1 as a standalone leaderboard. The benchmark was built by Cognition with maintainers of 36 open source repositories, who spent more than 40 hours per task authoring grading criteria covering correctness, test quality, scope discipline, style and adherence to codebase standards; the tasks are private to avoid contamination. Artificial Analysis runs the 100 tasks of FrontierCode 1.1 Main in Stirrup, its open source agent harness, capped at 250 turns per attempt, with three attempts per task. Claude Opus 5.5 (max with fallback) leads with an average score of 52% and a pass rate of 58%, followed by GPT-6.1 Sol (max) and Claude Fable 5.1 (max with fallback) at 49%. The leaderboard notes nine models from six labs sit between 43% and 52%, that MiMo-V2.6-Pro leads open weights models at 45%, and that 9 of the 100 tasks are passed by no model on any attempt. FrontierCode 1.1 is published as a standalone leaderboard and is not yet part of the Artificial Analysis Intelligence Index.