BenchmarkList

BenchmarkList is a reference site for tracking how AI models perform across a comprehensive range of benchmarks. It aggregates evaluation results from coding challenges, reasoning tests, agent evaluations, creative tasks, games, safety assessments, and multimodal benchmarks, then presents them in leaderboards with source linked provenance. If you're trying to understand how GPT 4, Claude, Gemini, or any of hundreds of other models compare on a specific test, this is where you go to find that data consolidated in one place rather than scattered across papers, blog posts, and announcement threads that you'd have to hunt down individually.

The core feature is the AI Capability Index, an experimental scoring system that ranks 867 models across 178 benchmarks using over 11,000 data points. The methodology addresses a fundamental problem in model comparison that anyone who has tried to do this seriously has run into. Different benchmarks use different scales, test different things, and were run by different teams at different times under different conditions. Comparing raw scores directly doesn't make sense when one benchmark reports percentages and another reports accuracy on a specific subset of questions. The Capability Index normalizes results to a common 0 to 1 scale, fits model capability and benchmark difficulty values jointly, equalizes benchmark influence so a few popular tests don't dominate the overall score, and anchors everything to readable numbers with GPT 4o set at 100. The result is a single capability score you can use to compare models that were never actually tested on the same benchmark directly.

Beyond the aggregate index, the site hosts individual benchmark leaderboards for specific evaluations. You can drill into FrontierCode, which measures whether model generated pull requests meet the standards of human code reviewers and maintainers, or TAU3 Bench, which tests multi turn agent interactions with simulated users across domains like airline booking, banking, retail, and telecom. Each benchmark page shows model rankings, score distributions, domain breakdowns where applicable, and links to the original sources so you can verify any number yourself and see the methodology behind it. The source linking is essential for credibility. Without it, benchmark aggregation becomes a trust exercise, and the whole point of this kind of resource is to make claims traceable so you don't have to take anyone's word for it.

The data visualization goes beyond simple ranking tables. There's a score versus coverage scatter plot that shows how performance relates to benchmark breadth, helping you identify models that score well on a narrow set of tests versus those with consistent performance across many different evaluations. A price performance analysis identifies the Pareto frontier of capability per dollar, which is useful if you're trying to pick a model that balances cost and quality for a production use case rather than just chasing the highest score regardless of price. The hardest benchmarks ranking by difficulty score shows which evaluations are actually differentiating at the frontier, and a largest residuals table highlights cases where a model's performance on a specific benchmark deviates most from its overall capability score, revealing strengths or weaknesses that get hidden when you only look at aggregate numbers.

BenchmarkList also offers custom benchmark services for companies and investors who need private evaluations that go beyond what's publicly available. If the public benchmarks don't cover the specific capability you care about, or if you need to run a proprietary model through a standardized test before making a purchasing or partnership decision, they'll design and run evaluations tailored to your requirements. This positions the site as both a free public resource for the broader community and a consulting service for organizations doing serious model selection work where the stakes are higher.

The target audience includes AI researchers tracking the state of the field, developers choosing which model to integrate into a product, and decision makers evaluating vendors or comparing options for enterprise deployments. The site is free to browse, with an email subscription option for updates when new models or benchmarks are added. It fills a gap between scattered academic papers, marketing claims from model providers that cherry pick favorable results, and the handful of community maintained leaderboards that only cover specific narrow domains. Having a single, source linked resource that spans the full landscape of benchmarks makes due diligence faster and reduces the risk of making decisions based on incomplete or cherry picked comparisons.