Referee.Chat

Referee.Chat is a web app built on a simple frustration, which is that a single AI chat will confidently declare a problem solved whether or not it actually is. Here you set a goal, define the standard it has to meet, and then a panel of up to 6 AI models works through the problem together. A separate referee model, running on Claude Opus 4.5 or GPT-5, rules on each acceptance criterion individually, and it only accepts a criterion when there's verifiable evidence behind it. Opinions alone can't settle anything.

The evidence requirement has teeth. The system searches academic sources including arXiv, OpenAlex, Crossref, and Europe PMC, so claims get tied to sources that can be re-checked rather than to a model's recollection. It can also run Python code to verify computational claims directly, which turns arguments about whether a calculation holds into an executed check. When the panel gets stuck, the system hands the specific blocking question to the model best equipped for it instead of letting the whole panel thrash in circles.

Work is organized as matches, and a match behaves more like a project than a chat session. Matches pause and resume, and established findings get saved into a shared ledger the panel builds on rather than re-litigating. Costs are billed per token for the models you choose and scale with the size of that ledger, not with how many rounds the panel takes. You set a spending limit per match, and hitting it pauses the match rather than failing it, so you can review what's been established and raise the limit if the work justifies it. That's a meaningful safety property for long-running problems, because nothing you've paid for gets thrown away when the budget runs out, the work simply waits for you to come back and decide whether it's worth continuing.

How hard the referee pushes is up to you. Quality bars range from Careful Expert to Publication Standard, Formal Proof, and Prize Standard, so the same machinery can sanity-check an analysis or grind on something you'd want to defend in front of reviewers. The model roster spans Anthropic, OpenAI, Google, Mistral, DeepSeek, and others, so the panel isn't a single vendor debating itself, and you choose which models sit on the panel for a given match. There's also a documented API alongside the web interface, so the goal, panel, and referee process can be wired into your own workflow rather than lived in through the browser.

The division of labor inside a match is worth understanding, because it's the whole design. The panel models are the workers, arguing, drafting, and attacking the problem from different directions. The referee is deliberately separate and deliberately strong, running on Claude Opus 4.5 or GPT-5, and its job isn't to contribute ideas but to check them. A criterion doesn't clear because the panel agrees. It clears because there's a source that can be re-checked or a computation that actually ran. That's a different trust model from asking one model to both solve a problem and grade its own solution, which is what an ordinary chat quietly does.

The natural audience is researchers, mathematicians, and professionals whose decisions need to survive scrutiny. If you've used a chatbot for anything rigorous, you've likely met the failure mode this attacks, plausible-sounding claims resting on invented citations or unchecked arithmetic. For that audience the appeal isn't speed, it's that a ruling comes with the sources and executed checks behind it, so the result can be handed to a colleague, a reviewer, or your future self without asking anyone to take a chatbot's word for it.

Pricing is prepaid usage with no subscription. New accounts get a one-time $1.00 starting credit, enough to run a complete match on public settings. After that you top up from $10 in any amount, credit never expires, unused credit is refundable on request, and paid accounts unlock private matches and higher spending limits. There are no seats, no minimums, and no recurring fees. If you'd rather pay your model provider directly, you can bring your own inference API key, in which case Referee.Chat charges a 5% infrastructure fee that only kicks in after your first $5.00 of usage, with private matches included.

It's a young product with an unusual shape, closer to a structured review process than a chat tool, and the billing model matches that. You pay for the work done, the spend cap protects you mid-match, and nothing renews in the background. For questions where being wrong is expensive, that's a sensible way to buy AI effort.