AgentsProof

AgentsProof is an evaluation platform for AI agents. You wrap your existing LLM and tool calls in a few SDK calls, and it captures the run as a trace, grades it against rules you wrote, and produces a report you can hand to someone else. The pitch on the site is aimed at a specific failure, you changed a prompt, something broke, and a user noticed before you did.

Integration is deliberately small. You install the package from npm, create a client with an API key, and call startRun with a project slug, the input, and a plain-English goal that anchors how the run gets graded. Each LLM or tool call gets wrapped in run.trace with a label and the function itself. When you call complete, you get back a public URL for the report. There are TypeScript and Python paths, and the site lists OpenAI, Anthropic, LangChain, CrewAI, the Vercel AI SDK and LlamaIndex as things it drops into, since it instruments your calls rather than replacing your framework.

Grading works on several layers, which is the more interesting part. Custom graders let you write what good means in plain English, something like the agent must never reveal user PII, and every run gets checked against those rules automatically. Goldens turn a run that passed into an executable test spec with success criteria, expected behavior and assertions attached, and you can pass a goldenId straight into startRun to test against it on demand. Trace assertions are the deterministic layer, structured checks that run without an LLM at all, so a rule like must_not_call:send_email or max_steps:10 fails the case outright and catches regressions an LLM judge might wave through. Synthetic variants use AI to generate edge cases from your existing Goldens so the suite grows without you hand-writing every test.

The onboarding is laid out as five steps and they map cleanly onto how you'd actually adopt it. Install the package, which the site describes as one package with zero config that works in any Node or edge runtime. Wrap your calls with run.trace, which it calls the whole integration. Add custom graders from the dashboard so the LLM checks every run against your rules. Save a passing run as a Golden from the trace view to set your success bar. Then run the proof suite. You can walk the output before writing any code, because example reports are live and public on the site. A report shows the project, how long ago the run happened and how many steps it took, an overall score out of 100, and the individual axis scores underneath it.

Proof suites tie it together. One SDK call runs every approved Golden through your agent and produces a report showing per-criterion pass and fail for each case alongside a five-axis score covering goal completion, tool accuracy, step efficiency, output quality and safety. The point the product keeps making is that a single number isn't enough, so results stay explainable down to the individual criterion rather than collapsing into one grade.

The reports being public and shareable is the angle that separates it from the obvious comparisons. AgentsProof puts itself next to LangSmith and Braintrust on its own comparison table and claims the differences are shareable public proof reports, a roughly five-minute SDK setup, saving real runs as test cases, and being designed for indie builders. That last one is the honest framing. This is built for a solo developer or small team who needs to show a client or a user that an agent behaves, not for an enterprise platform rollout.

Pricing is freemium and the free tier is usable rather than decorative. Free gives you one project, 200 eval runs a month, the default LLM grader, 10 golden test cases, one proof suite, public reports and basic email support, with no card required. Pro runs $29 a month or $290 a year and lifts that to unlimited projects, 10,000 eval runs a month, unlimited custom graders, Goldens and suites, private reports alongside public ones, and priority support. An eval run is counted each time startRun fires in the SDK, and hitting the free ceiling returns a 402 from the SDK while your previous runs stay available. Pro cancels anytime from the dashboard. The product is marked as being in beta, which is worth knowing before you build a release gate on top of it. Example reports are live and browsable on the site without an account, and there are public GitHub repositories for the project and its examples, so you can see the integration shape before committing any code. If you're shipping agents and your current quality process is running them a few times and eyeballing the output, this is aimed squarely at that gap.