VoiceDuel

VoiceDuel is a blind comparison platform where you talk to two AI voice models at once and vote on which one performed better. You write a character brief describing the persona you want both models to play, then speak with them in real time through your microphone. Each model attempts to embody that character in live conversation, and you judge the results without knowing which vendor is behind each response. The votes feed into a public leaderboard that ranks speech to speech models using statistical methodology borrowed from competitive gaming and chess ratings.

The problem it addresses is that evaluating voice AI is subjective and hard to do fairly through benchmarks alone. Reading accuracy scores or latency numbers doesn't tell you whether a model sounds natural in extended conversation, handles interruptions gracefully, stays in character when you push back, or recovers well from misunderstandings. VoiceDuel lets you experience the comparison directly, with both models responding to your actual voice in the same session. The blind setup removes the bias that comes from knowing which company made which model before you hear it, which tends to color perception when people already have expectations about a provider.

The platform currently compares models from three providers. OpenAI's GPT Realtime, xAI's Grok Voice, and Alibaba's Qwen Omni Realtime are the systems under evaluation. These are true speech to speech implementations where audio input goes directly to audio output, not text to speech layers sitting on top of transcription pipelines. The distinction matters because end to end voice models handle tone, pacing, and interruptions differently than cascaded systems that convert speech to text, generate a response, and synthesize it back to audio.

Importantly, VoiceDuel ranks configurations rather than just base models. Settings like turn detection sensitivity, voice selection, and system instructions all affect how a model performs in practice. Two deployments of the same underlying model can rank differently if they are tuned differently, and the platform captures that nuance. This makes the leaderboard more actionable for developers who need to choose not just a vendor but a specific configuration for their use case.

The leaderboard uses Bradley Terry methodology with 95% bootstrap confidence intervals, so you can see when a ranking is statistically meaningful versus still noise from limited voting data. Matchmaking favors less frequently paired configurations to gather more data where it is most needed. The system also counterbalances slot order, randomizing whether a model speaks first or second in each session, to prevent position bias from skewing results. This statistical rigor makes the rankings more useful than raw vote counts, especially when comparing models that are close in capability.

Privacy is handled with a clear policy. Audio streams directly to the provider APIs during the call but is never stored to disk by VoiceDuel. The only data that persists is call metadata, specifically your vote, which configuration pair was tested, and the call duration. Your actual voice recordings are not retained for training, review, or any other purpose. This matters for anyone uncomfortable with their voice being collected, which is a reasonable concern when testing AI voice systems from multiple vendors in quick succession.

The platform is free to use with no subscription, no credit system, and no visible monetization. It appears to be a research or demonstration project aimed at the community of developers and researchers evaluating voice AI options. There is no account system blocking access. You visit the site, allow microphone permissions, write a character prompt, and start talking. For developers building products that need real time voice interaction and want to compare providers before committing, this gives you a structured way to run that comparison without setting up your own testing infrastructure or burning API credits on each vendor's playground.

The focus on character embodiment is deliberate. Rather than generic small talk, the scenarios ask models to role play specific personas, which stresses their ability to maintain consistency, respond in character, and adapt to conversational turns. This is closer to how voice AI gets used in production for things like customer service agents, game NPCs, and interactive companions. If you are evaluating whether a voice model can hold character under pressure, VoiceDuel offers a faster feedback loop than building your own prototype first.