llmspeed

llmspeed is an open source simulator for comparing how local large language models are likely to feel on different computers. It helps answer a practical question that specification sheets rarely settle on their own. If someone buys a particular GPU, Mac, or CPU system, will the model they want actually fit, how long will the first response take, and how quickly will the rest of the answer appear? Instead of reducing that decision to a table of tokens per second, llmspeed streams sample text at the predicted pace. That makes the difference between a responsive setup and a frustrating one easier to understand before money is spent on hardware.

The main workflow starts with a hardware choice, a model, and a quantization. A user can run that configuration and watch an answer arrive after a simulated wait for the first token. The site reports expected generation speed, time to first token, and whether the selected model fits in available memory. It also distinguishes a clean fit from partial offload to system RAM and from a configuration that won't run. The catalog currently spans GPUs, Macs, and CPU hardware across CUDA, Metal, ROCm, Vulkan, and CPU backends. Dedicated pages let visitors browse by hardware or by model, so the tool works both for someone starting with a machine and for someone starting with a model requirement.

Race mode is the most useful part for direct buying comparisons. It runs two setups side by side, which turns an abstract difference in benchmark numbers into two visible response streams. A prospective buyer can compare a discrete GPU with a high memory Mac, try different quantizations, or see the effect of selecting hardware that forces some model weights into system memory. Configurations and races can be shared through links, which is handy when a team is discussing a purchase or helping someone choose a build. The simulation is aimed at short prompts on one device, so it gives a focused picture of interactive local inference rather than pretending to model every possible production workload.

Its estimates combine submitted benchmark results with calculations for combinations that haven't been measured directly. llmspeed labels each result as measured, derived, or projected, giving the reader an immediate signal about how much confidence to place in it. Measured means the same hardware, model, and quantization were benchmarked. Derived figures are interpolated from similar measured setups, while projected figures use hardware specifications and can cover announced hardware before real measurements appear. The calculation accounts for memory bandwidth during generation, compute during prompt processing, backend efficiency, context slowdown, model weights, key value cache, runtime overhead, and the slower system memory path used by partial offload.

That transparent confidence model is what separates llmspeed from a plain benchmark leaderboard. Raw scores are useful, but they often leave a newcomer to translate model size, quantization, memory capacity, prompt processing, and output rate into an actual experience. llmspeed performs that translation while keeping the source quality visible. Community members can contribute measurements by running llama.cpp's benchmark utility and submitting the output, engine version, prompt and generation figures, hardware details, and a source link. Results from other engines are also welcomed when the engine is identified. The site's prediction data is generated by its benchmark pipeline rather than hidden behind an unexplained rating.

The best audience is anyone choosing hardware for local model experimentation, from an individual building a workstation to a developer checking whether a target model is realistic for users' machines. It can narrow a shopping list, compare an existing computer with a possible upgrade, or provide a quick sanity check before downloading a very large model. It isn't a replacement for testing on the final machine. The maintainers say drivers, engine versions, flash attention, power limits, and thermals can move real results by 10 to 30 percent, and multi GPU setups aren't simulated. Projected results for unreleased hardware are also estimates that are meant to be replaced as measurements arrive.

Access is straightforward. The simulator is available on the web, its source and prediction data are public on GitHub, and the project uses the MIT license. There is no account requirement or paid plan presented on the official site. Developers can inspect the assumptions, run the static Vite and Preact application locally, or contribute a benchmark through the repository. That open posture makes llmspeed especially useful as an early decision aid. It offers a faster and more intuitive first pass than collecting scattered benchmark posts, while clearly showing where its output comes from and where a real local test still matters.