Crawlie Search

Crawlie Search is a hosted site-search product that builds its index straight from web crawls you've already run. It's one piece of Crawlie Cloud, a broader platform that crawls a site once and turns that data into several products, and Search is the part that gives visitors a fast way to find things on your pages. The pitch is plain, blazing-fast site search without the cost, set up in minutes. The problem it targets is the price and plumbing of traditional hosted search. Tools in that space often charge per record and per index operation, and they expect you to build and maintain a separate pipeline that pushes your content into their service. Crawlie skips that step because the crawl it already ran is the source, so there's no extra ingestion job to stand up or keep in sync.

The search itself is hybrid. It combines exact-match queries with semantic matching, so a visitor can type a precise term or ask a natural-language question and still land on the right page. Ranking uses BM25, a well-understood relevance model, and it handles typo tolerance and phrase queries so small mistakes and quoted strings both behave the way people expect. Because the index comes from the same crawl that powers the rest of the platform, the content stays consistent with what Crawlie already knows about the site. You aren't maintaining two versions of your data, one for monitoring and one for search, which is part of the appeal of doing the crawl once and reusing it everywhere.

Getting it onto a site is deliberately light. Paste a single script tag and you get a polished command-K style search dialog, the familiar keyboard-driven overlay, with keyboard controls, dark mode, and mobile handling already built in. If you'd rather own the interface, you can skip the widget and query a public JSON API directly, wiring results into your own front end. It's also agent-native. A hosted MCP endpoint lets Claude, Cursor, and any other MCP client query the same index, and there's a REST API alongside it, so the search that serves human visitors can also answer questions coming from AI tools without a second setup.

On top of serving queries, Crawlie Search reports on them. An analytics dashboard shows the top queries people are running and, more usefully, the content gaps where searches came back with nothing. That second view is a quiet content-strategy tool. If visitors keep searching for something the site doesn't cover, the dashboard makes that demand visible so a team can decide whether to write the page. It turns the search box into a feedback channel about what an audience actually wants, rather than just a way to move people around existing pages, and because it's tied to the crawl data the platform already holds, the picture stays current as the site changes.

Price is where Crawlie leans hardest on the comparison. It puts its own cost next to Algolia directly, and for 250,000 monthly searches it frames Algolia Grow at around $120 a month against its own figure of nothing during beta and roughly $6 at general availability. It's blunt that there are no per-record or indexing fees, ever, which is the charge that tends to inflate bills on record-priced search services. That makes it a natural fit for developers, side-project owners, and small teams who want real search on a site without an enterprise-sized invoice, especially anyone who already needed a crawler for SEO or monitoring and would rather not pay a second vendor for search.

On access, the product is free during beta, with searches and cited answers both carrying no charge while it's in that phase. At general availability the plan is usage-based rather than seat-based, with somewhere between 10,000 and 1,000,000 included monthly searches depending on the tier, overage billed at four cents per thousand searches, and cited answers priced at one dollar per thousand generations drawn from a prepaid wallet. The wider Crawlie Cloud platform it belongs to runs its own tiered plans, from a free Hobby level up through paid Pro, Business, and Scale plans, so a team can adopt search on its own or alongside the crawling, SEO, and monitoring pieces. The crawler engine underneath is open-source, published under an MIT license and installable from npm, with a command-line tool and a desktop app for people who want to run crawls locally before pushing anything to the hosted service, and it scales to as much as a million pages in a single crawl. Custom plans are available for larger deployments, and the same crawl feeds all of it.