You’ve in all probability by no means used Courageous Search. Nicely, circuitously. However the AI you ask questions may need, and it may need discovered your web site there. On Nov. 2, the New York Occasions sued Courageous, the corporate behind the Courageous browser and Courageous Search. The Occasions says Courageous copied greater than 30 billion webpages “with out figuring out itself or offering web site homeowners with any mechanism to dam its crawler,” then offered that content material to AI firms. Courageous hasn’t answered but, so for now these are claims. However why can’t you retain its crawler out with out holding Google out too? Courageous explains that itself, proper there on its assist web page.
Courageous Sells Its Search Index To Cohere, Mistral And Perplexity
Courageous sells entry to its search index to AI firms, by a product known as the Courageous Search API. When an AI assistant wants one thing current from the net, it doesn’t all the time go and browse the net itself. It might pay to look another person’s index as an alternative. Courageous’s About web page says the API is “Utilized by among the greatest names in AI, together with Cohere, Mistral, Perplexity, and You.com.” Its AI web page says the index holds “over 30 billion pages, with tens of millions of recent pages listed every day.”
The Occasions’s criticism opens by naming 5 AI firms it says Courageous provides: Anthropic, Perplexity, Mistral, xAI and Cohere. None of them is being sued. Claude is the fuzzy one, as a result of the proof is outdated. The criticism factors to a 2023 Courageous put up about Anthropic coaching Claude 2. In March 2025 Anthropic added Courageous Search to its listing of subprocessors, the skin firms it makes use of, which TechCrunch reported. The identical week, Simon Willison, a developer who writes a broadly learn weblog about AI instruments, requested Claude to seek for pelican information, and the ten outcomes it cited matched the identical search on Courageous precisely. Whether or not Claude nonetheless makes use of Courageous immediately, I couldn’t affirm.
Courageous’s Crawler Doesn’t Have A Title, So You Can’t Block It With out Blocking Google
Web sites resolve which crawlers might learn them in a small file known as robots.txt, and that solely works as a result of crawlers have names. OpenAI and Anthropic every publish the names of theirs, so you possibly can let one firm’s crawlers in and switch the opposite’s away.
Courageous’s assist web page, the one on the prime of this text, says its crawler “doesn’t promote a differentiated person agent.” In plain phrases, Courageous’s crawler doesn’t put on a reputation tag, and Courageous offers a motive: Some web sites let Google in and no person else, so a crawler with Courageous’s identify on it could get turned away. Courageous says it follows Google’s guidelines as an alternative: no matter Googlebot isn’t allowed to learn, Courageous received’t learn both.
So you possibly can’t let Google in and preserve Courageous out. You get each, or neither. To get out of Courageous’s index, the identical web page factors you to the noindex tag, which tells Google to drop the web page too. The Occasions calls this Courageous’s “refusal to launch the identify of its net crawler bot.”
My Web site Is In Courageous’s Index, And My Robots.txt Can’t Even Point out It
A Courageous seek for website:nohacks.co on Nov. 3 confirmed 16 of my pages on the primary web page of outcomes, 13 of them articles.
Courageous’s outcome even reveals the homepage description I changed earlier that very same day, so what Courageous serves is a saved copy, not a reside have a look at my web site. My robots.txt names 13 AI bots, from GPTBot to Bytespider, and blocks two of them, Meta’s and ByteDance’s. There’s no line I might add for Courageous. It’s onerous to dam a reputation that doesn’t exist.
Checking yours takes about 30 seconds: go to look.courageous.com and sort website: and your area. All the pieces that reveals up is within the index Courageous sells.
A Search Engine Sends Folks Again, And An Index Bought To AI Doesn’t Have To
Being in a search engine is a deal each web site understands. Somebody searches, sees your web page, clicks, and lands in your web site. Courageous Search’s odd outcomes work like that, although Courageous has added AI solutions of its personal, known as Ask Courageous, which The Occasions can be suing over.
Promoting the index is a distinct deal. The criticism quotes Courageous describing one a part of it, “LLM Context,” as handing over “extracted web page chunks and supply metadata in a form and size that satisfies the wants of an LLM with out want for additional scraping.” So the AI firm will get your web page, already chopped up, and writes a solution from it. Whether or not that reply sends anybody again to you is the AI firm’s name. It isn’t yours, and it isn’t one thing Courageous is promoting. That’s the half that bothers me.
A Court docket Will Resolve Whether or not Courageous Can Promote Its Index To AI Firms
The New York Occasions is claiming copyright infringement and getting round its paywall, amongst different issues, and the Every day Information, the Chicago Tribune and the Orlando Sentinel filed a separate case the identical day. Courageous hadn’t responded to both after I wrote this. It has argued its aspect earlier than, although: when Information Corp despatched it a cease-and-desist, Courageous sued first, saying it was solely indexing the net, which “all search engine operators should do to exist,” and in July 2026 Information Corp sued again, saying Courageous hides its crawlers so publishers can’t detect or block them, Semafor reported. A ruling on The Occasions’s pages received’t mechanically cowl anybody else’s, however it’ll say rather a lot about each web site in that index, and that’s in all probability yours too. So I’ll be watching this one.
Extra Assets:
This put up was initially revealed on No Hacks.
Featured Picture: FotoField/Shutterstock
