Adding Web Search to Your AI Tool Is an Integration, Not a Toggle

The big AI labs have been shipping web search as a bolt-on feature. You tick a box, the model can now look things up online instead of relying only on what it was trained on. It reads like a small upgrade. In practice, it's a new integration with its own costs, failure modes and things that can quietly go wrong, and most of the people switching it on haven't thought it through as one.

I've had a few conversations recently with business owners who want "the AI to be able to search the web" bolted onto a tool I'm building or maintaining for them. Usually the request is one sentence long. The actual decision underneath it isn't.

It's a third-party API call, same as any other

If I connect your system to Companies House, Xero or a courier's tracking API, you'd expect to ask: what does it cost per call, what happens if it's down, what do I do if it returns something wrong. Web search is the same kind of thing. The model doesn't magically know what's on the internet right now. It sends a query out to a search provider, gets a handful of results back, and reads them before answering you.

That means every search has a cost, a latency hit, and a chance of returning nothing useful. None of that goes away just because the interface is a chat box instead of a settings page.

The cost scales with usage you don't control

With a chatbot that only answers from its training data, your cost is mostly the per-message fee. Add web search and now some fraction of messages trigger one or more search calls, each billed separately. A customer-facing assistant that gets busy on a Friday afternoon can rack up a search bill that has nothing to do with how many customers you actually served that day, because the model decided to search twice for the same question or searched when it didn't need to.

I've written before about why any automation touching an external API needs a hard spending limit rather than just an alert that fires after the damage is done. Web search is exactly the kind of feature that benefits from that, because the trigger isn't a cron job you control, it's a model deciding on its own that it wants to look something up.

Caching stops you paying twice for the same answer

If ten customers ask your assistant the same question about opening hours, delivery zones or a product spec, there's no reason for that to be ten separate searches. A simple cache on the query, even a basic one keyed on the normalised question text, cuts the bill and makes responses faster. It's an easy thing to miss because the feature arrives looking finished out of the box. Nobody tells you it doesn't cache anything by default.

What did it actually read?

When a human employee tells a customer "I checked and we do offer that", you can ask them where they checked. When your AI assistant says the same thing after a web search, can you answer that question? If a customer later disputes what they were told, or a search result pointed to an out-of-date page that's since been corrected, you want a record of exactly which pages were fetched and what they said, attached to that conversation.

This is the same discipline as an audit trail anywhere else in the business: not just that something happened, but what triggered it and what the system was looking at when it decided. Without that log, a wrong answer becomes a "the AI said something weird" shrug rather than something you can actually trace and fix.

Not every result deserves to be trusted

A search result is just a web page someone else wrote, and the model treats whatever it contains as information to reason with. That's fine when the result is a product spec sheet. It's less fine when it's a forum post with wrong information, a competitor's marketing claim dressed up as fact, or a page specifically written to manipulate what an AI reads from it. Giving a model open access to the whole web means it can bring back anything, including content designed to steer it.

For a lot of business uses, a tighter setup works better than open web search: restrict it to a specific set of domains you trust, or point it at your own documentation and a handful of known sources rather than the open internet. That's more setup work than ticking the default box, but it's the difference between a feature that's useful and one that occasionally embarrasses you in front of a customer.

Treat it like what it is

None of this is a reason to avoid web search in a business tool. Being able to pull in current prices, stock levels from a public page, or a regulation that changed last month is genuinely useful. The point is that it's an integration, with cost controls, caching, logging and source rules, not a checkbox that makes the AI "smarter" for free. If you're adding it to something you've built or had built, it's worth scoping properly before it ships, rather than discovering the gaps when the first odd answer or the first unexpected bill shows up.