Most AI chatbots are broken on purpose.

Not because the people building them are dumb. Because the people buying them don't know what to ask for.

Here's what's happening on 90% of the chatbots you see on small business websites right now.

The setup nobody talks about

A business owner buys a chatbot. The platform asks them to "upload some documents" or "paste your FAQs." They paste a few paragraphs. The bot goes live.

Now a customer asks a question. Something specific. "Do you service Saratoga County on weekends?"

The bot has two options. Either it makes something up that sounds confident (and is wrong), or it spits back a generic non-answer. "Please contact us for more information."

Either way, the lead is gone.

What's actually broken

The bot doesn't know the business. It knows the internet.

When you ask a generic chatbot a question, it's pulling from its training data. That training data is the public web from a year ago. It does not include your service area, your pricing, your hours, your policies, or your products.

So when a real customer asks a real question, the bot guesses.

Think of it like hiring a receptionist who has never been to your office. They sound polite on the phone. They have no idea what your business does. Every customer call ends in confusion.

What RAG is, in plain English

RAG stands for Retrieval-Augmented Generation. The name is ugly. The concept is simple.

Imagine your receptionist gets a binder. Inside the binder is everything about your business. Service areas. Pricing. Common questions. Policies. Product specs.

When a customer asks a question, the receptionist doesn't guess. They flip to the right page in the binder, read the answer, then say it back in their own words.

That's RAG.

The "binder" is a vector database (Pinecone, in most builds). Your business data gets chopped into chunks and stored there. When a customer types a question, the system searches the database for the most relevant chunks, hands those chunks to the AI, and the AI answers using only that information.

No hallucinating. No making things up. The bot can only say what's actually in your binder.

What this looks like in production

Built a chat assistant for a home services client a few months ago. The pipeline:

1. Pulled their service descriptions, pricing structure, service area zip codes, and intake process into structured documents.

2. Chunked the documents and pushed them into a Pinecone index. 52 vectors total.

3. Connected the chat widget on their site to the Pinecone index plus an LLM.

4. Built escalation logic so anything the bot can't confidently answer routes to a human.

A customer asks "do you do dryer vent cleaning in Glens Falls?" The system searches the index, finds the service description and the zip code list, hands both to the model, and the model responds with a real answer based on real data.

If the customer asks something not in the binder ("can you fix my furnace?"), the bot says it doesn't handle that and routes them to the right place.

That's the difference between a chatbot that captures leads and a chatbot that loses them.

What to ask before buying one

If you're shopping for an AI chat assistant, three questions filter out 90% of the bad ones:

1. Does it use RAG with a real vector database, or is it just hooked up to ChatGPT?

2. Will the data be your business data, or generic templates?

3. What happens when the bot doesn't know the answer? Does it guess or escalate?

If the salesperson can't answer those questions, walk.

A real chat assistant is a system. Not a widget.