Why AI chatbots make things up, and how to stop yours
The fastest way to lose a customer's trust is to tell them something confidently that turns out to be false. A made-up refund policy or an invented feature can create real costs: angry emails, chargebacks, sometimes legal exposure.
The good news is that these "hallucinations" have understandable causes, and most of them are in your control.
Where made-up answers come from
Language models are trained to produce fluent, plausible text. When they have solid reference material, they use it. When they don't, they tend to fill the gap with something that sounds right.
In a support assistant, that gap usually shows up in one of four ways:
- The information simply isn't there. Nobody ever wrote down the answer, so the model improvises.
- The information is outdated. An old pricing page is still indexed and contradicts the new one.
- The question is ambiguous. "How long does it take?" could mean shipping, setup or a refund.
- The instructions invite guessing. Prompts like "always give a helpful answer" push the model to answer even when it shouldn't.
How CustGPT keeps answers grounded
Every reply goes through the same routine. First, CustGPT looks up the passages from your content that best match the question. Then the model is told to answer only from those passages, to cite which ones it used, and to say plainly when they don't contain the answer.
That's why you'll see small numbered citations in replies, and links to the source pages underneath. They aren't decoration: they make it easy for you and your visitors to check where an answer came from.
Five habits that reduce hallucinations further
- Write a good fallback message. Give the assistant a graceful exit, such as "I don't have that information, but our team can help at support@…". Models are far less likely to invent an answer when they have an approved alternative.
- Remove stale pages. If an old promotion or discontinued product is still in your sources, delete it. Contradictory material produces muddled answers.
- Add custom answers for high-stakes topics. Refunds, warranties, compliance and anything legal deserve exact wording that you control.
- Avoid "always answer" instructions. Ask for honesty instead: "If you're not sure, say so and point to the team."
- Review the thumbs-down replies. Visitors' feedback in the Conversations tab shows you precisely where your content has holes.
Accuracy is a content problem
It's tempting to treat hallucinations as a mysterious AI flaw. In practice, most wrong answers trace back to missing, outdated or ambiguous content. Fix the content and the answers follow, and your human support team benefits from that clean-up too.