Glossary · AI systems & marketing
AI hallucination
When an AI system states something false with full confidence, as if it were fact.
What it means
An AI hallucination is an answer from an AI system that is wrong or invented but delivered as fact. It can be a made-up price, a warranty that does not exist, a wrong phone number, or a citation to a source that was never written. The system is not lying on purpose. It is filling a gap with plausible text.
Why it matters for a local business
A chat agent that tells a Quincy customer you offer 24-hour emergency service when you close at 5 has created a promise you now have to break. ICS builds chat and phone agents to answer only from the business’s own approved facts, to say “let me get someone for that” when the answer is missing, and to pass the conversation to a human. In medical, dental, and legal settings that limit is not optional.
In practice
A Brockton HVAC company tests its chat agent with 50 real customer questions before launch. It gets 46 right, declines 3 correctly, and invents a $79 tune-up price on 1. The fix takes 10 minutes: add the real $129 price to the approved facts and require a handoff on any pricing question not listed.
Straight answers
- Why do AI systems hallucinate?
- Because they produce the most likely next words, not a looked-up fact. When the system has no solid information on a question, it can still write a fluent, confident answer that happens to be wrong. Fluency and accuracy are separate things, and the tone gives no warning.
- How do you stop an AI chat agent from making things up about my business?
- Limit what it can answer from. Give it a written set of your real prices, hours, service area, and policies, instruct it to say it does not know when the answer is not there, and hand off to a person for anything involving a price commitment, a diagnosis, or a complaint.
Related terms
-
AI chat agent
A trained assistant on your site or phone line that answers questions and books appointments around the clock.
-
AI receptionist
Software that answers calls and texts around the clock, books jobs, and hands real leads to a person.
-
Prompt
The written instructions and context given to an AI system to shape its response.
-
Chat handoff
The moment an automated chat passes a conversation, with its context, to a human.