Innovative Commercial Strategies Innovative Commercial Strategies CRE Advisory + AI Systems

Glossary · AI systems & marketing

AI hallucination

When an AI system states something false with full confidence, as if it were fact.

What it means

An AI hallucination is an answer from an AI system that is wrong or invented but delivered as fact. It can be a made-up price, a warranty that does not exist, a wrong phone number, or a citation to a source that was never written. The system is not lying on purpose. It is filling a gap with plausible text.

Why it matters for a local business

A chat agent that tells a Quincy customer you offer 24-hour emergency service when you close at 5 has created a promise you now have to break. ICS builds chat and phone agents to answer only from the business’s own approved facts, to say “let me get someone for that” when the answer is missing, and to pass the conversation to a human. In medical, dental, and legal settings that limit is not optional.

In practice

A Brockton HVAC company tests its chat agent with 50 real customer questions before launch. It gets 46 right, declines 3 correctly, and invents a $79 tune-up price on 1. The fix takes 10 minutes: add the real $129 price to the approved facts and require a handoff on any pricing question not listed.

Straight answers

Why do AI systems hallucinate?
Because they produce the most likely next words, not a looked-up fact. When the system has no solid information on a question, it can still write a fluent, confident answer that happens to be wrong. Fluency and accuracy are separate things, and the tone gives no warning.
How do you stop an AI chat agent from making things up about my business?
Limit what it can answer from. Give it a written set of your real prices, hours, service area, and policies, instruct it to say it does not know when the answer is not there, and hand off to a person for anything involving a price commitment, a diagnosis, or a complaint.

Related terms

← All 537 terms · See the systems and published pricing →

Hearing this term in a deal right now?

Call before you sign. A straight read on the clause, the number, or the pitch, usually in one conversation.