AI chatbots have gotten genuinely useful for customer service in the last few years — but "useful" has specific limits, and small businesses that skip past them tend to find out the hard way, usually in a public review.
What they're actually good at today
Repetitive, well-defined questions with a clear right answer: order status, business hours, return policy, basic account troubleshooting. After-hours first response, so a customer gets something immediately instead of waiting until morning. And triage — routing a question to the right human or department faster than a generic contact form does.
What they're still bad at
Anything emotionally charged — a genuinely upset customer usually wants to feel heard by a person, and a bot that responds with cheerful scripted empathy tends to make things worse, not better. Nuanced or unusual complaints that don't match the bot's training data. And anything where a wrong answer has real consequences — refund policy edge cases, safety-related questions, anything involving a customer's money where the bot might state something incorrectly with total confidence.
The real cost structure
The platform fee is rarely the whole story. Many chatbot tools price on usage — conversations or resolutions per month — which can spike unpredictably during a busy season or a product issue, exactly when you need it most and can least afford a surprise bill. Factor in, too, the human time still required: reviewing flagged conversations, correcting the bot's knowledge base, and handling every escalation it couldn't resolve. A chatbot reduces support workload; it rarely eliminates it.
How to tell if it's actually working
Deflection rate — the percentage of conversations the bot handles without a human — is the metric every vendor leads with, and it's the wrong one to optimize alone. A bot can have a high deflection rate by giving customers a mediocre answer they don't bother escalating. Pair it with a real satisfaction signal on bot-only conversations specifically, not your overall support score, so you can see whether people are actually satisfied or just giving up.
Red flags before you sign
No easy, fast handoff to a human when the bot is stuck — customers trapped in a bot loop are one of the fastest ways to generate a one-star review. No way to review conversation logs and catch what the bot's getting wrong. And vague answers about where customer data is processed and stored, especially if you operate internationally.
How to pilot it without annoying your customers
Start narrow — deploy it for one well-defined question type (order status, say) before letting it attempt open-ended support. Make the human handoff obvious and fast from day one. And review a sample of real conversations weekly for the first month, not just the metrics dashboard — reading actual transcripts surfaces problems a satisfaction score won't.