AI Voice Agents Explained: Inbound, Outbound and the Setup Behind Them

Two years ago, “AI phone agent” meant a demo that worked in the video and fell apart on the second question. That is no longer true. The current generation answers within a second, handles interruptions, books into a real calendar and transfers to a person when it should. It is now a practical tool for businesses that live on the phone — but it is a tool with a specific shape, and this article is about that shape.

I build voice agents for clients, mostly in GoHighLevel’s Voice AI and on Twilio-based stacks. Here is what they are, the difference between inbound and outbound, the components behind one, what they cost, and where they still need a human standing by. For the receptionist use case specifically, see AI receptionist for small business.

What an AI voice agent actually is

A phone call handled by software that does four things in a loop, many times a second:

1. Listens — converts the caller’s speech to text.
2. Understands — a language model works out what was said and what to do, using instructions and a knowledge base you wrote.
3. Acts — checks a calendar, looks up a contact, books, sends a text, transfers, or records details.
4. Speaks — converts its reply back to natural speech.

That is different from an IVR (“press 1 for sales”) and from a chatbot (text only). The caller talks normally; the agent talks back normally; and because it is connected to your systems, things actually happen during the call.

Inbound agents

The agent answers calls to your business number — all of them, or the ones the front desk cannot take, or the after-hours ones. Typical jobs:

• Answer FAQs (hours, location, services, rough pricing, what to bring).
• Book, reschedule or cancel appointments against a live calendar.
• Take a message with name, number and reason, and route it to the right person.
• Qualify enquiries with two or three questions and create the lead in the CRM.
• Transfer to a human on request, on detecting an emergency or complaint, or after failing to answer twice.

This is the receptionist pattern, and it is the lower-risk place to start because the caller initiated the conversation and wants something the agent can give.

Outbound agents

The agent places calls. The jobs are narrower, and the rules are stricter:

• Appointment reminders and confirmations — “This is Lakeside Dental confirming your appointment tomorrow at 2:30. Press or say 1 to confirm.”
• Speed-to-lead call-backs — a form is submitted, the agent calls within a minute, qualifies, and books. Combined with the instant SMS, this is a strong opening.
• Reactivation — past customers who opted in, with a genuine reason to return.
• Surveys and reviews — a short call after a job asking how it went.

The parts behind an agent

All-in-one platforms bundle the first four; the value you add is in the last two. An agent with perfect speech and a vague knowledge base is a polite waste of the caller’s time.

What a good agent’s instructions look like

Written in plain language, and short. A working structure:

1. Identity and disclosure. “You are the assistant for Lakeside Dental. Say at the start that you are an AI assistant.”
2. Goal. “Help callers book, reschedule or cancel appointments, answer questions from the knowledge base, and take messages.”
3. Boundaries. “Do not give medical advice, quote exact prices, or promise availability you have not checked.”
4. Handoffs. “Transfer to the front desk if the caller asks for a person, mentions pain or an emergency, or if you cannot answer after two attempts.”
5. Style. “Short sentences. One question at a time. Confirm the date and time back before booking.”
6. Actions available and when to use each.

Then the knowledge base: the thirty questions you actually get, with your actual answers. Same discipline as a website chatbot; the only difference is that on the phone, long answers are worse.

What it costs

Usage: per-minute pricing for the voice AI (typically a small fraction of a dollar per minute, varying by platform), plus the phone number and any transcription or CRM usage. A business handling a few hundred calls a month usually pays far less than a part-time employee. The one-off setup — flows, knowledge base, integrations and two or three weeks of listening and tuning — is the larger cost. Treat the setup as a project, the usage as a utility bill.

Where they still fall short

• Emotional calls. Complaints, distress, grief — transfer immediately. An AI that tries to handle these damages the business.
• Heavy accents, noisy lines, speakerphones. Better than a year ago, still not perfect. Give callers an easy path to a human.
• Anything requiring judgement. Discounts, exceptions, “can you squeeze me in?” — take the details, promise a call-back.
• Multi-step, account-specific support. “My invoice from March is wrong” needs a person with the account open.
• Latency spikes. Occasionally a reply takes two or three seconds; a well-scripted agent fills it with a natural “let me check that” rather than silence.

Measuring it

• Answer rate and average time to answer (should approach 100% and one second).
• Resolution rate: calls completed without transfer.
• Bookings created by the agent.
• Transfer reasons, weekly — the list of things to add to the knowledge base.
• Caller satisfaction: a one-question SMS after the call for the first month.

How to start

1. Inbound first, after-hours first. Lowest risk, clearest win.
2. Write the knowledge base from real calls before touching any platform.
3. Connect the calendar and CRM so the agent can act, not just talk. The CRM setup guide covers the calendar it depends on.
4. Run it on a test number for a week with staff and friendly customers.
5. Go live on overflow and after-hours; expand to all calls once the transfer rate is low.
6. Only then consider outbound, starting with reminders to existing customers.

Further reading: GoHighLevel Help Center · FCC: unwanted calls and texts.

FAQ

Want to hear an agent take a call for your business before committing? Book a call and I will set up a short demo on a test number using your own questions and calendar.