How does an outbound calling voice agent work?
- Authors
- Name
- Amit Shekhar
- Published on
An outbound calling voice agent is an AI voice agent that dials a phone number by itself, talks to the person who picks up, and saves the result of the call. Instead of waiting for people to call in, it starts the call.
In this blog, we will learn about how an outbound calling voice agent works. We will also see how it differs from an inbound call, how it decides whom to call and when, how a call reaches a phone, how it knows whether a human or a voicemail picked up, how every call ends with a saved outcome and a plan for the next try, and where it is used.
We will cover the following:
- What is an outbound calling voice agent?
- Inbound vs Outbound calls
- The big picture of an outbound call
- Step 1: Deciding whom to call and when
- Step 2: Placing the call through the phone network
- Step 3: Ringing, busy, or no answer
- Step 4: Human or voicemail?
- Step 5: Starting the conversation
- Step 6: Running the conversation
- Step 7: Ending the call and saving the outcome
- Where outbound calling voice agents are used
I am Amit Shekhar, Founder @ Outcome School, I have taught and mentored many developers, and their efforts landed them high-paying tech jobs, helped many tech companies in solving their unique problems, and created many open-source libraries being used by top companies. I am passionate about sharing knowledge through open-source, blogs, and videos.
I teach AI and Machine Learning at Outcome School.
Let's get started.
What is an outbound calling voice agent?
An outbound calling voice agent is an AI voice agent that places phone calls on its own, holds a spoken conversation with the person who answers, and saves the result of each call.
Let's say a clinic has 500 patients with appointments tomorrow. Someone has to call each patient and ask, "Can you confirm your appointment for tomorrow?" A small team of humans can take the whole day to do this.
An outbound calling voice agent can do the same job. It picks a patient from the list, dials the number, waits for the patient to answer, asks the question, understands the reply, marks the appointment as confirmed or cancelled, and moves to the next patient. It can also run many such calls at the same time.
Inbound vs Outbound calls
Before jumping into how it works, we must know how an outbound call is different from an inbound call.
In an inbound call, the customer calls us. The customer already knows why they are calling, and they expect someone to answer.
In an outbound call, we call the customer. The person does not know who is calling or why, so the agent must explain quickly. The phone can also ring with no answer, the line can be busy, or a voicemail can pick up instead of a human.
Let me tabulate the differences between inbound and outbound calls for your better understanding.
| Point | Inbound call | Outbound call |
|---|---|---|
| Who starts the call | The customer | The agent |
| Who speaks first | The agent greets the caller | Usually the person says "Hello?" |
| Voicemail possible | No | Yes |
| Unanswered calls | No retry needed | Must be retried later |
| Choosing the time | The customer chooses | We must choose a good time |
So, the conversation part is almost the same. The extra work is everything around it: dialing, waiting, detecting voicemail, and retrying.
We have a detailed blog on how a voice AI agent answers a phone call that explains the inbound side step by step.
The big picture of an outbound call
Let's first see the complete journey of one outbound call in a diagram.
+----------------------------------+
| Step 1: Pick a contact and time |
+----------------------------------+
↓
+----------------------------------+
| Step 2: Place the call |
+----------------------------------+
↓
+----------------------------------+
| Step 3: Ringing, busy, no answer |
+----------------------------------+
↓
+----------------------------------+
| Step 4: Human or voicemail? |
+----------------------------------+
↓
+----------------------------------+
| Step 5: Start the conversation |
+----------------------------------+
↓
+----------------------------------+
| Step 6: Run the conversation |
+----------------------------------+
↓
+----------------------------------+
| Step 7: Save outcome, plan retry |
+----------------------------------+
Now, let's understand each step one by one.
Step 1: Deciding whom to call and when
Every outbound calling system starts with a list of people to call. This list is called a campaign. A campaign has a goal, for example "confirm tomorrow's appointments", and a list of contacts with their phone numbers and some details about each one.
The part of the system that picks the next contact and starts the call is called the dialer. Before dialing, the dialer checks a few things:
- Is it a good time for this person? The dialer looks at the person's time zone and only calls within allowed calling hours. For example, it does not call anyone at 6 AM in their local time.
- Is this person allowed to be called? Many countries have rules about phone calls. For example, people can add their number to a "Do Not Call" list, and some calls need the person's consent first. The dialer must skip anyone who should not be called.
- How many calls can we run at once? Each live call needs its own speech recognition, LLM, and speech generation running at the same time, and the telephony account often has a limit on how many calls can run at once. So, the dialer limits how many calls run in parallel.
- Has this person been tried before? If we already called this person three times today, we must not call again right now.
Once a contact passes all these checks, the dialer prepares the details for this call, for example the patient's name, the appointment time, and the clinic name. These details go into the instructions for the LLM, so that the agent can greet the patient by name, for example Rahul, and talk about his exact appointment.
Now, the dialer is ready to place the call.
Step 2: Placing the call through the phone network
An AI agent runs on a server. A phone sits in someone's pocket. So, how does a call from a server reach a phone? The answer is a telephony provider.
A telephony provider is a company that connects the internet to the phone network. The phone network that all normal phones use is called the PSTN, which stands for Public Switched Telephone Network.
The flow is as below:
+------------------------+
| Our voice agent server |
+------------------------+
↓
+------------------------+
| Telephony provider |
+------------------------+
↓
+------------------------+
| PSTN (phone network) |
+------------------------+
↓
+------------------------+
| The person's phone |
+------------------------+
Our server asks the telephony provider to call a number. This request is often made through a web API, or through a protocol called SIP.
SIP stands for Session Initiation Protocol. It is the standard set of messages used to start and end internet phone calls. When a call starts, an INVITE message is sent, which means "I want to call this number." The other side replies with messages that tell us what is happening with the call.
The call also needs a caller ID, the number that shows up on the person's screen. Outbound systems use a real phone number that the business owns, so that the person can call back if needed.
Once the call is connected, the audio travels both ways. The person's voice comes to our server, and our agent's voice goes to the person's phone. Many telephony providers stream this audio to our server over a WebSocket connection. Phone audio is usually sampled at 8 kHz, which is lower quality than the 16 kHz or higher audio used in most apps, so the speech recognition model must work well with phone audio.
Planning your move into AI?
No matter which tech domain you work in today, with your effort, you can transition to these roles:
- AI Engineer
- Forward Deployed Engineer
- AI Agent Engineer
- Harness Engineer
- Voice AI Engineer
- AI Architect
- LLM Engineer
- LLM Inference Engineer
- Gen AI Engineer
- MLOps Engineer
- ML Engineer
- Deep Learning Engineer
Our AI and Machine Learning Program at Outcome School takes you there.
Step 3: Ringing, busy, or no answer
After the call is placed, the phone does not answer right away. A few different things can happen, and the system must handle each of them.
The telephony provider keeps sending updates about the call. These updates are called call progress events. With SIP, they come as reply messages with codes, as below:
180 Ringingmeans the phone is ringing.200 OKmeans someone or something picked up the call.486 Busy Heremeans the line is busy.- If nobody answers for a set time, for example 30 seconds, the call is treated as no answer and is ended.
- The number can also be invalid or disconnected, and the network tells us that too, for example with
404 Not Found.
Here, we must notice that 200 OK only tells us that the call was picked up. It does not tell us who picked up. It can be a human, or it can be a voicemail system. Both look exactly the same at this point.
So, how does the agent know who picked up? Let's see the next step.
Step 4: Human or voicemail?
This step is called Answering Machine Detection, or AMD for short.
Let's say the agent starts talking the moment the call is picked up. If a voicemail picked up, the agent talks over the voicemail greeting, and only half of the message gets recorded. On the other hand, if the agent stays silent for too long to be sure, a human will say "Hello? Hello?" and hang up.
So, the system must decide quickly. The best way to learn this is by taking an example.
When a human picks up, the audio usually sounds like this:
"Hello?" ... (silence, waiting for us to speak)
When a voicemail picks up, the audio usually sounds like this:
"Hi, you have reached Rahul. I am not available right now.
Please leave a message after the beep." ... (beep)
Here, we can notice the difference:
- A human greeting is short, usually one or two words, followed by silence.
- A voicemail greeting is long and continuous, and it often ends with a beep.
So, AMD listens to the first few seconds of audio and looks at these patterns: how long the first speech lasts, how long the silence after it is, and whether there is a beep. Some systems also turn the greeting into text and check for phrases like "leave a message" or "not available".
Based on this, AMD gives one of the following results:
- Human: the agent starts the conversation.
- Machine: the agent waits for the greeting to end, with a beep or with silence, and then leaves a short, pre-written message, or hangs up and tries again later. Leaving a recorded message like this is called a voicemail drop.
- Unknown: the system follows a default rule that we set, for example, treating it as a human.
AMD is never perfect. It is a tradeoff between speed and accuracy, and the result also changes from one phone network to another. If it decides too fast, it does more mistakes. If it waits too long, the human hangs up. So, we must tune the waiting time based on our use case.
A quick note for you
No matter which tech domain you work in, get familiar with these topics:
- LLM
- RAG
- MCP
- Agent
- Fine-tuning
- Quantization
We put it all together in one video:
AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, and Quantization
No need to stop reading - bookmark it and watch later when you get time. Future you will thank you.
Now, let's get back to the topic.
Step 5: Starting the conversation
Now, a human has picked up. Here comes a question that is special to outbound calls: who speaks first?
In an inbound call, the agent greets the caller. In an outbound call, the person who picks up usually says "Hello?" first. This is the same short greeting that AMD used in Step 4 to detect a human. So, a common rule is as below:
- Once AMD says human, the agent speaks its opening line right away.
- If the person stays silent for a short time, the agent speaks first anyway.
The opening line matters a lot in an outbound call, because the person does not know who is calling. A good opening line answers three questions in a few seconds: who is calling, from where, and why.
For example:
"Hi, am I speaking with Rahul? This is an AI assistant calling
from City Clinic about your appointment tomorrow at 10 AM."
Here, we can see that the agent:
- Confirms it is talking to the right person.
- Says that it is an AI assistant. Some places require this by law, and it also builds trust.
- Says the reason for the call immediately.
To make the opening fast, this first line can be generated and turned into audio before the call even connects. So, the moment AMD says human, the agent can start speaking with no delay.
Step 6: Running the conversation
From here, the call works like any other voice agent conversation. The audio goes through the voice pipeline as below:
+------------------------------+
| Person speaks on the phone |
+------------------------------+
↓
+------------------------------+
| Speech-to-Text (STT) |
+------------------------------+
↓
+------------------------------+
| LLM decides the reply |
+------------------------------+
↓
+------------------------------+
| Text-to-Speech (TTS) |
+------------------------------+
↓
+------------------------------+
| Agent's voice on the phone |
+------------------------------+
Here, we have:
- Speech-to-Text (STT): turns the person's voice into text.
- LLM (Large Language Model): reads the text, along with the instructions and the contact details from Step 1, and decides what to say next.
- Text-to-Speech (TTS): turns the reply into a voice and sends it down the phone line.
Some systems replace these three parts with a single speech-to-speech model, and the rest of the call flow stays the same.
The agent also needs tools to get real work done. For example, if the patient says, "Can I move it to Friday?", the LLM recommends calling a tool like reschedule_appointment with the new day, and our code runs it on the clinic's booking system. Then, the agent tells the patient the result.
A few situations come up often in outbound calls, and the agent must handle them:
- Wrong person: "Rahul is not home." The agent can ask for a better time to call back.
- Not interested: the agent must end the call politely and mark the person so that we do not call again.
- Interruptions: people often cut in with "Yes, yes, I know." The agent must stop speaking and listen.
- Transfer to a human: if the person asks for a real person, or the problem is too complex, the agent hands the call over to a human team member. When the human first gets a short summary of the call before being connected, it is called a warm transfer, so the person does not have to repeat everything.
To learn Tool use in Agents, Agent Architecture, and how to Design a Real-Time Voice AI Agent, and to build an AI Coding Agent from scratch, check out our AI and Machine Learning Program at Outcome School.
Step 7: Ending the call and saving the outcome
Every call must end with a saved result. This result is called the call outcome or disposition.
Some common call outcomes are as below:
confirmed: the patient confirmed the appointment.rescheduled: the appointment was moved to a new time.not_interested: the person said no.callback_requested: the person asked us to call at another time.voicemail_left: a voicemail picked up and we left a message.no_answerorbusy: nobody talked to us.transferred: the call was handed to a human.
For no_answer, busy, and voicemail_left, the outcome comes straight from Step 3 and Step 4. For a real conversation, the LLM is often asked at the end of the call to read the full transcript and pick one outcome from a fixed list. It also pulls out useful details, as below:
{ "outcome": "rescheduled", "new_time": "Friday 11 AM", "notes": "Prefers mornings" }
Here, we can see that the outcome is in a fixed format that our code can read and save into the clinic's system.
Now, the outcome decides what the dialer does next, as below:
- For
no_answer,busy, orvoicemail_left, the dialer schedules a retry, often a few hours later or on another day, up to a maximum number of tries. - For
callback_requested, the dialer schedules a call at the time the person asked for. - For
confirmed,rescheduled, ornot_interested, the contact is marked as done and is never called again for this campaign.
The system also saves the recording and the transcript, which teams use to check the quality of calls and to improve the agent's instructions.
Stay updated: Subscribe to our newsletter to get our latest AI and Machine Learning blogs straight to your inbox.
Where outbound calling voice agents are used
Outbound calling voice agents are used wherever a business needs to make many similar phone calls. Some common use cases are as below:
- Appointment reminders: clinics, salons, and service centers confirming or rescheduling bookings.
- Payment reminders: reminding customers about a due bill and sending them a link to pay.
- Delivery updates: confirming an address or a delivery time slot.
- Lead follow-up: calling people who filled a form on a website and asked to be contacted.
- Surveys and feedback: asking a few questions after a purchase or a service visit.
Now we must have understood what an outbound calling voice agent is, how it decides whom to call, how the call reaches a phone, how it tells a human from a voicemail, how it runs the conversation, and how every call ends with a saved outcome and a retry plan.
Frequently Asked Questions
Does an outbound calling voice agent tell people it is an AI?
Yes, a good one does. Its opening line says who is calling, from where, and why, and it says that it is an AI assistant. Some places require this by law, and it also builds trust with the person who picked up the call.
How many calls can an outbound calling voice agent run at the same time?
It can run many calls at the same time, but the dialer limits how many run in parallel. Each live call needs its own speech recognition, LLM, and speech generation running at the same time, and the telephony account often has a limit on how many calls can run at once.
What is a voicemail drop?
A voicemail drop is a short, pre-written message that the agent leaves when a voicemail picks up. The agent waits for the voicemail greeting to end, with a beep or with silence, and then plays the message, so that the full message gets recorded.
Can an outbound calling voice agent transfer the call to a human?
Yes. If the person asks for a real person, or the problem is too complex, the agent hands the call over to a human team member. In a warm transfer, the human first gets a short summary of the call, so the person does not have to repeat everything.
That's it for now.
Thanks
Amit Shekhar
Founder @ Outcome School
You can connect with me on:
Follow Outcome School on:
