How does a voice agent transfer a call to a human with context?
- Authors
- Name
- Amit Shekhar
- Published on
A voice agent transfers a call to a human with context by deciding that a human is needed, writing a short summary of the call, and moving the call and that summary to the human together, so that the caller never has to repeat the story.
In this blog, we will learn about how a voice agent transfers a call to a human with context. We will also see why a voice agent needs to hand off a call at all, how it decides the right moment to transfer, what context it must pass along, the difference between a cold transfer and a warm transfer, how the call and the context travel together, and what happens when no human is free to take the call.
We will cover the following:
- The problem: transferring a call without context
- What is a call transfer with context?
- When does a voice agent decide to transfer?
- What context must travel with the call?
- Cold transfer vs Warm transfer
- How do the call and the context travel together?
- Putting it all together in code
- What if no human is available?
I am Amit Shekhar, Founder @ Outcome School, I have taught and mentored many developers, and their efforts landed them high-paying tech jobs, helped many tech companies in solving their unique problems, and created many open-source libraries being used by top companies. I am passionate about sharing knowledge through open-source, blogs, and videos.
I teach AI and Machine Learning at Outcome School.
Let's get started.
The problem: transferring a call without context
A voice agent is an AI agent that we talk to on a phone call. It listens to what we say, turns it into text, lets an LLM (Large Language Model) decide the reply, and speaks the reply back to us.
A voice agent can handle many calls on its own, like checking an order or booking an appointment. But some calls need a human.
Let's say a caller named Riya calls her bank. She talks to the voice agent for three minutes. She gives her name, her account number, and explains that a payment of 5,000 rupees left her account twice. The agent verifies her identity and checks the transactions. Then, the agent decides that this refund needs a human.
Now, the agent transfers the call. A human agent named Rahul picks up and says:
"Hello, thank you for calling. Can I have your name and account number, please?"
Riya has to repeat her name, her account number, and the whole story. Three minutes of her time are wasted, and she is now more upset than before.
This is the problem. The call moved to a human, but the information did not move with it.
Phone systems have been transferring calls for decades. So, the real question is: how do we also transfer what the caller already told us?
What is a call transfer with context?
A call transfer with context is a handoff where the voice agent moves the live call to a human agent and, at the same time, gives that human everything the caller has already shared, so that the human can continue the conversation instead of restarting it.
Here, the word "context" means the information gathered during the call: who the caller is, what the problem is, what the agent has already checked, and what the caller wants.
With context, the same call goes like below:
Rahul picks up the call. Before he speaks, his screen already shows: "Riya Sharma, identity verified, payment of 5,000 rupees charged twice on 12 October, wants a refund of the second payment." Rahul says:
"Hi Riya, I can see that the payment of 5,000 rupees was charged twice. Let me process the refund for the second one right away."
Riya does not repeat anything. The call feels like one conversation, even though it moved from an AI to a human.
A transfer with context has three parts:
- Decide: The agent decides that a human is needed.
- Package: The agent collects the context and writes a short summary.
- Hand off: The agent moves the call and the context to the human together.
Now, let's understand each part one by one.
When does a voice agent decide to transfer?
If the agent transfers too early, humans get calls that the agent could have handled. If it transfers too late, the caller gets frustrated.
A voice agent usually transfers the call in the following situations:
- The caller asks for a human. The caller says "I want to talk to a person". This is the clearest signal, and the agent must respect it.
- The task is outside what the agent can do. The company does not let the agent approve refunds, change legal details, or handle a complaint about fraud.
- The agent is stuck. The agent has failed to understand the caller two or three times in a row.
- The caller is upset. The caller's words, or in some systems the tone of voice, show anger or distress, and a human can handle the situation better.
- A business rule says so. For example, every call about closing an account must go to a human.
We have a detailed blog on Human-in-the-Loop agents that explains how an agent hands control to a human at the right moments.
Now, the question is: how does the agent make this decision?
The answer is that we give the LLM a tool named something like transfer_to_human. In the system prompt, we describe when this tool must be used. When the LLM sees one of these situations, it recommends calling this tool along with a reason.
Note: The LLM never transfers the call by itself. It only recommends the tool and its inputs, and our code performs the actual transfer.
Some rules are too important to leave to the LLM. For example, "the agent failed to understand the caller three times" or "the caller mentioned fraud". For these, our code checks the rule directly and triggers the transfer, even if the LLM did not recommend it. This way, the most important rules never depend on the LLM alone.
Planning your move into AI?
No matter which tech domain you work in today, with your effort, you can transition to these roles:
- AI Engineer
- Forward Deployed Engineer
- AI Agent Engineer
- Harness Engineer
- Voice AI Engineer
- AI Architect
- LLM Engineer
- LLM Inference Engineer
- Gen AI Engineer
- MLOps Engineer
- ML Engineer
- Deep Learning Engineer
Our AI and Machine Learning Program at Outcome School takes you there.
What context must travel with the call?
Now that the agent has decided to transfer, it needs to package the context. The human agent has only a few seconds to read before saying hello, so the context must be short and useful.
A good context package contains the following:
- Caller identity: The caller's name, phone number, and account number.
- Verification status: Whether the agent has already verified the caller's identity. This saves the human from asking the security questions again.
- Reason for transfer: Why the call came to a human. For example, "refund needs human approval" or "caller asked for a human".
- Summary of the problem: Two or three sentences about what the caller wants.
- What has been done: What the agent has already checked or tried. For example, "found two payments of 5,000 rupees on 12 October".
- Caller mood: Whether the caller is calm or upset.
- Full transcript: The complete text of the conversation, kept as a backup.
Here, the most important item is the summary. A transcript of three minutes is too long to read in a few seconds. So, the agent asks the LLM to turn it into a short summary.
We can write the prompt for the summary as below:
You are preparing a handoff note for a human support agent.
Read the call transcript below and write a short note with:
- The caller's main problem in one sentence
- What has already been checked or done
- What the caller wants next
Keep it under 60 words. Use only facts from the transcript.
Transcript:
{transcript}
Here, we have asked the LLM to use only facts from the transcript, because the human will trust this note. If the LLM adds a detail that the caller never said, the human will act on wrong information.
For the same reason, facts like the account number and the verification status come from our own system, not from the LLM. The LLM only writes the summary.
The final context package looks like below:
{
"caller_name": "Riya Sharma",
"phone": "+91-98xxxxxx10",
"account_id": "ACC-48213",
"identity_verified": true,
"transfer_reason": "Refund of duplicate payment needs human approval",
"summary": "Payment of 5,000 rupees was charged twice on 12 October. Agent confirmed both charges. Caller wants the second charge refunded.",
"caller_mood": "frustrated",
"transcript_id": "call-7731"
}
Here, the full transcript is not inside the package. Only its ID is there, so that the human's screen can load it if needed.
To learn LLM Fundamentals, Prompt Engineering, and Context Engineering in depth, check out our AI and Machine Learning Program at Outcome School.
Cold transfer vs Warm transfer
Now, it's time to learn about the two ways to hand off the call.
Cold transfer
A cold transfer (also called a blind transfer) is when the voice agent sends the call to a human or a queue and leaves the call right away, without talking to the human first.
Let's say the agent tells Riya: "I am connecting you to a specialist now." Then, it sends the call to the refunds team and drops out. The next free human picks up the call.
Advantage: It is fast and simple. The caller reaches the queue without any extra wait.
Disadvantage: Nobody checks that the human has read the context. If the context did not reach the human's screen, the caller has to repeat everything.
Warm transfer
The issue with a cold transfer is that nobody confirms the handoff. Here comes the warm transfer into the picture.
A warm transfer (also called an attended transfer) is when the voice agent first connects to the human, briefs them about the call, and only then connects the caller to the human.
A warm transfer works in the following steps:
Step 1: The agent tells the caller: "Please hold for a moment while I connect you to a specialist." The caller hears hold music.
Step 2: The agent places a new call to a human agent.
Step 3: When the human picks up, the agent speaks a short briefing: "I have Riya Sharma on the line. Her identity is verified. A payment of 5,000 rupees was charged twice, and she wants the second one refunded."
This spoken briefing is often called a whisper, because only the human hears it, and the caller does not.
Step 4: The agent connects the caller and the human into one call.
Step 5: The agent leaves the call. Now, only Riya and Rahul are talking.
Advantage: The human hears the context before talking to the caller, and the caller hears that the human already knows the story.
Disadvantage: It takes longer, and the caller waits on hold during the briefing. It also needs a human to be free right now.
Let me tabulate the differences between Cold transfer and Warm transfer for your better understanding so that you can decide which one to use based on your use case.
| Point | Cold transfer | Warm transfer |
|---|---|---|
| Agent talks to human first | No | Yes |
| Speed | Fast | Slower |
| Caller waits on hold | Only in the queue | During the briefing too |
| Context delivery | Only on the human's screen | On the screen and spoken |
| Risk of caller repeating | Higher | Lower |
| Best for | Simple calls, big queues | Complex or sensitive calls |
In practice, many teams use a cold transfer to a queue for simple calls, and a warm transfer for upset callers or high-value cases.
A quick note for you
No matter which tech domain you work in, get familiar with these topics:
- LLM
- RAG
- MCP
- Agent
- Fine-tuning
- Quantization
We put it all together in one video:
AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, and Quantization
No need to stop reading - bookmark it and watch later when you get time. Future you will thank you.
Now, let's get back to the topic.
How do the call and the context travel together?
Now, the next big question is: how does the context reach the right human at the same moment as the call?
The call and the context travel on two different paths.
- The call is audio. It travels through the phone system.
- The context is data. It travels through our software, like the CRM (Customer Relationship Management) system that the human agents use.
If these two paths are not linked, Rahul picks up the call, but his screen shows nothing or shows the details of some other caller.
So, we need a way to link the call and the context. There are two common ways to do this.
Way 1: Send the context inside the call
Phone calls over the internet often use a protocol called SIP (Session Initiation Protocol). SIP is the set of rules that phone systems use to set up, move, and end calls.
SIP messages carry headers, which are small labeled pieces of information attached to the call. The voice agent can attach a small piece of data to the call in a header. For example, it can attach the account ID and the transfer reason. Some systems use custom headers that start with X-, and there is also a standard header named User-to-User built for passing such data along with a call.
When the call reaches the human's phone system, the system reads the header and opens the right caller details on the human's screen. This is called a screen pop, because the caller's details pop up on the screen when the call arrives.
SIP also has a method named REFER, which is commonly used for a cold transfer. With REFER, the voice agent tells the phone system "please move this call to this other destination", and then steps out.
Way 2: Store the context and look it up
The second way is to store the context package in a database, using the call ID as the key. When the call reaches the human, the human's software looks up the call ID and loads the package.
Here, the header needs to carry the call ID, not the whole summary. Headers have size limits, so a short ID is safer than a long text.
Most real systems combine both ways: a small ID travels inside the call, and the full context waits in the database.
We can see it in the diagram below:
+--------------------------------+
| Voice Agent |
+--------------------------------+
| |
↓ ↓
call + ID context package
| |
↓ ↓
+--------------+ +--------------+
| Phone System | | Database |
+--------------+ +--------------+
| |
↓ ↓
+--------------------------------+
| Human agent: call + screen pop |
+--------------------------------+
Here, the human's software uses the ID from the call to load the matching package from the database.
Stay updated: Subscribe to our newsletter to get our latest AI and Machine Learning blogs straight to your inbox.
Putting it all together in code
Now, let's see the code for a simple transfer flow. Here, I have kept only the important lines of the code and removed other lines for brevity.
First, we define the tool that the LLM can recommend, as below:
transfer_tool = {
"name": "transfer_to_human",
"description": "Transfer the call to a human agent. Use when the caller "
"asks for a human, is upset, or needs something you cannot do.",
"parameters": {
"type": "object",
"properties": {
"reason": { "type": "string" },
"team": { "type": "string", "enum": ["refunds", "fraud", "general"] }
},
"required": ["reason", "team"]
}
}
Here, we have described the tool and its two inputs: the reason and the team that must receive the call.
Then, we write the function that our code runs when the LLM recommends this tool, as below:
async def handle_transfer(call, reason, team):
# build the context package
summary = await llm_summarize(call.transcript)
context = {
"caller_name": call.customer.name,
"account_id": call.customer.account_id,
"identity_verified": call.customer.verified,
"transfer_reason": reason,
"summary": summary,
"transcript_id": call.id,
}
# store the context so the human's screen can load it
await context_store.save(call.id, context)
# no human is free, so offer a callback instead
if not await has_free_agent(team):
return await offer_callback(call, context)
# tell the caller what is happening
await call.say("I am connecting you to a specialist. Please hold for a moment.")
# move the call and attach the context ID
await call.transfer(
destination=team_queue(team),
headers={ "X-Context-Id": call.id },
)
Here, we can see the three parts that we discussed:
- Decide: The LLM recommended
transfer_to_humanwith a reason and a team. - Package: We combined a short summary from the LLM with facts from our own system.
- Hand off: We saved the context and transferred the call with the call ID in a header.
This code does a cold transfer to a team queue, with the context waiting on the human's screen.
If no human in the team is free, the code offers a callback instead.
One more thing to notice is the order. We save the context before we transfer the call. If we transfer first, the call can reach the human before the context is saved, and the screen will show nothing.
If we want to go deep into Tool use in Agents, Agent Architecture, and how to Design a Real-Time Voice AI Agent, and to build an AI Coding Agent from scratch, we can check out our AI and Machine Learning Program at Outcome School.
What if no human is available?
Sometimes, all human agents are busy, or the team is offline at night. So, before the transfer, our code checks whether any human in that team is free.
If nobody is free, the voice agent can do the following:
- Tell the caller the wait time. "All our specialists are busy. The expected wait time is about 6 minutes."
- Offer a callback. The agent books a callback, saves the context with it, and the human who calls back later sees the same summary.
- Create a ticket. For offline hours, the agent creates a support ticket with the full context, so that the team can pick it up the next morning.
In every case, the context is saved first. So, whenever a human finally talks to the caller, the human starts from where the agent stopped.
Now we must have understood how a voice agent transfers a call to a human with context, how it decides when to transfer, what it sends along, and how the call and the context reach the human together.
Frequently Asked Questions
Does the LLM transfer the call by itself?
No. The LLM never transfers the call by itself. It only recommends a tool like transfer_to_human along with a reason, and our code performs the actual transfer. For the most important rules, like a caller mentioning fraud, our code checks the rule directly and triggers the transfer even if the LLM did not recommend it.
Why should the account number and verification status not come from the LLM?
Because the human agent will trust the handoff note and act on it. If the LLM adds a detail that the caller never said, the human acts on wrong information. So, facts like the account number and the verification status come from our own system, and the LLM only writes the short summary.
What is a whisper in a warm transfer?
A whisper is the short spoken briefing that the voice agent gives the human agent during a warm transfer. Only the human hears it, and the caller does not. For example, the agent says who is on the line, that the identity is verified, and what the caller wants, before connecting the caller.
Why is the context saved before the call is transferred?
Because the call can reach the human before the context is saved if we transfer first, and then the human's screen will show nothing. Saving the context first means that whenever a human picks up the call, or calls back later, the matching summary is already waiting.
That's it for now.
Thanks
Amit Shekhar
Founder @ Outcome School
You can connect with me on:
Follow Outcome School on:
