AI Agent Blogs
How do half-duplex and full-duplex voice agents differ?
A half-duplex voice agent takes turns, which means it either listens or speaks, but never does both at once. A full-duplex voice agent listens and speaks at the same time, just like two people on a phone call.
Why does a voice agent cut people off mid-sentence?
A voice agent cuts people off mid-sentence because it often decides that a person has finished talking just by noticing a short silence, and people pause in the middle of their sentences all the time. When the agent hears that pause, it wrongly thinks it is its turn to speak, and it starts replying while the person still has more to say.
How does a voice agent know a phone number or email is not finished yet?
A voice agent knows that a phone number or email is not finished yet by combining a few signals: how long the person has been silent, what question the agent just asked, and whether the words heard so far look like a complete phone number or email. If the value still looks incomplete, the agent keeps listening instead of replying.
How does a voice agent handle long pauses when the user is thinking?
A voice agent handles long pauses by not treating every silence as the end of the user's turn. It checks the words, the voice, and the time together, and it waits longer when the user sounds like they are still thinking.
How does dynamic endpointing adapt to how fast a user talks?
Dynamic endpointing adapts to how fast a user talks by learning the length of that user's normal pauses and then changing how long the voice assistant waits in silence before it decides that the user has finished speaking. A fast talker gets a short wait, and a slow talker gets a long wait.
How does endpointing decide when the user has finished speaking?
Endpointing is the part of a voice assistant that decides when the user has finished speaking, so that the assistant knows when to stop listening and start replying. It does this by watching for silence, listening to how the user speaks, and reading what the user has said so far.
How do silence-based and model-based turn detection differ?
Silence-based turn detection decides that a person has finished speaking when it hears a long enough pause. Model-based turn detection uses a trained model that looks at what was said and how it was said to decide whether the person is done.
How does a voice agent hide the delay of a slow tool call?
A voice agent hides the delay of a slow tool call by never letting the line go silent. It speaks a short line while the tool runs, gives progress updates during a long wait, keeps the conversation going while the tool works in the background, and starts the tool early so that there is less delay to hide.
How do filler phrases hide latency in a voice agent?
Filler phrases hide latency in a voice agent by filling the silence with a short, natural line like "Let me check that for you" while the agent is still working on the real answer, so the user hears a reply right away instead of waiting in silence.
How does speculative LLM generation start replying before the user finishes?
Speculative LLM generation is a technique where a voice agent starts preparing its reply while the user is still finishing their sentence, keeps that reply hidden, and plays it only if the user's final words match what the reply was based on.
How does sentence chunking feed LLM output into TTS faster?
Sentence chunking feeds LLM output into TTS faster by cutting the reply into sentences while the LLM is still writing it, and sending each finished sentence to the TTS right away, so the voice starts speaking after the first sentence instead of after the full reply.
Why does the time-to-first-audio matter more than total response time?
Time-to-first-audio matters more than total response time because a person on a voice call feels only the silence before the agent starts speaking. Once the agent starts speaking, the person is busy listening, and the rest of the reply can keep being prepared in the background.
How does the voice-to-voice latency of a voice agent get measured?
Voice-to-voice latency of a voice agent gets measured as the time from the moment the user stops speaking to the moment the user hears the first sound of the agent's reply. We measure it by recording both voices on one timeline, finding the end of the user's speech and the start of the agent's speech, and subtracting the two.
Cascaded vs Speech-to-Speech: How to choose the right voice agent architecture?
Cascaded vs Speech-to-Speech is the choice between building a voice agent as a chain of three models (speech-to-text, then an LLM, then text-to-speech) or as one model that hears audio and speaks audio directly. We choose based on what matters most for our use case: control and accuracy, or speed and natural sound.
How do you build a latency budget for a voice AI agent?
A latency budget for a voice AI agent is a plan that splits the total waiting time we allow for a reply into smaller pieces, one for each step of the agent, so that the agent always starts replying fast enough to feel natural.
What is barge-in in a voice AI agent?
Barge-in in a voice AI agent is the ability of the user to interrupt the agent while it is speaking, and the ability of the agent to stop talking right away and listen.
What is turn-taking in a voice AI agent?
Turn-taking in a voice AI agent is the skill of knowing when the user has finished speaking, when the agent should start speaking, and when the agent should stop because the user wants to talk again.
How does ChatGPT remember you?
ChatGPT remembers you by saving small notes about you outside the AI model and adding those notes back into the conversation whenever you chat. The model itself does not change when you chat with it, it reads your notes before it replies.
How do Browser Agents work?
Browser Agents are AI agents that use a web browser like a human does. They look at a web page, decide what to click or type, do it, look at the new page, and repeat until the task is finished.
The Future of Software Development
The Future of Software Development is a world where AI models become so smart, so fast, and so cheap that we simply tell the computer what we want, and AI agents build, test, and ship it for us. The work of software engineers moves from writing every line of code to building, checking, and improving the systems around these AI models.
Design a Real-Time Voice AI Agent
A Real-Time Voice AI Agent listens to us, understands, thinks, takes actions, and talks back in a natural voice within a fraction of a second. Let's design one.
What is Graph Engineering?
Graph Engineering is designing an AI system as a graph, where every step is a node and every path between steps is an edge, instead of one giant prompt.
What is Loop Engineering?
Loop Engineering is designing the repeating cycle that an AI agent runs, so that it keeps making real progress on a task and stops at the right moment.
What are Agent Skills?
An Agent Skill is a folder of instructions, and optionally scripts, that an AI agent loads by itself only when the task needs it. Let's see how Skills work.
What is MCP (Model Context Protocol)?
MCP (Model Context Protocol) is an open standard that defines one common way for AI applications to connect to outside tools and data. It was created by Anthropic.
What is OKF (Open Knowledge Format)?
OKF (Open Knowledge Format) is an open standard for writing what an organization knows about its data as plain markdown files that any AI agent can read.
How does LangGraph work?
LangGraph is a framework to build LLM applications where the work is organized as a graph of steps, with nodes, edges, and a shared state.
How does LangChain work?
LangChain is a framework that helps us build LLM applications by connecting the LLM with our data, tools, and logic through chains, memory, and agents.
How do Computer-Use Agents work?
A computer-use agent is an AI program that operates a computer like a human. It looks at the screen, moves the mouse, clicks, and types to finish our goal.
What is Sakana Fugu? The Technical Report Explained
Sakana Fugu is a family of AI models that work like a conductor. They decide which other AI models should work on our question and combine their answers.
How does Function Calling work in LLMs?
Function Calling lets an LLM use external tools and APIs. The model does not run the function itself. It tells our code which function to call and with what inputs.
What is an AI Agent Loop?
The AI Agent Loop is the code that runs an AI Agent. It asks the LLM what to do, runs that action, feeds back the result, and repeats until the task is done.
What is AI Agent Observability? Traces, Spans, and Metrics Explained
AI Agent Observability means recording every step an AI Agent takes, like each thought and tool call, so that we can see why it behaved the way it did.
How AI Agents Communicate
AI agents communicate by sending messages to each other to share information, ask for help, and work together. We will learn the main ways and protocols.
What are AI SubAgents? Why We Need Them and How They Work
An AI SubAgent is a smaller, specialized agent that works under a main agent to handle one part of a big task, like a team member under a manager.
How to Evaluate AI Agents? Metrics, Methods, and Best Practices
AI Agent Evaluation is the process of measuring how well an AI Agent performs a task, by checking the final result, the steps it took, and the tools it used.
What is AI Orchestration? How It Works and the Common Patterns
AI Orchestration is the process of coordinating many AI parts, like LLMs, tools, data sources, and agents, so that they work together to finish a complex task.
What is Context Engineering?
Context Engineering is the practice of deciding what information goes into an LLM's context window, and how, so that the model can do its task reliably.
What is a Reflection Agent? How It Generates, Critiques, and Revises
A Reflection Agent is an AI Agent that writes a draft, critiques its own draft, and then writes a better version using that critique, again and again.
What is GraphRAG? How Knowledge Graphs Improve RAG
GraphRAG is RAG that uses a knowledge graph along with vector search to find better, more connected information before the LLM answers a question.
What is a Plan-and-Execute Agent and How Does It Work?
A Plan-and-Execute Agent is an AI Agent that first writes the full plan up front and then runs the plan step by step. Let's see how it differs from ReAct.
What is Agentic RAG? How It Works and When to Use It
Agentic RAG is a RAG system where an AI Agent decides when to search, what to search, and when to stop, instead of doing one fixed search before answering.
What is a ReAct Agent? How It Thinks and Acts, Explained
A ReAct Agent is an AI Agent built using the ReAct (Reasoning + Acting) pattern. It runs in a loop where the LLM thinks, takes an action, and observes the result.
What are Multi-Agent Systems and When Should We Use Them?
A Multi-Agent System is a setup where two or more LLM-driven agents, each with its own prompt and tools, work together on a shared task. Let's see when to use one.
How Does AI Agent Memory Work? The Memory Stack Explained
AI Agent Memory is the system that lets an LLM remember things across messages, sessions, and users. We will learn the memory stack and how it works.
What is an AI Agent? How It Works
An AI Agent is a system where an LLM takes a goal, decides the steps, uses tools, and keeps working in a loop until the goal is done. We will learn how it works.
What is Harness Engineering?
Harness Engineering is building the layer of code around an AI model that manages its inputs, outputs, tools, memory, errors, and evaluation for production.