How does DTMF keypad input work in a voice agent?
- Authors
- Name
- Amit Shekhar
- Published on
DTMF keypad input works by turning every key we press on a phone into a short sound made of two tones, which the phone system or the voice agent detects and turns back into the exact digit, so that the agent can use it just like a spoken answer.
In this blog, we will learn about how DTMF keypad input works in a voice agent. We will also see what DTMF is, why every key is made of two tones, how those tones travel and get detected, how the agent collects the digits and passes them to the LLM, how it keeps sensitive digits private, and how the agent presses keys by itself.
We will cover the following:
- What is DTMF?
- Why does a voice agent need keypad input?
- The keypad grid: two tones for every key
- How DTMF travels over a phone call
- How a voice agent detects the tones
- Collecting the digits: terminator key and timeout
- How the digits reach the LLM
- Keeping sensitive digits away from the LLM
- When the voice agent presses the keys
I am Amit Shekhar, Founder @ Outcome School, I have taught and mentored many developers, and their efforts landed them high-paying tech jobs, helped many tech companies in solving their unique problems, and created many open-source libraries being used by top companies. I am passionate about sharing knowledge through open-source, blogs, and videos.
I teach AI and Machine Learning at Outcome School.
Let's get started.
What is DTMF?
We all have heard this on a phone call: "Press 1 for billing. Press 2 for support." We press a key, we hear a short beep, and the system understands our choice.
That beep is DTMF.
DTMF stands for Dual-Tone Multi-Frequency. It is the system that a phone uses to send a pressed key as a sound made of two tones played at the same time.
Let's decompose the name:
- Dual-Tone: every key makes two tones together, not one.
- Multi-Frequency: there is a fixed set of different pitches (frequencies), and each key uses its own pair from that set.
A frequency is simply how high or low a sound is. A low hum has a low frequency. A whistle has a high frequency. Frequency is measured in Hertz (Hz), which means the number of times the sound wave goes up and down in one second.
DTMF was developed by the Bell System and launched for the public in 1963 under the name "Touch-Tone". It is still supported by phone networks today.
Why does a voice agent need keypad input?
A voice agent is a system built around an LLM (Large Language Model) that talks with us over a call. It listens to our voice, converts it into text using Speech-to-Text, sends that text to the LLM to decide what to say, and speaks the reply using Text-to-Speech.
So, the question is: if the agent can understand speech, why does it need the keypad at all?
The answer is that speech is not always the best way to give some kinds of input. Let's see the cases:
- Long numbers: An account number like "4 8 1 5 1 6 2 3 4 2" is easy to type but easy to mishear. Speech-to-Text can confuse "fifteen" with "fifty", or drop one digit in a noisy room.
- Private information: A PIN or the last four digits of a card must not be spoken out loud in a train or an office where others can hear.
- Noisy places: On a busy road, the microphone picks up traffic, and speech recognition struggles. A key press is much less affected by the noise around us.
- Accents and speech difficulties: Some callers find it easier to press a key than to say a word clearly enough for the system.
- Old phone menus: Many businesses already have menus built around "Press 1, Press 2", and callers are used to them. A voice agent that replaces such a menu often keeps keypad input working.
A key press gives one exact digit, with no guessing. This is why a voice agent still needs keypad input even when it understands speech well.
The keypad grid: two tones for every key
Now, let's understand how a key becomes a sound.
A phone keypad is a grid of 4 rows and 3 columns. The original DTMF design has one more column with the keys A, B, C, and D, which ordinary phones do not show, so the full grid is 4 rows and 4 columns.
Every row has its own low tone, and every column has its own high tone. When we press a key, the phone plays the low tone of its row and the high tone of its column at the same time.
| Row / Column | 1209 Hz | 1336 Hz | 1477 Hz | 1633 Hz |
|---|---|---|---|---|
| 697 Hz | 1 | 2 | 3 | A |
| 770 Hz | 4 | 5 | 6 | B |
| 852 Hz | 7 | 8 | 9 | C |
| 941 Hz | * | 0 | # | D |
Let's take an example. Suppose we press the key 5.
- The key 5 sits in the second row, so the phone plays the 770 Hz tone.
- The key 5 sits in the second column, so the phone also plays the 1336 Hz tone.
- Both tones play together, and we hear one short beep.
Similarly, pressing 9 plays 852 Hz and 1477 Hz together.
Now, the question is: why two tones? Why not one tone per key?
The answer is safety against mistakes. A human voice or music can easily produce one pure tone by accident.
But, it is very rare for normal speech to produce exactly one tone from the low group and exactly one tone from the high group, both at a steady loudness, at the same time.
The pitches are also chosen so that no DTMF tone is a simple multiple of another, which makes it even harder for a voice to produce a matching pair by accident. This is how the system avoids treating a word we say as a key press.
Planning your move into AI?
No matter which tech domain you work in today, with your effort, you can transition to these roles:
- AI Engineer
- Forward Deployed Engineer
- AI Agent Engineer
- Harness Engineer
- Voice AI Engineer
- AI Architect
- LLM Engineer
- LLM Inference Engineer
- Gen AI Engineer
- MLOps Engineer
- ML Engineer
- Deep Learning Engineer
Our AI and Machine Learning Program at Outcome School takes you there.
How DTMF travels over a phone call
Now that we know what the tones are, it's time to learn how they reach the voice agent.
A voice agent usually connects to real phone calls through a telephony provider, which is the service that links the phone network to our agent code. A key press can reach the agent in two main ways.
Way 1: In-band (inside the audio)
The beep travels inside the call audio, mixed with our voice. The agent receives the audio and has to listen for the two tones.
The issue with this approach is compression. Internet calls and mobile calls often squeeze the audio using a codec, which is a method to make the audio smaller so that it uses less network data. Codecs are built to keep human speech clear, not to keep pure tones exact. Some codecs change the tones so much that the detector misses the key or reads the wrong one. Let's see how the next approach solve this issue.
Way 2: Out-of-band (as a separate event)
Here, the phone system detects the key press close to the caller and sends it as a small message that says "the key 5 was pressed, and it was held for this long". The audio is not needed to carry the beep at all.
On internet calls, the most common standard for this is RFC 4733, which sends key presses as special "telephone-event" packets alongside the audio packets.
The flow looks like below:
Caller presses the key 5
↓
Phone system detects the key
↓
Sends event: digit = 5
↓
Telephony provider
↓
Voice agent receives "5"
Out-of-band is much more reliable because the digit is no longer a sound that can get damaged. This is why most telephony providers deliver key presses to the agent as events.
For example, when a provider streams the call audio to our agent over a WebSocket, which is a connection that stays open in both directions, it sends audio messages, and whenever the caller presses a key, it sends a separate message like below:
{ "event": "dtmf", "dtmf": { "digit": "5" } }
Here, I have kept only the important fields and removed other fields for brevity. We can see that the agent does not need to listen for any beep. The provider tells the agent the exact digit.
How a voice agent detects the tones
But, sometimes the key presses arrive only inside the audio, for example when a call passes through an old network that does not send events, or when the agent receives only the raw audio of the call. In that case, the voice agent has to detect the tones by itself.
The best way to learn this is by taking an example.
Suppose the agent receives a small slice of audio, around a few tens of milliseconds long. The agent wants to answer one question: "Is there a DTMF key in this slice, and which one?"
The detector does the following:
Step 1: It measures how strong each of the 8 DTMF frequencies (4 row tones and 4 column tones) is in the slice. A math method called the Goertzel algorithm is commonly used for this, because it checks only a few chosen frequencies instead of all of them, which makes it very fast.
Step 2: It picks the strongest tone from the low group and the strongest tone from the high group.
Step 3: It checks that these two tones are clearly stronger than everything else in the slice, and that their loudness is not too different from each other. This makes sure that noise or speech is not mistaken for a key.
Step 4: It checks that the same pair stays present for a minimum duration. A real key press lasts long enough, while a random sound from speech usually does not.
Step 5: It looks up the pair in the keypad grid. 770 Hz with 1336 Hz means the key 5.
Let's see this with sample values for the sake of understanding:
| Frequency | Strength |
|---|---|
| 697 Hz | weak |
| 770 Hz | strong |
| 852 Hz | weak |
| 941 Hz | weak |
| 1209 Hz | weak |
| 1336 Hz | strong |
| 1477 Hz | weak |
| 1633 Hz | weak |
Here, we can see that 770 Hz is the strongest row tone and 1336 Hz is the strongest column tone. Both are clearly above the rest. So, the detector reports the key 5.
One more thing to notice: when the user holds the key for a long time, the tone stays on for many slices. The detector must report the key only once, when the tone starts, and then wait for the tone to stop. Otherwise, one press of 5 becomes "5 5 5 5".
A quick note for you
No matter which tech domain you work in, get familiar with these topics:
- LLM
- RAG
- MCP
- Agent
- Fine-tuning
- Quantization
We put it all together in one video:
AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, and Quantization
No need to stop reading - bookmark it and watch later when you get time. Future you will thank you.
Now, let's get back to the topic.
Collecting the digits: terminator key and timeout
Now, the agent knows each key the moment it is pressed. But, a single digit is often not the full answer. An account number can have 10 digits. A PIN often has 4.
So, how does the agent know when the user has finished typing? The answer is that it uses a few rules.
- Terminator key: The agent says, "Enter your account number followed by the hash key." When the user presses
#, the agent knows the input is complete. - Fixed length: If the agent knows that the PIN is exactly 4 digits, it stops collecting after the 4th digit.
- Timeout: If the user stops pressing keys for a few seconds, the agent treats the input as complete and moves on.
Most agents use these together. Let's say the agent asks for a 10-digit account number:
- The user presses 4, 8, 1, 5 and then pauses for a long time. The timeout ends the input, and the agent can say, "I got only 4 digits. Please enter the full account number."
- The user presses all 10 digits and then
#. The input is complete. - The user presses all 10 digits and forgets
#. The agent stops at the 10th digit because it knows the length.
The * key often gets a special job too, like "press star to start again", which clears the digits collected so far.
There is one more important behavior. When the user presses a key while the agent is still speaking, the agent must stop speaking right away, because the key press shows that the user already knows what to do. This is the same idea as barge-in, where the user interrupts the agent by talking.
How the digits reach the LLM
Now, we have the complete digits. The next big question is: how does the LLM know about them?
In this kind of voice agent, the LLM works with text. So, the agent converts the collected digits into a short text message and adds it to the conversation, just like the written text of something the user said.
For example, the agent adds a line like below to the conversation:
User (keypad): 4815162342
Here, the label "keypad" tells the LLM that this came from key presses, not from speech. This matters because the LLM can trust a keypad input to be exact, while a spoken number can have errors.
Let's see the full flow:
User presses keys: 4 8 1 5 ... #
↓
Agent collects digits until #
↓
Agent builds text: "User (keypad): 4815162342"
↓
LLM reads the text and decides the next step
↓
LLM recommends a tool: look_up_account("4815162342")
↓
Agent runs the tool and gets the result
↓
Agent speaks the reply to the user
Here, we can notice that the LLM does not run the tool itself. The LLM recommends the tool and its input, and our agent code runs it. We have a detailed blog on how a voice agent hides the delay of a slow tool call that explains what the agent does while the tool runs.
A simple menu works the same way. If the agent says "Press 1 for billing, press 2 for support", and the user presses 2, the agent can send "User (keypad): 2" to the LLM, and the LLM understands that the user wants support.
To learn Tool use in Agents, Agent Architecture, and how to Design a Real-Time Voice AI Agent, from the keypad events to the LLM, check out our AI and Machine Learning Program at Outcome School.
Keeping sensitive digits away from the LLM
Some digits are private, like a card number, a PIN, or a security code. We do not want these digits to be stored in the conversation history, written into logs, or sent to an LLM at all.
So, for sensitive input, the agent does not pass the digits to the LLM. Instead, the agent code collects the digits and sends them directly to the secure system that needs them, like the payment service. The LLM only gets a result, for example:
System: the user entered the card number on the keypad, and it was verified
Here, the LLM knows that the step is complete and can continue the conversation, but it never sees the actual digits. This keeps private data out of the model, the transcripts, and the logs. Many payment systems also need this kind of separation to follow card security rules.
This is the reason a voice agent often asks for card numbers and PINs through the keypad instead of speech. Spoken digits go through Speech-to-Text and become text in the transcript. Key presses can be routed around the LLM completely.
Stay updated: Subscribe to our newsletter to get our latest AI and Machine Learning blogs straight to your inbox.
When the voice agent presses the keys
Now, it's time to learn about the opposite direction, where the agent presses the keys.
Sometimes, the voice agent itself calls another phone system. For example, an agent calls a company to check the status of an order, and the company's phone system says, "Press 3 for order status."
In this case, the agent must send DTMF. The LLM recommends which key to press, for example through a tool like press_keys("3"), and the agent code asks the telephony provider to send that key on the call. The provider then sends it either as tones in the audio or as an out-of-band event, exactly like a human pressing the key.
Now we must have understood how DTMF keypad input works in a voice agent, how the digits reach the LLM, and how sensitive digits stay away from it.
Frequently Asked Questions
What happens when the user holds a DTMF key down for a long time?
The voice agent still counts it as one key press. When a key is held, the tone stays on for many slices of audio, so the detector reports the key only once, when the tone starts, and then waits for the tone to stop. Otherwise, one press of 5 would become "5 5 5 5".
Can normal speech be mistaken for a DTMF key press?
Rarely. A voice can produce one pure tone by accident, but it is very rare for speech to produce exactly one tone from the low group and one from the high group, at a steady loudness, at the same time. The detector also checks that the pair is clearly stronger than everything else and lasts for a minimum duration.
What are the A, B, C, and D keys in DTMF?
They are the keys of a fourth column in the original DTMF design. This column uses the 1633 Hz high tone, so the full DTMF grid is 4 rows and 4 columns. Ordinary phones do not show these keys, so a normal phone keypad has only 4 rows and 3 columns.
Why do audio codecs break in-band DTMF?
Codecs squeeze the call audio so that it uses less network data, and they are built to keep human speech clear, not to keep pure tones exact. Some codecs change the tones so much that the detector misses the key or reads the wrong one. This is why most telephony providers send key presses as separate out-of-band events.
That's it for now.
Thanks
Amit Shekhar
Founder @ Outcome School
You can connect with me on:
Follow Outcome School on:
