Research
OpenYap Turn
A turn-taking model trained only on The Agentic Data Company's two-channel conversations. Our contribution to TurnBench.
Why turn-taking is hard
In conversation, people take turns without anyone counting them out. A speaker pauses to think and the listener waits. A speaker finishes and the listener comes in within a few hundred milliseconds. A listener says “mm-hm” without wanting the floor, or starts talking and takes it. A voice agent has to make the same calls from sound alone, as it happens, and silence is not enough: a pause inside a sentence and the gap after a finished one can be the same length. The two diagrams show the two confusions that matter most.
A pause inside a turn, and a real end of turn
Speaker A stops at the first gap but has not finished, and the system commits nothing. At the second gap A has finished and B takes over; the system commits an end of turn. The two gaps can be the same length, so silence alone cannot tell them apart.
A listener sound, and an interruption
B's short sound inside A's turn is a listener signal, not a bid for the floor. Later B starts while A is still talking and keeps going, and A stops; the system commits an interruption.
Get the first one wrong and the agent talks over a person who was still mid-sentence, or leaves a long gap after they finished. Get the second one wrong and the agent stops for every “yeah”, or ploughs on while the person is trying to take over. Both are judged within a few hundred milliseconds, from sound that is still arriving.
What it does
OpenYap Turn listens to a conversation between two people, one speaker per channel, and reports two kinds of event as they happen. An end of turn is the moment a speaker has finished and the floor is open. An interruption is a listener starting while the other is still talking and taking the floor. These are the two tasks of TurnBench [1], a public benchmark of turn-taking on recorded conversation.
Every event is reported causally: the decision uses only audio already heard, never what comes after. The model reads no transcript, uses no language model and does not know who is speaking. Its output is a time stamp per event, nothing more.
How it is scored
The benchmark holds a set of recorded conversations in which people have marked every end of turn and every interruption. A system listens to each conversation and emits its own time stamps. Each true event opens a window from a quarter of a second before it to three seconds after; the first call inside the window claims the event, and a stretch with no event can cost at most one false positive however many calls land in it. Recall is the share of true events claimed, the false-positive rate is the share of empty stretches charged, and latency is how long after the true event the claiming call came.
How one event is scored
a is the first call inside the window and claims the event: a hit. b and c add nothing either way.
d and e fall where no event is: one false positive for the stretch, however many calls land in it.
Two things follow. A system is ranked on recall only among those under the false-positive limit, so the two numbers are read together. And because a burst of calls in one window counts once, recall says whether the system noticed an event, not whether it made exactly one decision about it.
What it learned from
The learned part of the model is trained by self-supervision on a 1,984-hour slice of The Agentic Data Company's corpus, 2,500 of its conversations. They are real conversations between people who know each other: a speaker invites a friend or a family member, and the two of them talk, with no topic assigned and nothing scripted. Each is in their own place on their own microphone, so a call arrives as two tracks with one voice on each. That shape is what makes turn-taking available to learn. People who know each other interrupt more, backchannel more and leave shorter gaps, and with a track per speaker those overlaps stay separable instead of blurring into one waveform. Given both tracks, the network learns to forecast one kind of thing: the speakers' activity over the next two seconds, and readings derived from it. That activity is the whole training signal. No benchmark audio, no human turn labels and no transcripts go in. The benchmark's public development set was used only to choose the operating point, which is how listed systems set theirs.
- Real conversations between people who know each other: each speaker invites a friend or a family member, and the two of them talk.
- Nothing assigned: no topic, no prompt, no script.
- Each speaker in their own place on their own microphone, so a call arrives as two tracks with one voice on each.
- A 1,984-hour training slice, 2,500 conversations, drawn from a larger corpus.
- Self-supervised: the training target is the audio's own activity over the next two seconds.
- No benchmark audio, no human turn labels and no transcripts in training.
1,000 hours of the same corpus are public as Open Yap 1K, free for commercial and research use.
How it works
- 01
Voice activity
A voice-activity detector (Silero VAD) on each channel marks when that speaker is making sound.
- 02
Floor forecast
One two-channel transformer, 72.4 M parameters, predicts for every 20 ms frame how much of the next two seconds each speaker will be active, in coarse and fine bins, and whether each is speaking now. It also gives two derived readings, a takeover and a handover, all trained from the audio's own activity.
- 03
Commit rules
A small rule layer per task reads the detector and the same network and commits an event. Interruptions commit on the takeover reading from 0.15 s after a listener starts, or once the listener's voiced time reaches 0.71 s; ends of turn on the voice-activity rules plus the handover reading.
The forecast
The forecast is what voice activity cannot give. When a speaker goes quiet, the detector sees silence either way; the forecast says whether that speaker is likely to be back within the next two seconds or whether the floor is likely to pass. At every 20 ms step the network predicts, for each speaker, how much of the next two seconds that speaker will be active, as the share of each of four coarse bins and seven finer ones. Two readings are taken from it: how much of the coming stretch the current speaker stays silent, which is the signal of a turn ending, and whether one speaker is active while the other is not, which is the signal of the floor being held or taken. Two further outputs, a takeover and a handover, are trained from the same activity and read earlier.
The same past, two forecasts
A pause inside A's turn
A is forecast to come back and B to stay quiet: a hold, no event.
The end of A's turn
A is forecast to stay quiet and B to take the floor: an end of turn.
Left of the line is the audio heard so far, the same in both: A spoke until 0.3 s ago and B has been silent. Right of it is the forecast, one bar per bin of the next two seconds (0 to 0.2 s, 0.2 to 0.6, 0.6 to 1.2, 1.2 to 2), its height the predicted share of that bin in which the speaker is active. Yield reads A's bars from 0.2 s on: high on the right, low on the left. Floor claim reads from 0.6 s on whether one speaker is active and the other not: on the right, B. Illustrative values.
The rules wait for those readings before they commit. That wait is the latency in the results below, and it is also what keeps a pause from being called an end.
Causal by construction
A turn-taking model is only useful live, so every event has to be decided from the audio before it. Each emitted time is no earlier than the moment all the audio its decision used has been heard, including the small look-ahead of the resampler and the hangover of the voice-activity detector. The guarantee is tested rather than assumed: a conversation run whole and the same conversation cut off at any point must produce identical events up to the cut, and a sweep of settings is checked that way before anything is reported. Nothing on this page was produced with hindsight.
Results
Measured on the benchmark's development set, 38 conversations, within the benchmark's false-positive limit. The operating point was chosen on the same set, so held-out estimates stand beside the in-sample figures: the choice is repeated on one part of the set and scored on the rest, over 200 splits. Test-set results are on the leaderboard [2], under the entry The Agentic Data Company OpenYap Turn.
| Task | Recall | False-positive rate | Median latency |
|---|---|---|---|
| Interruption, in-sample | |||
| Interruption, in-sample | 0.9971 | 0.071 | 522 ms |
| Interruption, held-out | |||
| Interruption, held-out | 0.991 | 0.069–0.074 | |
| End of turn, in-sample | |||
| End of turn, in-sample | 0.9853 | 0.077 | 309 ms |
| End of turn, held-out | |||
| End of turn, held-out | 0.982–0.985 | 0.085–0.089 | |
The two tasks come out differently. Interruptions are the easier call for this system: a listener who starts and keeps going is visible on their own channel within a second. Ends of turn are harder, because the evidence is an absence, and the forecast has to say that the silence will last.
More data, bigger network
The same 18.4 M-parameter network was trained on five slices of the training set, from 89 hours to the full 1,984. Its hold-out loss fell at every step, from 0.339 to 0.321 nats, with most of the fall before 1,000 hours.
Hold-out loss
natsNetwork size then did what more hours alone did not. At the same 6,000 audio hours seen, a 72.4 M-parameter network improved both the hold-out loss and the floor-claim reading's agreement with hand-marked turn onsets, measured on a small set of windows that were never trained on. A 178.9 M-parameter network did no better on either, so both submitted detectors read the 72.4 M network, with the takeover and handover outputs added.
| Network | Hold-out loss | Agreement with hand-marked onsets |
|---|---|---|
| 18.4 M parameters | 0.3152 | 0.854 |
| 72.4 M parameters | 0.3131 | 0.875 |
| 178.9 M parameters | 0.3145 | 0.875 |
All three at 6,000 audio hours seen. Loss in nats; agreement as AUC on turn onsets that start in overlap with the other speaker.
Limits
Two channels, one voice each
The model reads two tracks, one speaker on each. A mixed recording of a room, a meeting or a phone line with both voices on one track is outside what it was built or measured for.
Sound only
It does not transcribe, does not know what was said, and does not identify speakers. A turn that ends on a word it would need to understand is called from timing and voice alone.
Latency is a chosen trade
The median wait is 522 ms for an interruption and 309 ms for an end of turn. The operating points were chosen under a latency cap; uncapped, the same model waited 871 and 475 ms for 0.26 points more end-of-turn recall. Calling earlier means more false calls inside the same budget.
More than one call per event
The benchmark counts a burst of calls inside an event's window as one detection, and the submitted setting commits more than one call per true event. A product that needs exactly one decision per event needs a stricter commit rule; that is the next step.
Dev-derived figures
The operating point was chosen on the benchmark's development set, and the figures here are measured on that same set, with held-out estimates beside them. The test figures are the leaderboard's.
One kind of recording
Training audio is calls between two people in separate locations on their own microphones. Meetings, in-person rooms, noisy environments and single-channel telephone audio were not measured.
Our view
We do not think a turn detector is the destination. Justin Uberti, who leads realtime AI at OpenAI, called turn detectors and any other low-parameter endpointing mechanism a dead end, and when asked what should do the job instead wrote: “It needs to be done by a large model that has significant conversational context and strong priors about human interactions.” [3] We share that view. A detector beside the pipeline hears only sound and timing. It does not know that a question was asked, that a story is mid-arc, that this listener cuts in and that one waits. The two confusions at the top of this page, where a pause and an ending sound the same, are settled by meaning and by habit, and only a model that holds the whole conversation can settle them well.
Why train a detector, then? Because those priors have to be learned from somewhere, and a detector is a clean test of whether recordings like these hold them. OpenYap Turn was given no turn labels, no transcripts and no benchmark audio, only two-channel recordings of people talking, and from those alone it learned to tell a pause from an ending and a listener sound from a takeover well enough for the results above. It reached that from sound and timing only, the part of the problem a detector can see. The result is not that a detector is enough; it is that the recordings carry the behaviour, and that a model which also understands what is being said has the rest of it there to learn.
Most speech data is a single channel, so the overlaps, the listener sounds and the half-second of handover that carry turn-taking are mixed into one waveform. With each speaker on their own channel they stay separable. Those recordings, with both channels, are what we think belongs inside the large models: not as a detector on the side, but as training data, so that a model's sense of when to speak is learned from how people do it.
The data
Everything above was learned from the corpus alone. 1,000 hours of it are public as Open Yap 1K: the same two-channel recordings, one speaker per track, with transcripts, free for commercial and research use.
References
- [1]
Freeman Jiang, Ramon Sanabria, Soham Deshmukh, et al. TurnBench: A Multi-Domain Benchmark for Turn-Taking Dynamics in Spoken Dialogue. arXiv:2608.25218, 2026. https://arxiv.org/abs/2608.25218
- [2]
TurnBench leaderboard. https://turnbench.sesame.com
- [3]
Justin Uberti. Post and reply on X, 13 October 2026. https://x.com/juberti/status/2110074827763757188