Park, Hansol, Song, Minkyung, Kim, Zinu, Lee, Yujun, Ahn, Hoseong, Choi, Hyunjin, Shim, Kyuhong · Oct 27, 2026
Tap2Listen targets instant speaker switching in streaming audio
Markdown · JSON · Original source
The Tap2Listen paper authors propose a streaming audio-visual target speaker extraction (AV-TSE) model that switches the selected speaker using that person’s face. They argue this addresses a limitation of existing systems, which keep one target speaker fixed for an entire clip, even though conversations call for frequent switches. Tap2Listen separates audio and visual processing: an audio encoder processes the mixture, while visual cues for the selected face condition the decoder. The authors say a switch takes effect immediately after a 0.12ms re-projection. On the LRS3-2mix , they report SI-SNRi of 9.68 dB, compared with 8.85 dB for a baseline at a comparable . After switching to a known face, they report full-quality output from the first frame; the baseline takes about 1 second to recover. They also report that the model runs in real on a single core. The reported results frame switching delay as a practical issue: a system that stays with the previous speaker after a listener selects someone else may miss the beginning of that person’s speech. The evidence here is an abstract’s account of test-set and runtime results; it does not establish how the model performs in other settings. How does Tap2Listen perform when the selected face is not known to the model?
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Oct 27, 2026 (2026-10-27).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-14T00:16:50.218Z. First observed: 2026-10-28T08:12:45.088Z. Retrieved: 2026-10-30T20:03:03.569Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
Evidence is limited to the abstract; the paper’s methods and evaluation details are not included here. The first-frame result is reported for a switch to a known face; performance when the selected face is unknown is not specified. The reported LRS3-2mix results and single--core runtime do not establish performance in other settings.
Open question
How does Tap2Listen perform when the selected face is not known to the model?
Source-stated
The authors propose Tap2Listen, a streaming AV-TSE model intended to switch target speakers immediately, addressing systems that keep one target fixed for a clip.
existing systems fix the target speaker for an entire clip, while conversation demands frequent and immediate target switches.
Source-stated
Tap2Listen assigns mixture processing to an audio encoder and conditions its decoder on visual cues for the selected face.
the target-agnostic audio encoder processes only the mixture, while the target-specific visual cues of the selected face condition only the decoder.
Source-stated
The authors report a 0.12ms re-projection for a switch and full-quality output from the first frame after switching to a known face; the baseline takes about 1 second to recover.
A switch requires only a 0.12ms re-projection and takes effect immediately. We also design component-wise losses
Source-stated
On the LRS3-2mix test set, the authors report SI-SNRi of 9.68 dB versus 8.85 dB for the baseline at a comparable parameter count, and real-time operation on one CPU core.
On the standard LRS3-2mix test set, Tap2Listen improves SI-SNRi from 8.85 to 9.68 dB at a comparable parameter count.
Escape Meridian interpretation
The reported recovery difference suggests that switching delay can affect whether the beginning of the newly selected speaker’s speech is available; the abstract does not establish performance outside its reported evaluation.
After a switch to a known face, the model produces full-quality output from the first frame, whereas the baseline needs about 1 second to recover.
Source and contributor context
Park, Hansol, Song, Minkyung, Kim, Zinu, Lee, Yujun, Ahn, Hoseong, Choi, Hyunjin, Shim, Kyuhong · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Held-out set
Data kept aside during training and used only to test how well a model works on examples it has not seen.
Model parameters
The internal numbers a model adjusts during training; their count is a rough measure of a model's size.
CPU
The central processing unit: a general-purpose processor that carries out a computer's instructions.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Editorial provenance
Version 1. Publication ID: 6a91e711-b221-4321-81e5-88f699ce3d3e.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-14T00:16:50.218Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 49045a1106e76e6e7d36c2bb6b5ba70aa57b231710b0d5c80a774ce439532aa7. Source text hash: bf92ca05f97e379d2a67a09bf71d74739e790ca9a5b5a024166cc6c6cb95420a.
Revision and source-check history
Version 1 · 2026-11-14T00:16:50.218Z
Tap2Listen targets instant speaker switching in streaming audio
The Tap2Listen paper authors propose a streaming audio-visual target speaker extraction (AV-TSE) model that switches the selected speaker using that person’s face. They argue this addresses a limitation of existing systems, which keep one target speaker fixed for an entire clip, even though conversations call for frequent switches. Tap2Listen separates audio and visual processing: an audio encoder processes the mixture, while visual cues for the selected face condition the decoder. The authors say a switch takes effect immediately after a 0.12ms re-projection. On the LRS3-2mix , they report SI-SNRi of 9.68 dB, compared with 8.85 dB for a baseline at a comparable . After switching to a known face, they report full-quality output from the first frame; the baseline takes about 1 second to recover. They also report that the model runs in real on a single core. The reported results frame switching delay as a practical issue: a system that stays with the previous speaker after a listener selects someone else may miss the beginning of that person’s speech. The evidence here is an abstract’s account of test-set and runtime results; it does not establish how the model performs in other settings. How does Tap2Listen perform when the selected face is not known to the model?
Original source
cs.AI · Article
Source details
Original source date: Oct 27, 2026 (2026-10-27).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-14T00:16:50.218Z. First observed: 2026-10-28T08:12:45.088Z. Retrieved: 2026-10-30T20:03:03.569Z.
Limits
Evidence is limited to the abstract; the paper’s methods and evaluation details are not included here. The first-frame result is reported for a switch to a known face; performance when the selected face is unknown is not specified. The reported LRS3-2mix results and single--core runtime do not establish performance in other settings.
Open question
How does Tap2Listen perform when the selected face is not known to the model?
Source-stated
The authors propose Tap2Listen, a streaming AV-TSE model intended to switch target speakers immediately, addressing systems that keep one target fixed for a clip.
existing systems fix the target speaker for an entire clip, while conversation demands frequent and immediate target switches.
Source-stated
Tap2Listen assigns mixture processing to an audio encoder and conditions its decoder on visual cues for the selected face.
the target-agnostic audio encoder processes only the mixture, while the target-specific visual cues of the selected face condition only the decoder.
Source-stated
The authors report a 0.12ms re-projection for a switch and full-quality output from the first frame after switching to a known face; the baseline takes about 1 second to recover.
A switch requires only a 0.12ms re-projection and takes effect immediately. We also design component-wise losses
Source-stated
On the LRS3-2mix test set, the authors report SI-SNRi of 9.68 dB versus 8.85 dB for the baseline at a comparable parameter count, and real-time operation on one CPU core.
On the standard LRS3-2mix test set, Tap2Listen improves SI-SNRi from 8.85 to 9.68 dB at a comparable parameter count.
Escape Meridian interpretation
The reported recovery difference suggests that switching delay can affect whether the beginning of the newly selected speaker’s speech is available; the abstract does not establish performance outside its reported evaluation.
After a switch to a known face, the model produces full-quality output from the first frame, whereas the baseline needs about 1 second to recover.
Source and contributor context
Park, Hansol, Song, Minkyung, Kim, Zinu, Lee, Yujun, Ahn, Hoseong, Choi, Hyunjin, Shim, Kyuhong · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Held-out set
Data kept aside during training and used only to test how well a model works on examples it has not seen.
Model parameters
The internal numbers a model adjusts during training; their count is a rough measure of a model's size.
CPU
The central processing unit: a general-purpose processor that carries out a computer's instructions.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Editorial provenance
Version 1. Publication ID: 6a91e711-b221-4321-81e5-88f699ce3d3e.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-14T00:16:50.218Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 49045a1106e76e6e7d36c2bb6b5ba70aa57b231710b0d5c80a774ce439532aa7. Source text hash: bf92ca05f97e379d2a67a09bf71d74739e790ca9a5b5a024166cc6c6cb95420a.
uncertain · 2026-11-14T00:16:50.218Z · anthropic/claude-opus-5.5
Publication: 6a91e711-b221-4321-81e5-88f699ce3d3e
attribution — pass: The reading attributes the proposal, design and results to the Tap2Listen paper's authors. It consistently frames numbers as author reports and does not present them as independently established facts.
we propose Tap2Listen, a streaming AV-TSE model designed for instant response to target switches
summary — uncertain: The reading never plainly says what AV-TSE does: isolate one person's speech from audio where several people talk at once. It mentions a 'mixture' without saying it is mixed voices, and gives no sense of whether higher SI-SNRi is better. A non-specialist may not grasp the core development. Smallest repair: in the first paragraph, add a phrase such as 'pulling one chosen person's speech out of overlapping audio in real time.' Keep the exact terms, add tooltips for SI-SNRi and re-projection, and note that the authors present the higher score as an improvement. The title and question are otherwise faithful.
uses a target speaker's face to extract the speaker's speech from an input audio mixture in real time
date — pass: The originalPublished date of 2026-10-27 matches the v1 arXiv submission date. The feed date of Oct 29 was correctly not used.
[Submitted on 27 Oct 2026]
support — uncertain: The factual claims match the abstract. However, the third paragraph says 'The reported results frame switching delay as a practical issue,' which attributes this framing to the source. The point that a system 'may miss the beginning of that person's speech' is an inference and is not labeled as Escape Meridian interpretation. The source says only that the baseline needs about 1 second to recover output quality, which is degraded quality rather than necessarily missed speech. Smallest repair: label this as Escape Meridian interpretation, and say 'may deliver degraded output for roughly the first second.' Also, the excerpt for claim 3 omits the known-face and first-frame result it cites.
the baseline needs about 1 second to recover
media — pass: The selected media is the verified direct arXiv abstract page for this paper. It is the most direct access to the source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
Data kept aside during training and used only to test how well a model works on examples it has not seen.
Definition referenceThe internal numbers a model adjusts during training; their count is a rough measure of a model's size.
Definition referenceTIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Explore TIMEThe central processing unit: a general-purpose processor that carries out a computer's instructions.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivData kept aside during training and used only to test how well a model works on examples it has not seen.
Definition referenceThe central processing unit: a general-purpose processor that carries out a computer's instructions.
Definition referenceData kept aside during training and used only to test how well a model works on examples it has not seen.
Definition referenceThe internal numbers a model adjusts during training; their count is a rough measure of a model's size.
Definition referenceTIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Explore TIMEThe central processing unit: a general-purpose processor that carries out a computer's instructions.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivData kept aside during training and used only to test how well a model works on examples it has not seen.
Definition referenceThe central processing unit: a general-purpose processor that carries out a computer's instructions.
Definition reference