Skip to content
InfoResearchPreprint

From Speech Tokens to Stolen Voices: Attacker-Chosen-Text Speaker Impersonation via Representation Inversion

Published
Record updated
View JSON

Summary

Researchers present STIA, a Speech Token Inversion Attack that generates speech with attacker-specified text in a victim's voice from a single short leaked speech representation. The attack needs neither the original audio nor its transcript. A conditional flow-matching model, trained on same-speaker utterance pairs from public speech datasets, combines the leaked speaker characteristics with the target text. With a Moshi representation leak corresponding to 3 seconds of speech, STIA reaches an average CAM++ speaker cosine similarity of 0.78 and a target-text word error rate of 3.80%.