Cao, H., Jiang, H., Liu, D., Wang, R., Min, G., Liu, J., Dustdar, S., & Lui, J. C. S. (2023). LiveProbe: Exploring Continuous Voice Liveness Detection via Phonemic Energy Response Patterns. IEEE Internet of Things Journal, 10(8), 7215–7228. https://doi.org/10.1109/JIOT.2022.3228819
continuous liveness detection; energy response pattern; Voice assistant
Voice assistants support contactless smart device control and thus act as a holy grail of human-computer interaction. However, recent studies reveal that an adversary can manipulate devices by vicious voice commands. This security risk is caused by only executing one-time liveness detection and lacking safeguard modules after service activation. Therefore, identifying speaker type (i.e., human articulators or loudspeakers) is critical in protecting voice-driven services during an entire interaction session. In this paper, we propose a continuous voice liveness detection approach LiveProbe, leveraging unique energy response patterns in frequency bands induced by distinct voice generation mechanisms. The rationality behind LiveProbe is presented in two aspects: human articulator reshapes initial voices by exquisitely coordinated movements of vocal organs, which act as band-pass filters generating unique energy responses; nevertheless, the internal modules of loudspeakers are position-fixed and cannot reproduce this response characteristic. To that end, we first work on voice generation mechanisms behind two-type speakers that cause spectrum differences. Then we elaborately construct signal processing and deep-learning modules to extract liveness features. Especially, our approach doesn’t interfere with normal voice interaction and needn’t to carry customized sensors. The experiment presents its effectiveness against potential attacks with a false acceptance rate of 0.51%.
National Natural Science Foundation of China National Natural Science Foundation of China National Natural Science Foundation of China National Natural Science Foundation of China China Scholarship Council Open Research Fund from Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) National Social Science Foundation of China
Grant U20A20181 Grant 61732017 Grant 61902060 Grant 61902122 Grant GML-KF- 22-23 Grant 19ZDA103