babylonpolice.com

We introduce a multilingual speaker change detection model (USM-SCD) that can
simultaneously detect speaker turns and perform ASR for 96 languages. This
model is adapted from a speech foundation model trained on a large quantity of
supervised and unsupervised data, demonstrating the utility of fine-tuning from
a large generic foundation model for a downstream task. We analyze the
performance of this multilingual speaker change detection model through a
series of ablation studies. We show that the USM-SCD model can achieve more
than 75% average speaker change detection F1 score across a test set that
consists of data from 96 languages. On American English, the USM-SCD model can
achieve an 85.8% speaker change detection F1 score across various public and
internal test sets, beating the previous monolingual baseline model by 21%
relative. We also show that we only need to fine-tune one-quarter of the
trainable model parameters to achieve the best model performance. The USM-SCD
model exhibits state-of-the-art ASR quality compared with a strong public ASR
baseline, making it suitable to handle both tasks with negligible additional
computational cost.