USM-Lite: Quantization and Sparsity Aware Fine-tuning for Speech Recognition with Universal Speech Models
CoRR(2023)
摘要
End-to-end automatic speech recognition (ASR) models have seen revolutionary
quality gains with the recent development of large-scale universal speech
models (USM). However, deploying these massive USMs is extremely expensive due
to the enormous memory usage and computational cost. Therefore, model
compression is an important research topic to fit USM-based ASR under budget in
real-world scenarios. In this study, we propose a USM fine-tuning approach for
ASR, with a low-bit quantization and N:M structured sparsity aware paradigm on
the model weights, reducing the model complexity from parameter precision and
matrix topology perspectives. We conducted extensive experiments with a
2-billion parameter USM on a large-scale voice search dataset to evaluate our
proposed method. A series of ablation studies validate the effectiveness of up
to int4 quantization and 2:4 sparsity. However, a single compression technique
fails to recover the performance well under extreme setups including int2
quantization and 1:4 sparsity. By contrast, our proposed method can compress
the model to have 9.4
error rate (WER) regressions. We also provided in-depth analyses on the results
and discussions on the limitations and potential solutions, which would be
valuable for future studies.
更多查看译文
关键词
speech recognition,model quantization,model pruning,sparsity,universal speech model
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要