Audio Processing with Large Language Models

Large Language Models (LLMs) have recently introduced a paradigm shift in Machine Learning, including in audio processing tasks. In automatic speech recognition, we have improved understanding quality across a large number of languages, trained within one universal model. In generation, prompt-based capabilities and naturalness have opened up new paradigm shifts. In translation, we can translate numerous language pairs in real time. Finally, in dialogue, we can build single end-to-end systems that understand and reply to user queries. These capabilities are transforming audio products across the industry. In this talk, I will detail the research and product impact of audio LLMs at Google.