p7eess.AS

Category

eess.AS

483 papers

EditSpeech: A Text Based Speech Editing System Using Partial Inference and Bidirectional Fusion

Daxin Tan, Liqun Deng, Yu Ting Yeung, Xin Jiang, Xiao Chen, Tan Lee

2107.01554

It's Raw! Audio Generation with State-Space Models

Karan Goel, Albert Gu, Chris Donahue, Christopher Ré

2202.09729

W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training

Yu-An Chung, Yu Zhang, Wei Han, Chung-Cheng Chiu, James Qin, Ruoming Pang, Yonghui Wu

2108.06209

Context-Aware Prosody Correction for Text-Based Speech Editing

Max Morrison, Lucas Rencker, Zeyu Jin, Nicholas J. Bryan, Juan-Pablo Caceres, Bryan Pardo

2102.08328

Music Transformer

Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer, Ian Simon, Curtis Hawthorne, Andrew M. Dai, Matthew D. Hoffman, Monica Dinculescu, Douglas Eck

1809.04281

WaveGrad: Estimating Gradients for Waveform Generation

Nanxin Chen, Yu Zhang, Heiga Zen, Ron J. Weiss, Mohammad Norouzi, William Chan

2009.00713

DiffWave: A Versatile Diffusion Model for Audio Synthesis

Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, Bryan Catanzaro

2009.09761

HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units

Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, Abdelrahman Mohamed

2106.07447

A Spectral Energy Distance for Parallel Speech Synthesis

Alexey A. Gritsenko, Tim Salimans, Rianne van den Berg, Jasper Snoek, Nal Kalchbrenner

2008.01160

VocGAN: A High-Fidelity Real-time Vocoder with a Hierarchically-nested Adversarial Network

Jinhyeok Yang, Junmo Lee, Youngik Kim, Hoonyoung Cho, Injung Kim

2007.15256

DDSP: Differentiable Digital Signal Processing

Jesse Engel, Lamtharn Hantrakul, Chenjie Gu, Adam Roberts

2001.04643

Jukebox: A Generative Model for Music

Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever

2005.00341

Efficient Neural Audio Synthesis

Nal Kalchbrenner, Erich Elsen, Karen Simonyan, Seb Noury, Norman Casagrande, Edward Lockhart, Florian Stimberg, Aaron van den Oord, Sander Dieleman, Koray Kavukcuoglu

1802.08435

ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech

Wei Ping, Kainan Peng, Jitong Chen

1807.07281

High Fidelity Speech Synthesis with Adversarial Networks

Mikołaj Bińkowski, Jeff Donahue, Sander Dieleman, Aidan Clark, Erich Elsen, Norman Casagrande, Luis C. Cobo, Karen Simonyan

1909.11646

WaveGlow: A Flow-based Generative Network for Speech Synthesis

Ryan Prenger, Rafael Valle, Bryan Catanzaro

1811.00002

Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram

Ryuichi Yamamoto, Eunwoo Song, Jae-Min Kim

1910.11480

MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis

Kundan Kumar, Rithesh Kumar, Thibault de Boissiere, Lucas Gestin, Wei Zhen Teoh, Jose Sotelo, Alexandre de Brebisson, Yoshua Bengio, Aaron Courville

1910.06711

GANSynth: Adversarial Neural Audio Synthesis

Jesse Engel, Kumar Krishna Agrawal, Shuo Chen, Ishaan Gulrajani, Chris Donahue, Adam Roberts

1902.08710

A$^3$T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editing

He Bai, Renjie Zheng, Junkun Chen, Xintong Li, Mingbo Ma, Liang Huang

2203.09690

MAESTRO: Matched Speech Text Representations through Modality Matching

Zhehuai Chen, Yu Zhang, Andrew Rosenberg, Bhuvana Ramabhadran, Pedro Moreno, Ankur Bapna, Heiga Zen

2204.03409

MelNet: A Generative Model for Audio in the Frequency Domain

Sean Vasquez, Mike Lewis

1906.01083

SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing

Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, Furu Wei

2110.07205

Fast Spectrogram Inversion using Multi-head Convolutional Neural Networks

Sercan O. Arik, Heewoo Jun, Gregory Diamos

1808.06719

Neural source-filter waveform models for statistical parametric speech synthesis

Xin Wang, Shinji Takaki, Junichi Yamagishi

1904.12088

Data Augmenting Contrastive Learning of Speech Representations in the Time Domain

Eugene Kharitonov, Morgane Rivière, Gabriel Synnaeve, Lior Wolf, Pierre-Emmanuel Mazaré, Matthijs Douze, Emmanuel Dupoux

2007.00991

Perceiver IO: A General Architecture for Structured Inputs & Outputs

Andrew Jaegle, Sebastian Borgeaud, Jean-Baptiste Alayrac, Carl Doersch, Catalin Ionescu, David Ding, Skanda Koppula, Daniel Zoran, Andrew Brock, Evan Shelhamer, Olivier Hénaff, Matthew M. Botvinick, Andrew Zisserman, Oriol Vinyals, Joāo Carreira

2107.14795

TorchAudio: Building Blocks for Audio and Speech Processing

Yao-Yuan Yang, Moto Hira, Zhaoheng Ni, Anjali Chourdia, Artyom Astafurov, Caroline Chen, Ching-Feng Yeh, Christian Puhrsch, David Pollack, Dmitriy Genzel, Donny Greenberg, Edward Z. Yang, Jason Lian, Jay Mahadeokar, Jeff Hwang, Ji Chen, Peter Goldsborough, Prabhat Roy, Sean Narenthiran, Shinji Watanabe, Soumith Chintala, Vincent Quenneville-Bélair, Yangyang Shi

2110.15018

Almost Unsupervised Text to Speech and Automatic Speech Recognition

Yi Ren, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, Tie-Yan Liu

1905.06791

Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task

Yun Tang, Juan Pino, Xian Li, Changhan Wang, Dmitriy Genzel

2107.05782

Recent Developments on ESPnet Toolkit Boosted by Conformer

Pengcheng Guo, Florian Boyer, Xuankai Chang, Tomoki Hayashi, Yosuke Higuchi, Hirofumi Inaguma, Naoyuki Kamo, Chenda Li, Daniel Garcia-Romero, Jiatong Shi, Jing Shi, Shinji Watanabe, Kun Wei, Wangyou Zhang, Yuekai Zhang

2010.13956

Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers

Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, Furu Wei

2301.02111

Conformer: Convolution-augmented Transformer for Speech Recognition

Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, Ruoming Pang

2005.08100

wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations

Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, Michael Auli

2006.11477

Self-Training for End-to-End Speech Translation

Juan Pino, Qiantong Xu, Xutai Ma, Mohammad Javad Dousti, Yun Tang

2006.02490

Dual-decoder Transformer for Joint Automatic Speech Recognition and Multilingual Speech Translation

Hang Le, Juan Pino, Changhan Wang, Jiatao Gu, Didier Schwab, Laurent Besacier

2011.00747

DeCoAR 2.0: Deep Contextualized Acoustic Representations with Vector Quantization

Shaoshi Ling, Yuzong Liu

2012.06659

FastSpeech 2: Fast and High-Quality End-to-End Text to Speech

Yi Ren, Chenxu Hu, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, Tie-Yan Liu

2006.04558

fairseq S2T: Fast Speech-to-Text Modeling with fairseq

Changhan Wang, Yun Tang, Xutai Ma, Anne Wu, Sravya Popuri, Dmytro Okhonko, Juan Pino

2010.05171

LPCNet: Improving Neural Speech Synthesis Through Linear Prediction

Jean-Marc Valin, Jan Skoglund

1810.11846

BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition

Yu Zhang, Daniel S. Park, Wei Han, James Qin, Anmol Gulati, Joel Shor, Aren Jansen, Yuanzhong Xu, Yanping Huang, Shibo Wang, Zongwei Zhou, Bo Li, Min Ma, William Chan, Jiahui Yu, Yongqiang Wang, Liangliang Cao, Khe Chai Sim, Bhuvana Ramabhadran, Tara N. Sainath, Françoise Beaufays, Zhifeng Chen, Quoc V. Le, Chung-Cheng Chiu, Ruoming Pang, Yonghui Wu

2109.13226

End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures

Gabriel Synnaeve, Qiantong Xu, Jacob Kahn, Tatiana Likhomanenko, Edouard Grave, Vineel Pratap, Anuroop Sriram, Vitaliy Liptchinsky, Ronan Collobert

1911.08460

End-to-End Audio Visual Scene-Aware Dialog using Multimodal Attention-Based Video Features

Chiori Hori, Huda Alamri, Jue Wang, Gordon Wichern, Takaaki Hori, Anoop Cherian, Tim K. Marks, Vincent Cartillier, Raphael Gontijo Lopes, Abhishek Das, Irfan Essa, Dhruv Batra, Devi Parikh

1806.08409

Curriculum Pre-training for End-to-End Speech Translation

Chengyi Wang, Yu Wu, Shujie Liu, Ming Zhou, Zhenglu Yang

2004.10093

CoVoST 2 and Massively Multilingual Speech-to-Text Translation

Changhan Wang, Anne Wu, Juan Pino

2007.10310

Libri-Light: A Benchmark for ASR with Limited or No Supervision

Jacob Kahn, Morgane Rivière, Weiyi Zheng, Evgeny Kharitonov, Qiantong Xu, Pierre-Emmanuel Mazaré, Julien Karadayi, Vitaliy Liptchinsky, Ronan Collobert, Christian Fuegen, Tatiana Likhomanenko, Gabriel Synnaeve, Armand Joulin, Abdelrahman Mohamed, Emmanuel Dupoux

1912.07875

Multilingual End-to-End Speech Translation

Hirofumi Inaguma, Kevin Duh, Tatsuya Kawahara, Shinji Watanabe

1910.00254

Analyzing ASR pretraining for low-resource speech-to-text translation

Mihaela C. Stoian, Sameer Bansal, Sharon Goldwater

1910.10762

Scaling End-to-End Models for Large-Scale Multilingual ASR

Bo Li, Ruoming Pang, Tara N. Sainath, Anmol Gulati, Yu Zhang, James Qin, Parisa Haghani, W. Ronny Huang, Min Ma, Junwen Bai

2104.14830

On the limit of English conversational speech recognition

Zoltán Tüske, George Saon, Brian Kingsbury

2105.00982

Injecting Text in Self-Supervised Speech Pretraining

Zhehuai Chen, Yu Zhang, Andrew Rosenberg, Bhuvana Ramabhadran, Gary Wang, Pedro Moreno

2108.12226

One-To-Many Multilingual End-to-end Speech Translation

Mattia Antonino Di Gangi, Matteo Negri, Marco Turchi

1910.03320

Bridging the Gap between Pre-Training and Fine-Tuning for End-to-End Speech Translation

Chengyi Wang, Yu Wu, Shujie Liu, Zhenglu Yang, Ming Zhou

1909.07575

SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition

Daniel S. Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D. Cubuk, Quoc V. Le

1904.08779

A Comparative Study on Transformer vs RNN in Speech Applications

Shigeki Karita, Nanxin Chen, Tomoki Hayashi, Takaaki Hori, Hirofumi Inaguma, Ziyan Jiang, Masao Someki, Nelson Enrique Yalta Soplin, Ryuichi Yamamoto, Xiaofei Wang, Shinji Watanabe, Takenori Yoshimura, Wangyou Zhang

1909.06317

Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debates

Javier Iranzo-Sánchez, Joan Albert Silvestre-Cerdà, Javier Jorge, Nahuel Roselló, Adrià Giménez, Albert Sanchis, Jorge Civera, Alfons Juan

1911.03167

Improving Transformer-based Speech Recognition Using Unsupervised Pre-training

Dongwei Jiang, Xiaoning Lei, Wubo Li, Ne Luo, Yuxuan Hu, Wei Zou, Xiangang Li

1910.09932

MLS: A Large-Scale Multilingual Dataset for Speech Research

Vineel Pratap, Qiantong Xu, Anuroop Sriram, Gabriel Synnaeve, Ronan Collobert

2012.03411

A Better and Faster End-to-End Model for Streaming ASR

Bo Li, Anmol Gulati, Jiahui Yu, Tara N. Sainath, Chung-Cheng Chiu, Arun Narayanan, Shuo-Yiin Chang, Ruoming Pang, Yanzhang He, James Qin, Wei Han, Qiao Liang, Yu Zhang, Trevor Strohman, Yonghui Wu

2011.10798

Advancing RNN Transducer Technology for Speech Recognition

George Saon, Zoltan Tueske, Daniel Bolanos, Brian Kingsbury

2103.09935

Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameters

Vineel Pratap, Anuroop Sriram, Paden Tomasello, Awni Hannun, Vitaliy Liptchinsky, Gabriel Synnaeve, Ronan Collobert

2007.03001

Syllable-Based Sequence-to-Sequence Speech Recognition with the Transformer in Mandarin Chinese

Shiyu Zhou, Linhao Dong, Shuang Xu, Bo Xu

1804.10752

RWTH ASR Systems for LibriSpeech: Hybrid vs Attention -- w/o Data Augmentation

Christoph Lüscher, Eugen Beck, Kazuki Irie, Markus Kitza, Wilfried Michel, Albert Zeyer, Ralf Schlüter, Hermann Ney

1905.03072

Weakly Supervised Deep Recurrent Neural Networks for Basic Dance Step Generation

Nelson Yalta, Shinji Watanabe, Kazuhiro Nakadai, Tetsuya Ogata

1807.01126

Multilingual Speech Recognition With A Single End-To-End Model

Shubham Toshniwal, Tara N. Sainath, Ron J. Weiss, Bo Li, Pedro Moreno, Eugene Weinstein, Kanishka Rao

1711.01694

A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognition

Shubham Toshniwal, Anjuli Kannan, Chung-Cheng Chiu, Yonghui Wu, Tara N Sainath, Karen Livescu

1807.10857

State-of-the-art Speech Recognition With Sequence-to-Sequence Models

Chung-Cheng Chiu, Tara N. Sainath, Yonghui Wu, Rohit Prabhavalkar, Patrick Nguyen, Zhifeng Chen, Anjuli Kannan, Ron J. Weiss, Kanishka Rao, Ekaterina Gonina, Navdeep Jaitly, Bo Li, Jan Chorowski, Michiel Bacchiani

1712.01769

An analysis of incorporating an external language model into a sequence-to-sequence model

Anjuli Kannan, Yonghui Wu, Patrick Nguyen, Tara N. Sainath, Zhifeng Chen, Rohit Prabhavalkar

1712.01996

Multi-task self-supervised learning for Robust Speech Recognition

Mirco Ravanelli, Jianyuan Zhong, Santiago Pascual, Pawel Swietojanski, Joao Monteiro, Jan Trmal, Yoshua Bengio

2001.09239

Learning Robust and Multilingual Speech Representations

Kazuya Kawakami, Luyu Wang, Chris Dyer, Phil Blunsom, Aaron van den Oord

2001.11128

Neural Voice Cloning with a Few Samples

Sercan O. Arik, Jitong Chen, Kainan Peng, Wei Ping, Yanqi Zhou

1802.06006

Investigation of enhanced Tacotron text-to-speech synthesis systems with self-attention for pitch accent language

Yusuke Yasuda, Xin Wang, Shinji Takaki, Junichi Yamagishi

1810.11960

MusicLM: Generating Music From Text

Andrea Agostinelli, Timo I. Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, Matt Sharifi, Neil Zeghidour, Christian Frank

2301.11325

Very Deep Self-Attention Networks for End-to-End Speech Recognition

Ngoc-Quan Pham, Thai-Son Nguyen, Jan Niehues, Markus Müller, Sebastian Stüker, Alexander Waibel

1904.13377

On the Comparison of Popular End-to-End Models for Large Scale Speech Recognition

Jinyu Li, Yu Wu, Yashesh Gaur, Chengyi Wang, Rui Zhao, Shujie Liu

2005.14327

State-of-the-Art Speech Recognition Using Multi-Stream Self-Attention With Dilated 1D Convolutions

Kyu J. Han, Ramon Prieto, Kaixing Wu, Tao Ma

1910.00716

Direct speech-to-speech translation with a sequence-to-sequence model

Ye Jia, Ron J. Weiss, Fadi Biadsy, Wolfgang Macherey, Melvin Johnson, Zhifeng Chen, Yonghui Wu

1904.06037

BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data

Mateusz Łajszczak, Guillermo Cámbara, Yang Li, Fatih Beyhan, Arent van Korlaar, Fan Yang, Arnaud Joly, Álvaro Martín-Cortinas, Ammar Abbas, Adam Michalski, Alexis Moinet, Sri Karlapati, Ewa Muszyńska, Haohan Guo, Bartosz Putrycz, Soledad López Gambino, Kayeon Yoo, Elena Sokolova, Thomas Drugman

2402.08093

A Comparison of Modeling Units in Sequence-to-Sequence Speech Recognition with the Transformer on Mandarin Chinese

Shiyu Zhou, Linhao Dong, Shuang Xu, Bo Xu

1805.06239

Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducer

Kanishka Rao, Haşim Sak, Rohit Prabhavalkar

1801.00841

End-to-End Speech Recognition From the Raw Waveform

Neil Zeghidour, Nicolas Usunier, Gabriel Synnaeve, Ronan Collobert, Emmanuel Dupoux

1806.07098

Learning Noise-Invariant Representations for Robust Speech Recognition

Davis Liang, Zhiheng Huang, Zachary C. Lipton

1807.06610

Minimum Word Error Rate Training for Attention-based Sequence-to-Sequence Models

Rohit Prabhavalkar, Tara N. Sainath, Yonghui Wu, Patrick Nguyen, Zhifeng Chen, Chung-Cheng Chiu, Anjuli Kannan

1712.01818

Quaternion Convolutional Neural Networks for End-to-End Automatic Speech Recognition

Titouan Parcollet, Ying Zhang, Mohamed Morchid, Chiheb Trabelsi, Georges Linarès, Renato De Mori, Yoshua Bengio

1806.07789

Multi-modality Latent Interaction Network for Visual Question Answering

Peng Gao, Haoxuan You, Zhanpeng Zhang, Xiaogang Wang, Hongsheng Li

1908.04289

Learning to Separate Object Sounds by Watching Unlabeled Video

Ruohan Gao, Rogerio Feris, Kristen Grauman

1804.01665

The Sound of Pixels

Hang Zhao, Chuang Gan, Andrew Rouditchenko, Carl Vondrick, Josh McDermott, Antonio Torralba

1804.03160

VoxCeleb2: Deep Speaker Recognition

Joon Son Chung, Arsha Nagrani, Andrew Zisserman

1806.05622

Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separation

Ariel Ephrat, Inbar Mosseri, Oran Lang, Tali Dekel, Kevin Wilson, Avinatan Hassidim, William T. Freeman, Michael Rubinstein

1804.03619

Objects that Sound

Relja Arandjelović, Andrew Zisserman

1712.06651

Cascaded encoders for unifying streaming and non-streaming ASR

Arun Narayanan, Tara N. Sainath, Ruoming Pang, Jiahui Yu, Chung-Cheng Chiu, Rohit Prabhavalkar, Ehsan Variani, Trevor Strohman

2010.14606

Learning Problem-agnostic Speech Representations from Multiple Self-supervised Tasks

Santiago Pascual, Mirco Ravanelli, Joan Serrà, Antonio Bonafonte, Yoshua Bengio

1904.03416

Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning

Wei Ping, Kainan Peng, Andrew Gibiansky, Sercan O. Arik, Ajay Kannan, Sharan Narang, Jonathan Raiman, John Miller

1710.07654

Interpretable Convolutional Filters with SincNet

Mirco Ravanelli, Yoshua Bengio

1811.09725

The PyTorch-Kaldi Speech Recognition Toolkit

Mirco Ravanelli, Titouan Parcollet, Yoshua Bengio

1811.07453

Neural source-filter-based waveform model for statistical parametric speech synthesis

Xin Wang, Shinji Takaki, Junichi Yamagishi

1810.11946

Self-Training for End-to-End Speech Recognition

Jacob Kahn, Ann Lee, Awni Hannun

1909.09116

Robust Speech Recognition via Large-Scale Weak Supervision

Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, Ilya Sutskever

2212.04356

Self-Attention Networks for Connectionist Temporal Classification in Speech Recognition

Julian Salazar, Katrin Kirchhoff, Zhiheng Huang

1901.10055

ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context

Wei Han, Zhengdong Zhang, Yu Zhang, Jiahui Yu, Chung-Cheng Chiu, James Qin, Anmol Gulati, Ruoming Pang, Yonghui Wu

2005.03191

Multilingual sequence-to-sequence speech recognition: architecture, transfer learning, and language modeling

Jaejin Cho, Murali Karthick Baskar, Ruizhi Li, Matthew Wiesner, Sri Harish Mallidi, Nelson Yalta, Martin Karafiat, Shinji Watanabe, Takaaki Hori

1810.03459

Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis

Ye Jia, Yu Zhang, Ron J. Weiss, Quan Wang, Jonathan Shen, Fei Ren, Zhifeng Chen, Patrick Nguyen, Ruoming Pang, Ignacio Lopez Moreno, Yonghui Wu

1806.04558

The Microsoft 2016 Conversational Speech Recognition System

W. Xiong, J. Droppo, X. Huang, F. Seide, M. Seltzer, A. Stolcke, D. Yu, G. Zweig

1609.03528

Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis

Yuxuan Wang, Daisy Stanton, Yu Zhang, RJ Skerry-Ryan, Eric Battenberg, Joel Shor, Ying Xiao, Fei Ren, Ye Jia, Rif A. Saurous

1803.09017

Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron

RJ Skerry-Ryan, Eric Battenberg, Ying Xiao, Yuxuan Wang, Daisy Stanton, Joel Shor, Ron J. Weiss, Rob Clark, Rif A. Saurous

1803.09047

AudioLM: a Language Modeling Approach to Audio Generation

Zalán Borsos, Raphaël Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, Neil Zeghidour

2209.03143

Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data

Wei-Ning Hsu, Yu Zhang, James Glass

1709.07902

Light Gated Recurrent Units for Speech Recognition

Mirco Ravanelli, Philemon Brakel, Maurizio Omologo, Yoshua Bengio

1803.10225

Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis

Rafael Valle, Kevin Shih, Ryan Prenger, Bryan Catanzaro

2005.05957

AudioPaLM: A Large Language Model That Can Speak and Listen

Paul K. Rubenstein, Chulayuth Asawaroengchai, Duc Dung Nguyen, Ankur Bapna, Zalán Borsos, Félix de Chaumont Quitry, Peter Chen, Dalia El Badawy, Wei Han, Eugene Kharitonov, Hannah Muckenhirn, Dirk Padfield, James Qin, Danny Rozenberg, Tara Sainath, Johan Schalkwyk, Matt Sharifi, Michelle Tadmor Ramanovich, Marco Tagliasacchi, Alexandru Tudor, Mihajlo Velimirović, Damien Vincent, Jiahui Yu, Yongqiang Wang, Vicky Zayats, Neil Zeghidour, Yu Zhang, Zhishuai Zhang, Lukas Zilka, Christian Frank

2306.12925

Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervision

Eugene Kharitonov, Damien Vincent, Zalán Borsos, Raphaël Marinier, Sertan Girgin, Olivier Pietquin, Matt Sharifi, Marco Tagliasacchi, Neil Zeghidour

2302.03540

Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages

Yu Zhang, Wei Han, James Qin, Yongqiang Wang, Ankur Bapna, Zhehuai Chen, Nanxin Chen, Bo Li, Vera Axelrod, Gary Wang, Zhong Meng, Ke Hu, Andrew Rosenberg, Rohit Prabhavalkar, Daniel S. Park, Parisa Haghani, Jason Riesa, Ginger Perng, Hagen Soltau, Trevor Strohman, Bhuvana Ramabhadran, Tara Sainath, Pedro Moreno, Chung-Cheng Chiu, Johan Schalkwyk, Françoise Beaufays, Yonghui Wu

2303.01037

SoundStorm: Efficient Parallel Audio Generation

Zalán Borsos, Matt Sharifi, Damien Vincent, Eugene Kharitonov, Neil Zeghidour, Marco Tagliasacchi

2305.09636

Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling

Ziqiang Zhang, Long Zhou, Chengyi Wang, Sanyuan Chen, Yu Wu, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, Furu Wei

2303.03926

JEIT: Joint End-to-End Model and Internal Language Model Training for Speech Recognition

Zhong Meng, Weiran Wang, Rohit Prabhavalkar, Tara N. Sainath, Tongzhou Chen, Ehsan Variani, Yu Zhang, Bo Li, Andrew Rosenberg, Bhuvana Ramabhadran

2302.08583

JOIST: A Joint Speech and Text Streaming Model For ASR

Tara N. Sainath, Rohit Prabhavalkar, Ankur Bapna, Yu Zhang, Zhouyuan Huo, Zhehuai Chen, Bo Li, Weiran Wang, Trevor Strohman

2210.07353

SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-training

Ziqiang Zhang, Long Zhou, Junyi Ao, Shujie Liu, Lirong Dai, Jinyu Li, Furu Wei

2210.03730

Conditional Generative Adversarial Networks for Speech Enhancement and Noise-Robust Speaker Verification

Daniel Michelsanti, Zheng-Hua Tan

1709.01703

RNN-T Models Fail to Generalize to Out-of-Domain Audio: Causes and Solutions

Chung-Cheng Chiu, Arun Narayanan, Wei Han, Rohit Prabhavalkar, Yu Zhang, Navdeep Jaitly, Ruoming Pang, Tara N. Sainath, Patrick Nguyen, Liangliang Cao, Yonghui Wu

2005.03271

Masked Spectrogram Prediction For Self-Supervised Audio Pre-Training

Dading Chong, Helin Wang, Peilin Zhou, Qingcheng Zeng

2204.12768

Masked Spectrogram Modeling using Masked Autoencoders for Learning General-purpose Audio Representation

Daisuke Niizumi, Daiki Takeuchi, Yasunori Ohishi, Noboru Harada, Kunio Kashino

2204.12260

Cycle-consistency training for end-to-end speech recognition

Takaaki Hori, Ramon Astudillo, Tomoki Hayashi, Yu Zhang, Shinji Watanabe, Jonathan Le Roux

1811.01690

Lessons from Building Acoustic Models with a Million Hours of Speech

Sree Hari Krishnan Parthasarathi, Nikko Strom

1904.01624

AudioGen: Textually Guided Audio Generation

Felix Kreuk, Gabriel Synnaeve, Adam Polyak, Uriel Singer, Alexandre Défossez, Jade Copet, Devi Parikh, Yaniv Taigman, Yossi Adi

2209.15352

Unsupervised Speech Recognition via Segmental Empirical Output Distribution Matching

Chih-Kuan Yeh, Jianshu Chen, Chengzhu Yu, Dong Yu

1812.09323

Direct Simultaneous Speech-to-Text Translation Assisted by Synchronized Streaming ASR

Junkun Chen, Mingbo Ma, Renjie Zheng, Liang Huang

2106.06636

HEAR: Holistic Evaluation of Audio Representations

Joseph Turian, Jordie Shier, Humair Raj Khan, Bhiksha Raj, Björn W. Schuller, Christian J. Steinmetz, Colin Malloy, George Tzanetakis, Gissel Velarde, Kirk McNally, Max Henry, Nicolas Pinto, Camille Noufi, Christian Clough, Dorien Herremans, Eduardo Fonseca, Jesse Engel, Justin Salamon, Philippe Esling, Pranay Manocha, Shinji Watanabe, Zeyu Jin, Yonatan Bisk

2203.03022

SSAST: Self-Supervised Audio Spectrogram Transformer

Yuan Gong, Cheng-I Jeff Lai, Yu-An Chung, James Glass

2110.09784

Single headed attention based sequence-to-sequence model for state-of-the-art results on Switchboard

Zoltán Tüske, George Saon, Kartik Audhkhasi, Brian Kingsbury

2001.07263

A Hybrid DSP/Deep Learning Approach to Real-Time Full-Band Speech Enhancement

Jean-Marc Valin

1709.08243

Unsupervised pretraining transfers well across languages

Morgane Rivière, Armand Joulin, Pierre-Emmanuel Mazaré, Emmanuel Dupoux

2002.02848

A Density Ratio Approach to Language Model Fusion in End-To-End Automatic Speech Recognition

Erik McDermott, Hasim Sak, Ehsan Variani

2002.11268

SpeechBrain: A General-Purpose Speech Toolkit

Mirco Ravanelli, Titouan Parcollet, Peter Plantinga, Aku Rouhe, Samuele Cornell, Loren Lugosch, Cem Subakan, Nauman Dawalatabad, Abdelwahab Heba, Jianyuan Zhong, Ju-Chieh Chou, Sung-Lin Yeh, Szu-Wei Fu, Chien-Feng Liao, Elena Rastorgueva, François Grondin, William Aris, Hwidong Na, Yan Gao, Renato De Mori, Yoshua Bengio

2106.04624

The Zero Resource Speech Challenge 2019: TTS without T

Ewan Dunbar, Robin Algayres, Julien Karadayi, Mathieu Bernard, Juan Benjumea, Xuan-Nga Cao, Lucie Miskic, Charlotte Dugrain, Lucas Ondel, Alan W. Black, Laurent Besacier, Sakriani Sakti, Emmanuel Dupoux

1904.11469

Wavenet based low rate speech coding

W. Bastiaan Kleijn, Felicia S. C. Lim, Alejandro Luebs, Jan Skoglund, Florian Stimberg, Quan Wang, Thomas C. Walters

1712.01120

Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss

Qian Zhang, Han Lu, Hasim Sak, Anshuman Tripathi, Erik McDermott, Stephen Koo, Shankar Kumar

2002.02562

FastPitch: Parallel Text-to-speech with Pitch Prediction

Adrian Łańcucki

2006.06873

Improving sequence-to-sequence speech recognition training with on-the-fly data augmentation

Thai-Son Nguyen, Sebastian Stueker, Jan Niehues, Alex Waibel

1910.13296

Self-training and Pre-training are Complementary for Speech Recognition

Qiantong Xu, Alexei Baevski, Tatiana Likhomanenko, Paden Tomasello, Alexis Conneau, Ronan Collobert, Gabriel Synnaeve, Michael Auli

2010.11430

VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation

Changhan Wang, Morgane Rivière, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, Emmanuel Dupoux

2101.00390

HoME: a Household Multimodal Environment

Simon Brodeur, Ethan Perez, Ankesh Anand, Florian Golemo, Luca Celotti, Florian Strub, Jean Rouat, Hugo Larochelle, Aaron Courville

1711.11017

Transformer-based Acoustic Modeling for Hybrid Speech Recognition

Yongqiang Wang, Abdelrahman Mohamed, Duc Le, Chunxi Liu, Alex Xiao, Jay Mahadeokar, Hongzhao Huang, Andros Tjandra, Xiaohui Zhang, Frank Zhang, Christian Fuegen, Geoffrey Zweig, Michael L. Seltzer

1910.09799

A New Training Pipeline for an Improved Neural Transducer

Albert Zeyer, André Merboldt, Ralf Schlüter, Hermann Ney

2005.09319

Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognition

Felix Wu, Kwangyoun Kim, Jing Pan, Kyu Han, Kilian Q. Weinberger, Yoav Artzi

2109.06870

TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech

Andy T. Liu, Shang-Wen Li, Hung-yi Lee

2007.06028

MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement

Szu-Wei Fu, Cheng Yu, Tsun-An Hsieh, Peter Plantinga, Mirco Ravanelli, Xugang Lu, Yu Tsao

2104.03538

AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselines

Yao Shi, Hui Bu, Xin Xu, Shaoji Zhang, Ming Li

2010.11567

QuartzNet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions

Samuel Kriman, Stanislav Beliaev, Boris Ginsburg, Jocelyn Huang, Oleksii Kuchaiev, Vitaly Lavrukhin, Ryan Leary, Jason Li, Yang Zhang

1910.10261

SUPERB: Speech processing Universal PERformance Benchmark

Shu-wen Yang, Po-Han Chi, Yung-Sung Chuang, Cheng-I Jeff Lai, Kushal Lakhotia, Yist Y. Lin, Andy T. Liu, Jiatong Shi, Xuankai Chang, Guan-Ting Lin, Tzu-Hsien Huang, Wei-Cheng Tseng, Ko-tik Lee, Da-Rong Liu, Zili Huang, Shuyan Dong, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, Hung-yi Lee

2105.01051

Unsupervised acoustic unit discovery for speech synthesis using discrete latent-variable neural networks

Ryan Eloff, André Nortje, Benjamin van Niekerk, Avashna Govender, Leanne Nortje, Arnu Pretorius, Elan van Biljon, Ewald van der Westhuizen, Lisa van Staden, Herman Kamper

1904.07556

Unsupervised Speech Recognition

Alexei Baevski, Wei-Ning Hsu, Alexis Conneau, Michael Auli

2105.11084

HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

Jungil Kong, Jaehyeon Kim, Jaekyoung Bae

2010.05646

HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks

Jiaqi Su, Zeyu Jin, Adam Finkelstein

2006.05694

Unsupervised speech representation learning using WaveNet autoencoders

Jan Chorowski, Ron J. Weiss, Samy Bengio, Aäron van den Oord

1901.08810

ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit

Tomoki Hayashi, Ryuichi Yamamoto, Katsuki Inoue, Takenori Yoshimura, Shinji Watanabe, Tomoki Toda, Kazuya Takeda, Yu Zhang, Xu Tan

1910.10909

Unsupervised End-to-End Learning of Discrete Linguistic Units for Voice Conversion

Andy T. Liu, Po-chun Hsu, Hung-yi Lee

1905.11563

Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Model

Anjuli Kannan, Arindrima Datta, Tara N. Sainath, Eugene Weinstein, Bhuvana Ramabhadran, Yonghui Wu, Ankur Bapna, Zhifeng Chen, Seungji Lee

1909.05330

Zero resource speech synthesis using transcripts derived from perceptual acoustic units

Karthik Pandia D S, Hema A Murthy

2006.04372

FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularization

Jiahui Yu, Chung-Cheng Chiu, Bo Li, Shuo-yiin Chang, Tara N. Sainath, Yanzhang He, Arun Narayanan, Wei Han, Anmol Gulati, Yonghui Wu, Ruoming Pang

2010.11148

Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard tasks

Federico Landini, Ján Profant, Mireia Diez, Lukáš Burget

2012.14952

Towards Fast and Accurate Streaming End-to-End ASR

Bo Li, Shuo-yiin Chang, Tara N. Sainath, Ruoming Pang, Yanzhang He, Trevor Strohman, Yonghui Wu

2004.11544

VQVAE Unsupervised Unit Discovery and Multi-scale Code2Spec Inverter for Zerospeech Challenge 2019

Andros Tjandra, Berrak Sisman, Mingyang Zhang, Sakriani Sakti, Haizhou Li, Satoshi Nakamura

1905.11449

Consecutive Decoding for Speech-to-text Translation

Qianqian Dong, Mingxuan Wang, Hao Zhou, Shuang Xu, Bo Xu, Lei Li

2009.09737

Relative Positional Encoding for Transformers with Linear Complexity

Antoine Liutkus, Ondřej Cífka, Shih-Lun Wu, Umut Şimşekli, Yi-Hsuan Yang, Gaël Richard

2105.08399

Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding

Hang Zhang, Xin Li, Lidong Bing

2306.02858

Generalized End-to-End Loss for Speaker Verification

Li Wan, Quan Wang, Alan Papir, Ignacio Lopez Moreno

1710.10467

Hybrid Autoregressive Transducer (hat)

Ehsan Variani, David Rybach, Cyril Allauzen, Michael Riley

2003.07705

Perceiver: General Perception with Iterative Attention

Andrew Jaegle, Felix Gimeno, Andrew Brock, Andrew Zisserman, Oriol Vinyals, Joao Carreira

2103.03206

TernaryBERT: Distillation-aware Ultra-low Bit BERT

Wei Zhang, Lu Hou, Yichun Yin, Lifeng Shang, Xiao Chen, Xin Jiang, Qun Liu

2009.12812

UniAudio: An Audio Foundation Model Toward Universal Audio Generation

Dongchao Yang, Jinchuan Tian, Xu Tan, Rongjie Huang, Songxiang Liu, Xuankai Chang, Jiatong Shi, Sheng Zhao, Jiang Bian, Zhou Zhao, Xixin Wu, Helen Meng

2310.00704

Efficient Content-Based Sparse Attention with Routing Transformers

Aurko Roy, Mohammad Saffar, Ashish Vaswani, David Grangier

2003.05997

Multi-class Spectral Clustering with Overlaps for Speaker Diarization

Desh Raj, Zili Huang, Sanjeev Khudanpur

2011.02900

This Time with Feeling: Learning Expressive Musical Performance

Sageev Oore, Ian Simon, Sander Dieleman, Douglas Eck, Karen Simonyan

1808.03715

Voice Impersonation using Generative Adversarial Networks

Yang Gao, Rita Singh, Bhiksha Raj

1802.06840

Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphs

Wen-Yi Hsiao, Jen-Yu Liu, Yin-Cheng Yeh, Yi-Hsuan Yang

2101.02402

Discrete Audio Representation as an Alternative to Mel-Spectrograms for Speaker and Speech Recognition

Krishna C. Puvvada, Nithin Rao Koluguri, Kunal Dhawan, Jagadeesh Balam, Boris Ginsburg

2309.10922

WGANSing: A Multi-Voice Singing Voice Synthesizer Based on the Wasserstein-GAN

Pritish Chandna, Merlijn Blaauw, Jordi Bonada, Emilia Gomez

1903.10729

SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models

Xin Zhang, Dong Zhang, Shimin Li, Yaqian Zhou, Xipeng Qiu

2308.16692

Low-Latency Sequence-to-Sequence Speech Recognition and Translation by Partial Hypothesis Selection

Danni Liu, Gerasimos Spanakis, Jan Niehues

2005.11185

Low Latency End-to-End Streaming Speech Recognition with a Scout Network

Chengyi Wang, Yu Wu, Shujie Liu, Jinyu Li, Liang Lu, Guoli Ye, Ming Zhou

2003.10369

XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale

Arun Babu, Changhan Wang, Andros Tjandra, Kushal Lakhotia, Qiantong Xu, Naman Goyal, Kritika Singh, Patrick von Platen, Yatharth Saraf, Juan Pino, Alexei Baevski, Alexis Conneau, Michael Auli

2111.09296

Two-Pass End-to-End Speech Recognition

Tara N. Sainath, Ruoming Pang, David Rybach, Yanzhang He, Rohit Prabhavalkar, Wei Li, Mirkó Visontai, Qiao Liang, Trevor Strohman, Yonghui Wu, Ian McGraw, Chung-Cheng Chiu

1908.10992

Rethinking CNN Models for Audio Classification

Kamalesh Palanisamy, Dipika Singhania, Angela Yao

2007.11154

Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition

Yu Zhang, James Qin, Daniel S. Park, Wei Han, Chung-Cheng Chiu, Ruoming Pang, Quoc V. Le, Yonghui Wu

2010.10504

VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset

Sihan Chen, Handong Li, Qunbo Wang, Zijia Zhao, Mingzhen Sun, Xinxin Zhu, Jing Liu

2305.18500

Efficient Training of Audio Transformers with Patchout

Khaled Koutini, Jan Schlüter, Hamid Eghbal-zadeh, Gerhard Widmer

2110.05069

Large-Scale Pre-Training of End-to-End Multi-Talker ASR for Meeting Transcription with Single Distant Microphone

Naoyuki Kanda, Guoli Ye, Yu Wu, Yashesh Gaur, Xiaofei Wang, Zhong Meng, Zhuo Chen, Takuya Yoshioka

2103.16776

End-to-End Adversarial Text-to-Speech

Jeff Donahue, Sander Dieleman, Mikołaj Bińkowski, Erich Elsen, Karen Simonyan

2006.03575

Recursive Visual Sound Separation Using Minus-Plus Net

Xudong Xu, Bo Dai, Dahua Lin

1908.11602

The challenge of realistic music generation: modelling raw audio at scale

Sander Dieleman, Aäron van den Oord, Karen Simonyan

1806.10474

Improving GANs for Speech Enhancement

Huy Phan, Ian V. McLoughlin, Lam Pham, Oliver Y. Chén, Philipp Koch, Maarten De Vos, Alfred Mertins

2001.05532

Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training

Wei-Ning Hsu, Anuroop Sriram, Alexei Baevski, Tatiana Likhomanenko, Qiantong Xu, Vineel Pratap, Jacob Kahn, Ann Lee, Ronan Collobert, Gabriel Synnaeve, Michael Auli

2104.01027

Unsupervised Pre-training of Bidirectional Speech Encoders via Masked Reconstruction

Weiran Wang, Qingming Tang, Karen Livescu

2001.10603

Sequence-to-Sequence Acoustic Modeling for Voice Conversion

Jing-Xuan Zhang, Zhen-Hua Ling, Li-Juan Liu, Yuan Jiang, Li-Rong Dai

1810.06865

ImageBind-LLM: Multi-modality Instruction Tuning

Jiaming Han, Renrui Zhang, Wenqi Shao, Peng Gao, Peng Xu, Han Xiao, Kaipeng Zhang, Chris Liu, Song Wen, Ziyu Guo, Xudong Lu, Shuai Ren, Yafei Wen, Xiaoxin Chen, Xiangyu Yue, Hongsheng Li, Yu Qiao

2309.03905

Textless Direct Speech-to-Speech Translation with Discrete Speech Representation

Xinjian Li, Ye Jia, Chung-Cheng Chiu

2211.00115

NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models

Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Yanqing Liu, Yichong Leng, Kaitao Song, Siliang Tang, Zhizheng Wu, Tao Qin, Xiang-Yang Li, Wei Ye, Shikun Zhang, Jiang Bian, Lei He, Jinyu Li, Sheng Zhao

2403.03100

MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement

Szu-Wei Fu, Chien-Feng Liao, Yu Tsao, Shou-De Lin

1905.04874

Joint Unsupervised and Supervised Training for Multilingual ASR

Junwen Bai, Bo Li, Yu Zhang, Ankur Bapna, Nikhil Siddhartha, Khe Chai Sim, Tara N. Sainath

2111.08137

Deep Contextualized Acoustic Representations For Semi-Supervised Speech Recognition

Shaoshi Ling, Yuzong Liu, Julian Salazar, Katrin Kirchhoff

1912.01679

Towards Reducing the Need for Speech Training Data To Build Spoken Language Understanding Systems

Samuel Thomas, Hong-Kwang J. Kuo, Brian Kingsbury, George Saon

2203.00006

Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesis

Guangzhi Sun, Yu Zhang, Ron J. Weiss, Yuan Cao, Heiga Zen, Yonghui Wu

2002.03785

Masked Autoencoders that Listen

Po-Yao Huang, Hu Xu, Juncheng Li, Alexei Baevski, Michael Auli, Wojciech Galuba, Florian Metze, Christoph Feichtenhofer

2207.06405

Internal Language Model Adaptation with Text-Only Data for End-to-End Speech Recognition

Zhong Meng, Yashesh Gaur, Naoyuki Kanda, Jinyu Li, Xie Chen, Yu Wu, Yifan Gong

2110.05354

Joint Pre-Training with Speech and Bilingual Text for Direct Speech to Speech Translation

Kun Wei, Long Zhou, Ziqiang Zhang, Liping Chen, Shujie Liu, Lei He, Jinyu Li, Furu Wei

2210.17027

Audiobox: Unified Audio Generation with Natural Language Prompts

Apoorv Vyas, Bowen Shi, Matthew Le, Andros Tjandra, Yi-Chiao Wu, Baishan Guo, Jiemin Zhang, Xinyue Zhang, Robert Adkins, William Ngan, Jeff Wang, Ivan Cruz, Bapi Akula, Akinniyi Akinyemi, Brian Ellis, Rashel Moritz, Yael Yungster, Alice Rakotoarison, Liang Tan, Chris Summers, Carleigh Wood, Joshua Lane, Mary Williamson, Wei-Ning Hsu

2312.15821

textless-lib: a Library for Textless Spoken Language Processing

Eugene Kharitonov, Jade Copet, Kushal Lakhotia, Tu Anh Nguyen, Paden Tomasello, Ann Lee, Ali Elkahky, Wei-Ning Hsu, Abdelrahman Mohamed, Emmanuel Dupoux, Yossi Adi

2202.07359

iMetricGAN: Intelligibility Enhancement for Speech-in-Noise using Generative Adversarial Network-based Metric Learning

Haoyu Li, Szu-Wei Fu, Yu Tsao, Junichi Yamagishi

2004.00932

Semantic Mask for Transformer based End-to-End Speech Recognition

Chengyi Wang, Yu Wu, Yujiao Du, Jinyu Li, Shujie Liu, Liang Lu, Shuo Ren, Guoli Ye, Sheng Zhao, Ming Zhou

1912.03010

SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations

Paul-Ambroise Duquenne, Hongyu Gong, Ning Dong, Jingfei Du, Ann Lee, Vedanuj Goswani, Changhan Wang, Juan Pino, Benoît Sagot, Holger Schwenk

2211.04508

NeMo: a toolkit for building AI applications using Neural Modules

Oleksii Kuchaiev, Jason Li, Huyen Nguyen, Oleksii Hrinchuk, Ryan Leary, Boris Ginsburg, Samuel Kriman, Stanislav Beliaev, Vitaly Lavrukhin, Jack Cook, Patrice Castonguay, Mariya Popova, Jocelyn Huang, Jonathan M. Cohen

1909.09577

A comparison of end-to-end models for long-form speech recognition

Chung-Cheng Chiu, Wei Han, Yu Zhang, Ruoming Pang, Sergey Kishchenko, Patrick Nguyen, Arun Narayanan, Hank Liao, Shuyuan Zhang, Anjuli Kannan, Rohit Prabhavalkar, Zhifeng Chen, Tara Sainath, Yonghui Wu

1911.02242

Towards Learning Universal Audio Representations

Luyu Wang, Pauline Luc, Yan Wu, Adria Recasens, Lucas Smaira, Andrew Brock, Andrew Jaegle, Jean-Baptiste Alayrac, Sander Dieleman, Joao Carreira, Aaron van den Oord

2111.12124

Multilingual End-to-End Speech Recognition with A Single Transformer on Low-Resource Languages

Shiyu Zhou, Shuang Xu, Bo Xu

1806.05059

A comparison of recent waveform generation and acoustic modeling methods for neural-network-based speech synthesis

Xin Wang, Jaime Lorenzo-Trueba, Shinji Takaki, Lauri Juvela, Junichi Yamagishi

1804.02549

Spike-Triggered Non-Autoregressive Transformer for End-to-End Speech Recognition

Zhengkun Tian, Jiangyan Yi, Jianhua Tao, Ye Bai, Shuai Zhang, Zhengqi Wen

2005.07903

Parrotron: An End-to-End Speech-to-Speech Conversion Model and its Applications to Hearing-Impaired Speech and Speech Separation

Fadi Biadsy, Ron J. Weiss, Pedro J. Moreno, Dimitri Kanevsky, Ye Jia

1904.04169

Real Time Speech Enhancement in the Waveform Domain

Alexandre Defossez, Gabriel Synnaeve, Yossi Adi

2006.12847

WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing

Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Jian Wu, Michael Zeng, Xiangzhan Yu, Furu Wei

2110.13900

Symbolic Music Generation with Diffusion Models

Gautam Mittal, Jesse Engel, Curtis Hawthorne, Ian Simon

2103.16091

Vector-Quantized Autoregressive Predictive Coding

Yu-An Chung, Hao Tang, James Glass

2005.08392

General-purpose, long-context autoregressive modeling with Perceiver AR

Curtis Hawthorne, Andrew Jaegle, Cătălina Cangea, Sebastian Borgeaud, Charlie Nash, Mateusz Malinowski, Sander Dieleman, Oriol Vinyals, Matthew Botvinick, Ian Simon, Hannah Sheahan, Neil Zeghidour, Jean-Baptiste Alayrac, João Carreira, Jesse Engel

2202.07765

AdaSpeech: Adaptive Text to Speech for Custom Voice

Mingjian Chen, Xu Tan, Bohan Li, Yanqing Liu, Tao Qin, Sheng Zhao, Tie-Yan Liu

2103.00993

Highway Long Short-Term Memory RNNs for Distant Speech Recognition

Yu Zhang, Guoguo Chen, Dong Yu, Kaisheng Yao, Sanjeev Khudanpur, James Glass

1510.08983

Audio ALBERT: A Lite BERT for Self-supervised Learning of Audio Representation

Po-Han Chi, Pei-Hung Chung, Tsung-Han Wu, Chun-Cheng Hsieh, Yen-Hao Chen, Shang-Wen Li, Hung-yi Lee

2005.08575

Audio-Visual Scene Analysis with Self-Supervised Multisensory Features

Andrew Owens, Alexei A. Efros

1804.03641

Robust and fine-grained prosody control of end-to-end speech synthesis

Younggun Lee, Taesu Kim

1811.02122

Jasper: An End-to-End Convolutional Neural Acoustic Model

Jason Li, Vitaly Lavrukhin, Boris Ginsburg, Ryan Leary, Oleksii Kuchaiev, Jonathan M. Cohen, Huyen Nguyen, Ravi Teja Gadde

1904.03288

DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism

Jinglin Liu, Chengxi Li, Yi Ren, Feiyang Chen, Zhou Zhao

2105.02446

Mellotron: Multispeaker expressive voice synthesis by conditioning on rhythm, pitch and global style tokens

Rafael Valle, Jason Li, Ryan Prenger, Bryan Catanzaro

1910.11997

Diff-TTS: A Denoising Diffusion Model for Text-to-Speech

Myeonghun Jeong, Hyeongju Kim, Sung Jun Cheon, Byoung Jin Choi, Nam Soo Kim

2104.01409

Transformer Transducer: One Model Unifying Streaming and Non-streaming Speech Recognition

Anshuman Tripathi, Jaeyoung Kim, Qian Zhang, Han Lu, Hasim Sak

2010.03192

Wav2CLIP: Learning Robust Audio Representations From CLIP

Ho-Hsiang Wu, Prem Seetharaman, Kundan Kumar, Juan Pablo Bello

2110.11499

Text-Free Prosody-Aware Generative Spoken Language Modeling

Eugene Kharitonov, Ann Lee, Adam Polyak, Yossi Adi, Jade Copet, Kushal Lakhotia, Tu-Anh Nguyen, Morgane Rivière, Abdelrahman Mohamed, Emmanuel Dupoux, Wei-Ning Hsu

2109.03264

Parallel and Flexible Sampling from Autoregressive Models via Langevin Dynamics

Vivek Jayaram, John Thickstun

2105.08164

Improving End-to-End Speech Recognition with Policy Learning

Yingbo Zhou, Caiming Xiong, Richard Socher

1712.07101

DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressors

Chandan K A Reddy, Vishak Gopal, Ross Cutler

2010.15258

From Speaker Verification to Multispeaker Speech Synthesis, Deep Transfer with Feedback Constraint

Zexin Cai, Chuxiong Zhang, Ming Li

2005.04587

LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech

Heiga Zen, Viet Dang, Rob Clark, Yu Zhang, Ron J. Weiss, Ye Jia, Zhifeng Chen, Yonghui Wu

1904.02882

Emotion Recognition from Speech Using Wav2vec 2.0 Embeddings

Leonardo Pepino, Pablo Riera, Luciana Ferrer

2104.03502

SpecAugment on Large Scale Datasets

Daniel S. Park, Yu Zhang, Chung-Cheng Chiu, Youzheng Chen, Bo Li, William Chan, Quoc V. Le, Yonghui Wu

1912.05533

Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages

Felix Wu, Kwangyoun Kim, Shinji Watanabe, Kyu Han, Ryan McDonald, Kilian Q. Weinberger, Yoav Artzi

2205.01086

Adversarially Trained End-to-end Korean Singing Voice Synthesis System

Juheon Lee, Hyeong-Seok Choi, Chang-Bin Jeon, Junghyun Koo, Kyogu Lee

1908.01919

WaveGrad 2: Iterative Refinement for Text-to-Speech Synthesis

Nanxin Chen, Yu Zhang, Heiga Zen, Ron J. Weiss, Mohammad Norouzi, Najim Dehak, William Chan

2106.09660

LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Search

Renqian Luo, Xu Tan, Rui Wang, Tao Qin, Jinzhu Li, Sheng Zhao, Enhong Chen, Tie-Yan Liu

2102.04040

VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers

Sanyuan Chen, Shujie Liu, Long Zhou, Yanqing Liu, Xu Tan, Jinyu Li, Sheng Zhao, Yao Qian, Furu Wei

2406.05370

Generating diverse and natural text-to-speech samples using a quantized fine-grained VAE and auto-regressive prosody prior

Guangzhi Sun, Yu Zhang, Ron J. Weiss, Yuan Cao, Heiga Zen, Andrew Rosenberg, Bhuvana Ramabhadran, Yonghui Wu

2002.03788

Knowledge Transfer from Large-scale Pretrained Language Models to End-to-end Speech Recognizers

Yotaro Kubo, Shigeki Karita, Michiel Bacchiani

2202.07894

Cross-modal Contrastive Learning for Speech Translation

Rong Ye, Mingxuan Wang, Lei Li

2205.02444

Exploring Pre-training with Alignments for RNN Transducer based End-to-End Speech Recognition

Hu Hu, Rui Zhao, Jinyu Li, Liang Lu, Yifan Gong

2005.00572

Rethinking Evaluation in ASR: Are Our Models Robust Enough?

Tatiana Likhomanenko, Qiantong Xu, Vineel Pratap, Paden Tomasello, Jacob Kahn, Gilad Avidov, Ronan Collobert, Gabriel Synnaeve

2010.11745

The fifth 'CHiME' Speech Separation and Recognition Challenge: Dataset, task and baselines

Jon Barker, Shinji Watanabe, Emmanuel Vincent, Jan Trmal

1803.10609

MAE-AST: Masked Autoencoding Audio Spectrogram Transformer

Alan Baade, Puyuan Peng, David Harwath

2203.16691

Naver at ActivityNet Challenge 2019 -- Task B Active Speaker Detection (AVA)

Joon Son Chung

1906.10555

Speaker Recognition from Raw Waveform with SincNet

Mirco Ravanelli, Yoshua Bengio

1808.00158

High-Fidelity Audio Compression with Improved RVQGAN

Rithesh Kumar, Prem Seetharaman, Alejandro Luebs, Ishaan Kumar, Kundan Kumar

2306.06546

Sound-Guided Semantic Image Manipulation

Seung Hyun Lee, Wonseok Roh, Wonmin Byeon, Sang Ho Yoon, Chan Young Kim, Jinkyu Kim, Sangpil Kim

2112.00007

A Review of Speaker Diarization: Recent Advances with Deep Learning

Tae Jin Park, Naoyuki Kanda, Dimitrios Dimitriadis, Kyu J. Han, Shinji Watanabe, Shrikanth Narayanan

2101.09624

TranSpeech: Speech-to-Speech Translation With Bilateral Perturbation

Rongjie Huang, Jinglin Liu, Huadai Liu, Yi Ren, Lichao Zhang, Jinzheng He, Zhou Zhao

2205.12523

Speaker Diarization with Region Proposal Network

Zili Huang, Shinji Watanabe, Yusuke Fujita, Paola Garcia, Yiwen Shao, Daniel Povey, Sanjeev Khudanpur

2002.06220

Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speech

Geng Yang, Shan Yang, Kai Liu, Peng Fang, Wei Chen, Lei Xie

2005.05106

The Zero Resource Speech Benchmark 2021: Metrics and baselines for unsupervised spoken language modeling

Tu Anh Nguyen, Maureen de Seyssel, Patricia Rozé, Morgane Rivière, Evgeny Kharitonov, Alexei Baevski, Ewan Dunbar, Emmanuel Dupoux

2011.11588

BYOL-S: Learning Self-supervised Speech Representations by Bootstrapping

Gasser Elbanna, Neil Scheidwasser-Clow, Mikolaj Kegler, Pierre Beckmann, Karl El Hajal, Milos Cernak

2206.12038

Multi-target DoA Estimation with an Audio-visual Fusion Mechanism

Xinyuan Qian, Maulik Madhavi, Zexu Pan, Jiadong Wang, Haizhou Li

2105.06107

MultiSpeech: Multi-Speaker Text to Speech with Transformer

Mingjian Chen, Xu Tan, Yi Ren, Jin Xu, Hao Sun, Sheng Zhao, Tao Qin, Tie-Yan Liu

2006.04664

A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music

Adam Roberts, Jesse Engel, Colin Raffel, Curtis Hawthorne, Douglas Eck

1803.05428

CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordings

Shinji Watanabe, Michael Mandel, Jon Barker, Emmanuel Vincent, Ashish Arora, Xuankai Chang, Sanjeev Khudanpur, Vimal Manohar, Daniel Povey, Desh Raj, David Snyder, Aswin Shanmugam Subramanian, Jan Trmal, Bar Ben Yair, Christoph Boeddeker, Zhaoheng Ni, Yusuke Fujita, Shota Horiguchi, Naoyuki Kanda, Takuya Yoshioka, Neville Ryant

2004.09249

LM-VC: Zero-shot Voice Conversion via Speech Generation based on Language Models

Zhichao Wang, Yuanzhe Chen, Lei Xie, Qiao Tian, Yuping Wang

2306.10521

AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining

Haohe Liu, Yi Yuan, Xubo Liu, Xinhao Mei, Qiuqiang Kong, Qiao Tian, Yuping Wang, Wenwu Wang, Yuxuan Wang, Mark D. Plumbley

2308.05734

Towards Generalized Speech Enhancement with Generative Adversarial Networks

Santiago Pascual, Joan Serrà, Antonio Bonafonte

1904.03418

Streaming end-to-end multi-talker speech recognition

Liang Lu, Naoyuki Kanda, Jinyu Li, Yifan Gong

2011.13148

Exploring Speech Enhancement with Generative Adversarial Networks for Robust Speech Recognition

Chris Donahue, Bo Li, Rohit Prabhavalkar

1711.05747

The Jazz Transformer on the Front Line: Exploring the Shortcomings of AI-composed Music through Quantitative Measures

Shih-Lun Wu, Yi-Hsuan Yang

2008.01307

Exploring wav2vec 2.0 on speaker verification and language identification

Zhiyun Fan, Meng Li, Shiyu Zhou, Bo Xu

2012.06185

SpeechLM: Enhanced Speech Pre-Training with Unpaired Textual Data

Ziqiang Zhang, Sanyuan Chen, Long Zhou, Yu Wu, Shuo Ren, Shujie Liu, Zhuoyuan Yao, Xun Gong, Lirong Dai, Jinyu Li, Furu Wei

2209.15329

UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data

Chengyi Wang, Yu Wu, Yao Qian, Kenichi Kumatani, Shujie Liu, Furu Wei, Michael Zeng, Xuedong Huang

2101.07597

Large-scale ASR Domain Adaptation using Self- and Semi-supervised Learning

Dongseong Hwang, Ananya Misra, Zhouyuan Huo, Nikhil Siddhartha, Shefali Garg, David Qiu, Khe Chai Sim, Trevor Strohman, Françoise Beaufays, Yanzhang He

2110.00165

Overlap-aware diarization: resegmentation using neural end-to-end overlapped speech detection

Latané Bullock, Hervé Bredin, Leibny Paola Garcia-Perera

1910.11646

MorpheuS: generating structured music with constrained patterns and tension

Dorien Herremans, Elaine Chew

1812.04832

Auto-Tuning Spectral Clustering for Speaker Diarization Using Normalized Maximum Eigengap

Tae Jin Park, Kyu J. Han, Manoj Kumar, Shrikanth Narayanan

2003.02405

ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities

Peng Wang, Shijie Wang, Junyang Lin, Shuai Bai, Xiaohuan Zhou, Jingren Zhou, Xinggang Wang, Chang Zhou

2305.11172

GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audio

Guoguo Chen, Shuzhou Chai, Guanbo Wang, Jiayu Du, Wei-Qiang Zhang, Chao Weng, Dan Su, Daniel Povey, Jan Trmal, Junbo Zhang, Mingjie Jin, Sanjeev Khudanpur, Shinji Watanabe, Shuaijiang Zhao, Wei Zou, Xiangang Li, Xuchen Yao, Yongqing Wang, Yujun Wang, Zhao You, Zhiyong Yan

2106.06909

Continuous speech separation: dataset and analysis

Zhuo Chen, Takuya Yoshioka, Liang Lu, Tianyan Zhou, Zhong Meng, Yi Luo, Jian Wu, Xiong Xiao, Jinyu Li

2001.11482

Integration of speech separation, diarization, and recognition for multi-speaker meetings: System description, comparison, and analysis

Desh Raj, Pavel Denisov, Zhuo Chen, Hakan Erdogan, Zili Huang, Maokui He, Shinji Watanabe, Jun Du, Takuya Yoshioka, Yi Luo, Naoyuki Kanda, Jinyu Li, Scott Wisdom, John R. Hershey

2011.02014

Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation tasks

Soumi Maiti, Yifan Peng, Shukjae Choi, Jee-weon Jung, Xuankai Chang, Shinji Watanabe

2309.07937

Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capability

Jinyu Li, Rui Zhao, Zhong Meng, Yanqing Liu, Wenning Wei, Sarangarajan Parthasarathy, Vadim Mazalov, Zhenghao Wang, Lei He, Sheng Zhao, Yifan Gong

2007.15188

Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation

Yi Luo, Nima Mesgarani

1809.07454

WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research

Xinhao Mei, Chutong Meng, Haohe Liu, Qiuqiang Kong, Tom Ko, Chengqi Zhao, Mark D. Plumbley, Yuexian Zou, Wenwu Wang

2303.17395

Leveraging unsupervised and weakly-supervised data to improve direct speech-to-speech translation

Ye Jia, Yifan Ding, Ankur Bapna, Colin Cherry, Yu Zhang, Alexis Conneau, Nobuyuki Morioka

2203.13339

Streaming automatic speech recognition with the transformer model

Niko Moritz, Takaaki Hori, Jonathan Le Roux

2001.02674

The RWTH ASR System for TED-LIUM Release 2: Improving Hybrid HMM with SpecAugment

Wei Zhou, Wilfried Michel, Kazuki Irie, Markus Kitza, Ralf Schlüter, Hermann Ney

2004.00960

HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesis

Sang-Hoon Lee, Ha-Yeong Choi, Seung-Bin Kim, Seong-Whan Lee

2311.12454

BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation

Daisuke Niizumi, Daiki Takeuchi, Yasunori Ohishi, Noboru Harada, Kunio Kashino

2103.06695

SpeechX: Neural Codec Language Model as a Versatile Speech Transformer

Xiaofei Wang, Manthan Thakker, Zhuo Chen, Naoyuki Kanda, Sefik Emre Eskimez, Sanyuan Chen, Min Tang, Shujie Liu, Jinyu Li, Takuya Yoshioka

2308.06873

Leveraging Weakly Supervised Data to Improve End-to-End Speech-to-Text Translation

Ye Jia, Melvin Johnson, Wolfgang Macherey, Ron J. Weiss, Yuan Cao, Chung-Cheng Chiu, Naveen Ari, Stella Laurenzo, Yonghui Wu

1811.02050

Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks

Takuhiro Kaneko, Hirokazu Kameoka

1711.11293

Perceptual Loss based Speech Denoising with an ensemble of Audio Pattern Recognition and Self-Supervised Models

Saurabh Kataria, Jesús Villalba, Najim Dehak

2010.11860

DiffProsody: Diffusion-based Latent Prosody Generation for Expressive Speech Synthesis with Prosody Conditional Adversarial Training

Hyung-Seok Oh, Sang-Hoon Lee, Seong-Whan Lee

2307.16549

Fitting New Speakers Based on a Short Untranscribed Sample

Eliya Nachmani, Adam Polyak, Yaniv Taigman, Lior Wolf

1802.06984

DNN-based Source Enhancement to Increase Objective Sound Quality Assessment Score

Yuma Koizumi, Kenta Niwa, Yusuke Hioka, Kazunori Kobayashi, Yoichi Haneda

1810.09137

Unsupervised Cross-lingual Representation Learning for Speech Recognition

Alexis Conneau, Alexei Baevski, Ronan Collobert, Abdelrahman Mohamed, Michael Auli

2006.13979

AutoSpeech: Neural Architecture Search for Speaker Recognition

Shaojin Ding, Tianlong Chen, Xinyu Gong, Weiwei Zha, Zhangyang Wang

2005.03215

Transformer-based Online CTC/attention End-to-End Speech Recognition Architecture

Haoran Miao, Gaofeng Cheng, Changfeng Gao, Pengyuan Zhang, Yonghong Yan

2001.08290

Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attention

Hideyuki Tachibana, Katsuya Uenoyama, Shunsuke Aihara

1710.08969

Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale

Matthew Le, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, Wei-Ning Hsu

2306.15687

Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection

Ruijie Tao, Zexu Pan, Rohan Kumar Das, Xinyuan Qian, Mike Zheng Shou, Haizhou Li

2107.06592

HiFi-Codec: Group-residual Vector quantization for High Fidelity Audio Codec

Dongchao Yang, Songxiang Liu, Rongjie Huang, Jinchuan Tian, Chao Weng, Yuexian Zou

2305.02765

On Using SpecAugment for End-to-End Speech Translation

Parnia Bahar, Albert Zeyer, Ralf Schlüter, Hermann Ney

1911.08876

Forward Attention in Sequence-to-sequence Acoustic Modelling for Speech Synthesis

Jing-Xuan Zhang, Zhen-Hua Ling, Li-Rong Dai

1807.06736

Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation

Sravya Popuri, Peng-Jen Chen, Changhan Wang, Juan Pino, Yossi Adi, Jiatao Gu, Wei-Ning Hsu, Ann Lee

2204.02967

SoundStream: An End-to-End Neural Audio Codec

Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, Marco Tagliasacchi

2107.03312

Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoders

Chen Xu, Bojie Hu, Yanyang Li, Yuhao Zhang, shen huang, Qi Ju, Tong Xiao, Jingbo Zhu

2105.05752

Slow-Fast Auditory Streams For Audio Recognition

Evangelos Kazakos, Arsha Nagrani, Andrew Zisserman, Dima Damen

2103.03516

Unsupervised Domain Adaptation for Speech Recognition via Uncertainty Driven Self-Training

Sameer Khurana, Niko Moritz, Takaaki Hori, Jonathan Le Roux

2011.13439

Target-Speaker Voice Activity Detection: a Novel Approach for Multi-Speaker Diarization in a Dinner Party Scenario

Ivan Medennikov, Maxim Korenevsky, Tatiana Prisyach, Yuri Khokhlov, Mariya Korenevskaya, Ivan Sorokin, Tatiana Timofeeva, Anton Mitrofanov, Andrei Andrusenko, Ivan Podluzhny, Aleksandr Laptev, Aleksei Romanenko

2005.07272

ESResNet: Environmental Sound Classification Based on Visual Domain Models

Andrey Guzhov, Federico Raue, Jörn Hees, Andreas Dengel

2004.07301

VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset

Jing Liu, Sihan Chen, Xingjian He, Longteng Guo, Xinxin Zhu, Weining Wang, Jinhui Tang

2304.08345

Sound Localization by Self-Supervised Time Delay Estimation

Ziyang Chen, David F. Fouhey, Andrew Owens

2204.12489

Learning with Learned Loss Function: Speech Enhancement with Quality-Net to Improve Perceptual Evaluation of Speech Quality

Szu-Wei Fu, Chien-Feng Liao, Yu Tsao

1905.01898

Investigation of End-To-End Speaker-Attributed ASR for Continuous Multi-Talker Recordings

Naoyuki Kanda, Xuankai Chang, Yashesh Gaur, Xiaofei Wang, Zhong Meng, Zhuo Chen, Takuya Yoshioka

2008.04546

Synchronous Transformers for End-to-End Speech Recognition

Zhengkun Tian, Jiangyan Yi, Ye Bai, Jianhua Tao, Shuai Zhang, Zhengqi Wen

1912.02958

SALMONN: Towards Generic Hearing Abilities for Large Language Models

Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, Chao Zhang

2310.13289

Deep context: end-to-end contextual speech recognition

Golan Pundak, Tara N. Sainath, Rohit Prabhavalkar, Anjuli Kannan, Ding Zhao

1808.02480

Streaming Multi-speaker ASR with RNN-T

Ilya Sklyar, Anna Piunova, Yulan Liu

2011.11671

An Unsupervised Autoregressive Model for Speech Representation Learning

Yu-An Chung, Wei-Ning Hsu, Hao Tang, James Glass

1904.03240

Achieving Human Parity in Conversational Speech Recognition

W. Xiong, J. Droppo, X. Huang, F. Seide, M. Seltzer, A. Stolcke, D. Yu, G. Zweig

1610.05256

Recognizing long-form speech using streaming end-to-end models

Arun Narayanan, Rohit Prabhavalkar, Chung-Cheng Chiu, David Rybach, Tara N. Sainath, Trevor Strohman

1910.11455

A Functional Taxonomy of Music Generation Systems

Dorien Herremans, Ching-Hua Chuan, Elaine Chew

1812.04186

A comprehensive study of speech separation: spectrogram vs waveform separation

Fahimeh Bahmaninezhad, Jian Wu, Rongzhi Gu, Shi-Xiong Zhang, Yong Xu, Meng Yu, Dong Yu

1905.07497

Robust Sequence-to-Sequence Acoustic Modeling with Stepwise Monotonic Attention for Neural TTS

Mutian He, Yan Deng, Lei He

1906.00672

Real-time Speech Frequency Bandwidth Extension

Yunpeng Li, Marco Tagliasacchi, Oleg Rybakov, Victor Ungureanu, Dominik Roblek

2010.10677

AudioCLIP: Extending CLIP to Image, Text and Audio

Andrey Guzhov, Federico Raue, Jörn Hees, Andreas Dengel

2106.13043

Adversarial Learning of Raw Speech Features for Domain Invariant Speech Recognition

Aditay Tripathi, Aanchan Mohan, Saket Anand, Maneesh Singh

1805.08615

AdaDurIAN: Few-shot Adaptation for Neural Text-to-Speech with DurIAN

Zewang Zhang, Qiao Tian, Heng Lu, Ling-Hui Chen, Shan Liu

2005.05642

RNN-T For Latency Controlled ASR With Improved Beam Search

Mahaveer Jain, Kjell Schubert, Jay Mahadeokar, Ching-Feng Yeh, Kaustubh Kalgaonkar, Anuroop Sriram, Christian Fuegen, Michael L. Seltzer

1911.01629

Learning to Speak Fluently in a Foreign Language: Multilingual Speech Synthesis and Cross-Language Voice Cloning

Yu Zhang, Ron J. Weiss, Heiga Zen, Yonghui Wu, Zhifeng Chen, RJ Skerry-Ryan, Ye Jia, Andrew Rosenberg, Bhuvana Ramabhadran

1907.04448

An Open source Implementation of ITU-T Recommendation P.808 with Validation

Babak Naderi, Ross Cutler

2005.08138

VioLA: Unified Codec Language Models for Speech Recognition, Synthesis, and Translation

Tianrui Wang, Long Zhou, Ziqiang Zhang, Yu Wu, Shujie Liu, Yashesh Gaur, Zhuo Chen, Jinyu Li, Furu Wei

2305.16107

Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders

Andy T. Liu, Shu-wen Yang, Po-Han Chi, Po-chun Hsu, Hung-yi Lee

1910.12638

Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Dataset

Xie Chen, Yu Wu, Zhenghao Wang, Shujie Liu, Jinyu Li

2010.11395

AudioLDM: Text-to-Audio Generation with Latent Diffusion Models

Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, Mark D. Plumbley

2301.12503

SEANet: A Multi-modal Speech Enhancement Network

Marco Tagliasacchi, Yunpeng Li, Karolis Misiunas, Dominik Roblek

2009.02095

AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head

Rongjie Huang, Mingze Li, Dongchao Yang, Jiatong Shi, Xuankai Chang, Zhenhui Ye, Yuning Wu, Zhiqing Hong, Jiawei Huang, Jinglin Liu, Yi Ren, Zhou Zhao, Shinji Watanabe

2304.12995

DurIAN: Duration Informed Attention Network For Multimodal Synthesis

Chengzhu Yu, Heng Lu, Na Hu, Meng Yu, Chao Weng, Kun Xu, Peng Liu, Deyi Tuo, Shiyin Kang, Guangzhi Lei, Dan Su, Dong Yu

1909.01700

Speaker Diarization with LSTM

Quan Wang, Carlton Downey, Li Wan, Philip Andrew Mansfield, Ignacio Lopez Moreno

1710.10468

MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment

Hao-Wen Dong, Wen-Yi Hsiao, Li-Chia Yang, Yi-Hsuan Yang

1709.06298

Music Style Transfer: A Position Paper

Shuqi Dai, Zheng Zhang, Gus G. Xia

1803.06841

Can we steal your vocal identity from the Internet?: Initial investigation of cloning Obama's voice using GAN, WaveNet and low-quality found data

Jaime Lorenzo-Trueba, Fuming Fang, Xin Wang, Isao Echizen, Junichi Yamagishi, Tomi Kinnunen

1803.00860

FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech

Alexis Conneau, Min Ma, Simran Khanuja, Yu Zhang, Vera Axelrod, Siddharth Dalmia, Jason Riesa, Clara Rivera, Ankur Bapna

2205.12446

Machine Speech Chain with One-shot Speaker Adaptation

Andros Tjandra, Sakriani Sakti, Satoshi Nakamura

1803.10525

Multi-target Voice Conversion without Parallel Data by Adversarially Learning Disentangled Audio Representations

Ju-chieh Chou, Cheng-chieh Yeh, Hung-yi Lee, Lin-shan Lee

1804.02812

Adaptive Feature Selection for End-to-End Speech Translation

Biao Zhang, Ivan Titov, Barry Haddow, Rico Sennrich

2010.08518

Twin Regularization for online speech recognition

Mirco Ravanelli, Dmitriy Serdyuk, Yoshua Bengio

1804.05374

Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spaces

Yu-An Chung, Wei-Hung Weng, Schrasing Tong, James Glass

1805.07467

Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation

Daniel Stoller, Sebastian Ewert, Simon Dixon

1806.03185

Learning a Latent Space of Multitrack Measures

Ian Simon, Adam Roberts, Colin Raffel, Jesse Engel, Curtis Hawthorne, Douglas Eck

1806.00195

Conditioning Deep Generative Raw Audio Models for Structured Automatic Music

Rachel Manzelli, Vijay Thakkar, Ali Siahkamari, Brian Kulis

1806.09905

Predicting Expressive Speaking Style From Text In End-To-End Speech Synthesis

Daisy Stanton, Yuxuan Wang, RJ Skerry-Ryan

1808.01410

Wasserstein GAN and Waveform Loss-based Acoustic Model Training for Multi-speaker Text-to-Speech Synthesis Systems Using a WaveNet Vocoder

Yi Zhao, Shinji Takaki, Hieu-Thi Luong, Junichi Yamagishi, Daisuke Saito, Nobuaki Minematsu

1807.11679

AdaSpeech 2: Adaptive Text to Speech with Untranscribed Data

Yuzi Yan, Xu Tan, Bohan Li, Tao Qin, Sheng Zhao, Yuan Shen, Tie-Yan Liu

2104.09715

Semi-Supervised Learning with Data Augmentation for End-to-End ASR

Felix Weninger, Franco Mana, Roberto Gemello, Jesús Andrés-Ferrer, Puming Zhan

2007.13876

Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognition

Zhong Meng, Sarangarajan Parthasarathy, Eric Sun, Yashesh Gaur, Naoyuki Kanda, Liang Lu, Xie Chen, Rui Zhao, Jinyu Li, Yifan Gong

2011.01991

Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesis

Yu-An Chung, Yuxuan Wang, Wei-Ning Hsu, Yu Zhang, RJ Skerry-Ryan

1808.10128

Contrastive Learning of Musical Representations

Janne Spijkervet, John Ashley Burgoyne

2103.09410

Noise2Music: Text-conditioned Music Generation with Diffusion Models

Qingqing Huang, Daniel S. Park, Tao Wang, Timo I. Denk, Andy Ly, Nanxin Chen, Zhengdong Zhang, Zhishuai Zhang, Jiahui Yu, Christian Frank, Jesse Engel, Quoc V. Le, William Chan, Zhifeng Chen, Wei Han

2302.03917

A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling

Haihong E, Peiqing Niu, Zhongfu Chen, Meina Song

1907.00390

R-CRNN: Region-based Convolutional Recurrent Neural Network for Audio Event Detection

Chieh-Chi Kao, Weiran Wang, Ming Sun, Chao Wang

1808.06627

Jointly Fine-Tuning "BERT-like" Self Supervised Models to Improve Multimodal Speech Emotion Recognition

Shamane Siriwardhana, Andrew Reis, Rivindu Weerasekera, Suranga Nanayakkara

2008.06682

Neural Dubber: Dubbing for Videos According to Scripts

Chenxu Hu, Qiao Tian, Tingle Li, Yuping Wang, Yuxuan Wang, Hang Zhao

2110.08243

LakhNES: Improving multi-instrumental music generation with cross-domain pre-training

Chris Donahue, Huanru Henry Mao, Yiting Ethan Li, Garrison W. Cottrell, Julian McAuley

1907.04868

Weighted Speech Distortion Losses for Neural-network-based Real-time Speech Enhancement

Yangyang Xia, Sebastian Braun, Chandan K. A. Reddy, Harishchandra Dubey, Ross Cutler, Ivan Tashev

2001.10601

Personal VAD: Speaker-Conditioned Voice Activity Detection

Shaojin Ding, Quan Wang, Shuo-yiin Chang, Li Wan, Ignacio Lopez Moreno

1908.04284

Speech Resynthesis from Discrete Disentangled Self-Supervised Representations

Adam Polyak, Yossi Adi, Jade Copet, Eugene Kharitonov, Kushal Lakhotia, Wei-Ning Hsu, Abdelrahman Mohamed, Emmanuel Dupoux

2104.00355

Advances in Online Audio-Visual Meeting Transcription

Takuya Yoshioka, Igor Abramovski, Cem Aksoylar, Zhuo Chen, Moshe David, Dimitrios Dimitriadis, Yifan Gong, Ilya Gurvich, Xuedong Huang, Yan Huang, Aviv Hurvitz, Li Jiang, Sharon Koubi, Eyal Krupka, Ido Leichter, Changliang Liu, Partha Parthasarathy, Alon Vinnikov, Lingfeng Wu, Xiong Xiao, Wayne Xiong, Huaming Wang, Zhenghao Wang, Jun Zhang, Yong Zhao, Tianyan Zhou

1912.04979

WHAM!: Extending Speech Separation to Noisy Environments

Gordon Wichern, Joe Antognini, Michael Flynn, Licheng Richard Zhu, Emmett McQuinn, Dwight Crow, Ethan Manilow, Jonathan Le Roux

1907.01160

Fast End-to-End Speech Recognition via Non-Autoregressive Models and Cross-Modal Knowledge Transferring from BERT

Ye Bai, Jiangyan Yi, Jianhua Tao, Zhengkun Tian, Zhengqi Wen, Shuai Zhang

2102.07594

MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound

Rowan Zellers, Jiasen Lu, Ximing Lu, Youngjae Yu, Yanpeng Zhao, Mohammadreza Salehi, Aditya Kusupati, Jack Hessel, Ali Farhadi, Yejin Choi

2201.02639

JDI-T: Jointly trained Duration Informed Transformer for Text-To-Speech without Explicit Alignment

Dan Lim, Won Jang, Gyeonghwan O, Heayoung Park, Bongwan Kim, Jaesam Yoon

2005.07799

WavJourney: Compositional Audio Creation with Large Language Models

Xubo Liu, Zhongkai Zhu, Haohe Liu, Yi Yuan, Meng Cui, Qiushi Huang, Jinhua Liang, Yin Cao, Qiuqiang Kong, Mark D. Plumbley, Wenwu Wang

2307.14335

UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation

Huaishao Luo, Lei Ji, Botian Shi, Haoyang Huang, Nan Duan, Tianrui Li, Jason Li, Taroon Bharti, Ming Zhou

2002.06353

End-to-End Neural Speaker Diarization with Permutation-Free Objectives

Yusuke Fujita, Naoyuki Kanda, Shota Horiguchi, Kenji Nagamatsu, Shinji Watanabe

1909.05952

ParaTTS: Learning Linguistic and Prosodic Cross-sentence Information in Paragraph-based TTS

Liumeng Xue, Frank K. Soong, Shaofei Zhang, Lei Xie

2209.06484

Wav2vec-C: A Self-supervised Model for Speech Representation Learning

Samik Sadhu, Di He, Che-Wei Huang, Sri Harish Mallidi, Minhua Wu, Ariya Rastrow, Andreas Stolcke, Jasha Droppo, Roland Maas

2103.08393

VGGSound: A Large-scale Audio-Visual Dataset

Honglie Chen, Weidi Xie, Andrea Vedaldi, Andrew Zisserman

2004.14368

Perfect match: Improved cross-modal embeddings for audio-visual synchronisation

Soo-Whan Chung, Joon Son Chung, Hong-Goo Kang

1809.08001

Crafting Adversarial Examples For Speech Paralinguistics Applications

Yuan Gong, Christian Poellabauer

1711.03280

FunCodec: A Fundamental, Reproducible and Integrable Open-source Toolkit for Neural Speech Codec

Zhihao Du, Shiliang Zhang, Kai Hu, Siqi Zheng

2309.07405

End-to-End Speaker Diarization as Post-Processing

Shota Horiguchi, Paola Garcia, Yusuke Fujita, Shinji Watanabe, Kenji Nagamatsu

2012.10055

Hello Edge: Keyword Spotting on Microcontrollers

Yundong Zhang, Naveen Suda, Liangzhen Lai, Vikas Chandra

1711.07128

Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias

Ziyue Jiang, Yi Ren, Zhenhui Ye, Jinglin Liu, Chen Zhang, Qian Yang, Shengpeng Ji, Rongjie Huang, Chunfeng Wang, Xiang Yin, Zejun Ma, Zhou Zhao

2306.03509

Connecting Speech Encoder and Large Language Model for ASR

Wenyi Yu, Changli Tang, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, Chao Zhang

2309.13963

ICASSP 2021 Deep Noise Suppression Challenge

Chandan K A Reddy, Harishchandra Dubey, Vishak Gopal, Ross Cutler, Sebastian Braun, Hannes Gamper, Robert Aichner, Sriram Srinivasan

2009.06122

VisualEchoes: Spatial Image Representation Learning through Echolocation

Ruohan Gao, Changan Chen, Ziad Al-Halah, Carl Schissler, Kristen Grauman

2005.01616

Serialized Output Training for End-to-End Overlapped Speech Recognition

Naoyuki Kanda, Yashesh Gaur, Xiaofei Wang, Zhong Meng, Takuya Yoshioka

2003.12687

Seed-TTS: A Family of High-Quality Versatile Speech Generation Models

Philip Anastassiou, Jiawei Chen, Jitong Chen, Yuanzhe Chen, Zhuo Chen, Ziyi Chen, Jian Cong, Lelai Deng, Chuang Ding, Lu Gao, Mingqing Gong, Peisong Huang, Qingqing Huang, Zhiying Huang, Yuanyuan Huo, Dongya Jia, Chumin Li, Feiya Li, Hui Li, Jiaxin Li, Xiaoyang Li, Xingxing Li, Lin Liu, Shouda Liu, Sichao Liu, Xudong Liu, Yuchen Liu, Zhengxi Liu, Lu Lu, Junjie Pan, Xin Wang, Yuping Wang, Yuxuan Wang, Zhen Wei, Jian Wu, Chao Yao, Yifeng Yang, Yuanhao Yi, Junteng Zhang, Qidi Zhang, Shuo Zhang, Wenjie Zhang, Yang Zhang, Zilin Zhao, Dejian Zhong, Xiaobin Zhuang

2406.02430

Powerset multi-class cross entropy loss for neural speaker diarization

Alexis Plaquet, Hervé Bredin

2310.13025

Imputer: Sequence Modelling via Imputation and Dynamic Programming

William Chan, Chitwan Saharia, Geoffrey Hinton, Mohammad Norouzi, Navdeep Jaitly

2002.08926

HiddenSinger: High-Quality Singing Voice Synthesis via Neural Audio Codec and Latent Diffusion Models

Ji-Sang Hwang, Sang-Hoon Lee, Seong-Whan Lee

2306.06814

Music Understanding LLaMA: Advancing Text-to-Music Generation with Question Answering and Captioning

Shansong Liu, Atin Sakkeer Hussain, Chenshuo Sun, Ying Shan

2308.11276

End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction

Zhong-Qiu Wang, Jonathan Le Roux, DeLiang Wang, John R. Hershey

1804.10204

The Second DIHARD Diarization Challenge: Dataset, task, and baselines

Neville Ryant, Kenneth Church, Christopher Cieri, Alejandrina Cristia, Jun Du, Sriram Ganapathy, Mark Liberman

1906.07839

Robust Self-Supervised Audio-Visual Speech Recognition

Bowen Shi, Wei-Ning Hsu, Abdelrahman Mohamed

2201.01763

Exploring Tradeoffs in Models for Low-latency Speech Enhancement

Kevin Wilson, Michael Chinen, Jeremy Thorpe, Brian Patton, John Hershey, Rif A. Saurous, Jan Skoglund, Richard F. Lyon

1811.07030

End-to-End Speaker-Attributed ASR with Transformer

Naoyuki Kanda, Guoli Ye, Yashesh Gaur, Xiaofei Wang, Zhong Meng, Zhuo Chen, Takuya Yoshioka

2104.02128

LP-MusicCaps: LLM-Based Pseudo Music Captioning

SeungHeon Doh, Keunwoo Choi, Jongpil Lee, Juhan Nam

2307.16372

Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory

Li Siyao, Weijiang Yu, Tianpei Gu, Chunze Lin, Quan Wang, Chen Qian, Chen Change Loy, Ziwei Liu

2203.13055

Learning from Between-class Examples for Deep Sound Recognition

Yuji Tokozume, Yoshitaka Ushiku, Tatsuya Harada

1711.10282

Audio to Body Dynamics

Eli Shlizerman, Lucio M. Dery, Hayden Schoen, Ira Kemelmacher-Shlizerman

1712.09382

Universal Paralinguistic Speech Representations Using Self-Supervised Conformers

Joel Shor, Aren Jansen, Wei Han, Daniel Park, Yu Zhang

2110.04621

CVSS Corpus and Massively Multilingual Speech-to-Speech Translation

Ye Jia, Michelle Tadmor Ramanovich, Quan Wang, Heiga Zen

2201.03713

Deep Convolutional Neural Network with Mixup for Environmental Sound Classification

Zhichao Zhang, Shugong Xu, Shan Cao, Shunqing Zhang

1808.08405

DenoiSpeech: Denoising Text to Speech with Frame-Level Noise Modeling

Chen Zhang, Yi Ren, Xu Tan, Jinglin Liu, Kejun Zhang, Tao Qin, Sheng Zhao, Tie-Yan Liu

2012.09547

Generative Spoken Dialogue Language Modeling

Tu Anh Nguyen, Eugene Kharitonov, Jade Copet, Yossi Adi, Wei-Ning Hsu, Ali Elkahky, Paden Tomasello, Robin Algayres, Benoit Sagot, Abdelrahman Mohamed, Emmanuel Dupoux

2203.16502

Contrastive Learning of General-Purpose Audio Representations

Aaqib Saeed, David Grangier, Neil Zeghidour

2010.10915

F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching

Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, Jian Zhao, Kai Yu, Xie Chen

2410.06885

Phasebook and Friends: Leveraging Discrete Representations for Source Separation

Jonathan Le Roux, Gordon Wichern, Shinji Watanabe, Andy Sarroff, John R. Hershey

1810.01395

Semi-supervised ASR by End-to-end Self-training

Yang Chen, Weiran Wang, Chao Wang

2001.09128

Variational Autoencoders for Learning Latent Representations of Speech Emotion: A Preliminary Study

Siddique Latif, Rajib Rana, Junaid Qadir, Julien Epps

1712.08708

Listen to Look: Action Recognition by Previewing Audio

Ruohan Gao, Tae-Hyun Oh, Kristen Grauman, Lorenzo Torresani

1912.04487

LMCodec: A Low Bitrate Speech Codec With Causal Transformer Models

Teerapat Jenrungrot, Michael Chinen, W. Bastiaan Kleijn, Jan Skoglund, Zalán Borsos, Neil Zeghidour, Marco Tagliasacchi

2303.12984

Generative Pre-Training for Speech with Autoregressive Predictive Coding

Yu-An Chung, James Glass

1910.12607

Mask CTC: Non-Autoregressive End-to-End ASR with CTC and Mask Predict

Yosuke Higuchi, Shinji Watanabe, Nanxin Chen, Tetsuji Ogawa, Tetsunori Kobayashi

2005.08700

ERANNs: Efficient Residual Audio Neural Networks for Audio Pattern Recognition

Sergey Verbitskiy, Vladimir Berikov, Viacheslav Vyshegorodtsev

2106.01621

Differentiable Consistency Constraints for Improved Deep Speech Enhancement

Scott Wisdom, John R. Hershey, Kevin Wilson, Jeremy Thorpe, Michael Chinen, Brian Patton, Rif A. Saurous

1811.08521

Convolutional-Recurrent Neural Networks for Speech Enhancement

Han Zhao, Shuayb Zarar, Ivan Tashev, Chin-Hui Lee

1805.00579

Attention-Based Models for Text-Dependent Speaker Verification

F A Rezaur Rahman Chowdhury, Quan Wang, Ignacio Lopez Moreno, Li Wan

1710.10470

CAMP: a Two-Stage Approach to Modelling Prosody in Context

Zack Hodari, Alexis Moinet, Sri Karlapati, Jaime Lorenzo-Trueba, Thomas Merritt, Arnaud Joly, Ammar Abbas, Penny Karanasou, Thomas Drugman

2011.01175

Symbolic inductive bias for visually grounded learning of spoken language

Grzegorz Chrupała

1812.09244

GPU-accelerated Guided Source Separation for Meeting Transcription

Desh Raj, Daniel Povey, Sanjeev Khudanpur

2212.05271

A scalable noisy speech dataset and online subjective test framework

Chandan K. A. Reddy, Ebrahim Beyrami, Jamie Pool, Ross Cutler, Sriram Srinivasan, Johannes Gehrke

1909.08050

AVLnet: Learning Audio-Visual Language Representations from Instructional Videos

Andrew Rouditchenko, Angie Boggust, David Harwath, Brian Chen, Dhiraj Joshi, Samuel Thomas, Kartik Audhkhasi, Hilde Kuehne, Rameswar Panda, Rogerio Feris, Brian Kingsbury, Michael Picheny, Antonio Torralba, James Glass

2006.09199

The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Speech Quality and Testing Framework

Chandan K. A. Reddy, Ebrahim Beyrami, Harishchandra Dubey, Vishak Gopal, Roger Cheng, Ross Cutler, Sergiy Matusevych, Robert Aichner, Ashkan Aazami, Sebastian Braun, Puneet Rana, Sriram Srinivasan, Johannes Gehrke

2001.08662

Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generation

Xian Liu, Yinghao Xu, Qianyi Wu, Hang Zhou, Wayne Wu, Bolei Zhou

2201.07786

Time Domain Audio Visual Speech Separation

Jian Wu, Yong Xu, Shi-Xiong Zhang, Lian-Wu Chen, Meng Yu, Lei Xie, Dong Yu

1904.03760

Discriminative Multi-modality Speech Recognition

Bo Xu, Cheng Lu, Yandong Guo, Jacob Wang

2005.05592

Bytes are All You Need: End-to-End Multilingual Speech Recognition and Synthesis with Bytes

Bo Li, Yu Zhang, Tara Sainath, Yonghui Wu, William Chan

1811.09021

A Simple Baseline for Audio-Visual Scene-Aware Dialog

Idan Schwartz, Alexander Schwing, Tamir Hazan

1904.05876

Prompting Large Language Models for Zero-Shot Domain Adaptation in Speech Recognition

Yuang Li, Yu Wu, Jinyu Li, Shujie Liu

2306.16007

Attentive Temporal Pooling for Conformer-based Streaming Language Identification in Long-form Speech

Quan Wang, Yang Yu, Jason Pelecanos, Yiling Huang, Ignacio Lopez Moreno

2202.12163

Diff-HierVC: Diffusion-based Hierarchical Voice Conversion with Robust Pitch Generation and Masked Prior for Zero-shot Speaker Adaptation

Ha-Yeong Choi, Sang-Hoon Lee, Seong-Whan Lee

2311.04693

TasNet: time-domain audio separation network for real-time, single-channel speech separation

Yi Luo, Nima Mesgarani

1711.00541

FurcaNeXt: End-to-end monaural speech separation with dynamic gated dilated temporal convolutional networks

Liwen Zhang, Ziqiang Shi, Jiqing Han, Anyan Shi, Ding Ma

1902.04891

Prosodic Representation Learning and Contextual Sampling for Neural Text-to-Speech

Sri Karlapati, Ammar Abbas, Zack Hodari, Alexis Moinet, Arnaud Joly, Penny Karanasou, Thomas Drugman

2011.02252

Neural Style Transfer for Audio Spectograms

Prateek Verma, Julius O. Smith

1801.01589

PHASEN: A Phase-and-Harmonics-Aware Speech Enhancement Network

Dacheng Yin, Chong Luo, Zhiwei Xiong, Wenjun Zeng

1911.04697

Targeted Adversarial Examples for Black Box Audio Systems

Rohan Taori, Amog Kamsetty, Brenton Chu, Nikita Vemuri

1805.07820

SDR - half-baked or well done?

Jonathan Le Roux, Scott Wisdom, Hakan Erdogan, John R. Hershey

1811.02508

Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentation

Felix Kreuk, Joseph Keshet, Yossi Adi

2007.13465

Listen, Think, and Understand

Yuan Gong, Hongyin Luo, Alexander H. Liu, Leonid Karlinsky, James Glass

2305.10790

Foley Sound Synthesis at the DCASE 2023 Challenge

Keunwoo Choi, Jaekwon Im, Laurie Heller, Brian McFee, Keisuke Imoto, Yuki Okamoto, Mathieu Lagrange, Shinosuke Takamichi

2304.12521

A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild

K R Prajwal, Rudrabha Mukhopadhyay, Vinay Namboodiri, C V Jawahar

2008.10010

VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction

Chaoyou Fu, Haojia Lin, Xiong Wang, Yi-Fan Zhang, Yunhang Shen, Xiaoyu Liu, Haoyu Cao, Zuwei Long, Heting Gao, Ke Li, Long Ma, Xiawu Zheng, Rongrong Ji, Xing Sun, Caifeng Shan, Ran He

2501.01957

The 2018 Signal Separation Evaluation Campaign

Fabian-Robert Stöter, Antoine Liutkus, Nobutaka Ito

1804.06267

DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversion

Ha-Yeong Choi, Sang-Hoon Lee, Seong-Whan Lee

2305.15816

M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge

Fan Yu, Shiliang Zhang, Yihui Fu, Lei Xie, Siqi Zheng, Zhihao Du, Weilong Huang, Pengcheng Guo, Zhijie Yan, Bin Ma, Xin Xu, Hui Bu

2110.07393

Speech Recognition and Multi-Speaker Diarization of Long Conversations

Huanru Henry Mao, Shuyang Li, Julian McAuley, Garrison Cottrell

2005.08072

BatVision: Learning to See 3D Spatial Layout with Two Ears

Jesper Haahr Christensen, Sascha Hornauer, Stella Yu

1912.07011

CLAR: Contrastive Learning of Auditory Representations

Haider Al-Tahan, Yalda Mohsenzadeh

2010.09542

Data augmentation and loss normalization for deep noise suppression

Sebastian Braun, Ivan Tashev

2008.06412

Towards Learning a Universal Non-Semantic Representation of Speech

Joel Shor, Aren Jansen, Ronnie Maor, Oran Lang, Omry Tuval, Felix de Chaumont Quitry, Marco Tagliasacchi, Ira Shavitt, Dotan Emanuel, Yinnon Haviv

2002.12764

Low-Latency Speaker-Independent Continuous Speech Separation

Takuya Yoshioka, Zhuo Chen, Changliang Liu, Xiong Xiao, Hakan Erdogan, Dimitrios Dimitriadis

1904.06478

Unsupervised Cross-Domain Singing Voice Conversion

Adam Polyak, Lior Wolf, Yossi Adi, Yaniv Taigman

2008.02830

Deep Learning Based Phase Reconstruction for Speaker Separation: A Trigonometric Perspective

Zhong-Qiu Wang, Ke Tan, DeLiang Wang

1811.09010

Towards Data Distillation for End-to-end Spoken Conversational Question Answering

Chenyu You, Nuo Chen, Fenglin Liu, Dongchao Yang, Yuexian Zou

2010.08923

SpeechAlign: Aligning Speech Generation to Human Preferences

Dong Zhang, Zhaowei Li, Shimin Li, Xin Zhang, Pengyu Wang, Yaqian Zhou, Xipeng Qiu

2404.05600

Speech Denoising with Deep Feature Losses

Francois G. Germain, Qifeng Chen, Vladlen Koltun

1806.10522

Domain Adaptation via Teacher-Student Learning for End-to-End Speech Recognition

Zhong Meng, Jinyu Li, Yashesh Gaur, Yifan Gong

2001.01798

Enhancing Monotonic Multihead Attention for Streaming ASR

Hirofumi Inaguma, Masato Mimura, Tatsuya Kawahara

2005.09394

PoCoNet: Better Speech Enhancement with Frequency-Positional Embeddings, Semi-Supervised Conversational Data, and Biased Loss

Umut Isik, Ritwik Giri, Neerad Phansalkar, Jean-Marc Valin, Karim Helwani, Arvindh Krishnaswamy

2008.04470

A Study of Non-autoregressive Model for Sequence Generation

Yi Ren, Jinglin Liu, Xu Tan, Zhou Zhao, Sheng Zhao, Tie-Yan Liu

2004.10454

Hierarchical Generative Modeling for Controllable Speech Synthesis

Wei-Ning Hsu, Yu Zhang, Ron J. Weiss, Heiga Zen, Yonghui Wu, Yuxuan Wang, Yuan Cao, Ye Jia, Zhifeng Chen, Jonathan Shen, Patrick Nguyen, Ruoming Pang

1810.07217

SING: Symbol-to-Instrument Neural Generator

Alexandre Défossez, Neil Zeghidour, Nicolas Usunier, Léon Bottou, Francis Bach

1810.09785

Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset

Curtis Hawthorne, Andriy Stasyuk, Adam Roberts, Ian Simon, Cheng-Zhi Anna Huang, Sander Dieleman, Erich Elsen, Jesse Engel, Douglas Eck

1810.12247

ResGrad: Residual Denoising Diffusion Probabilistic Models for Text to Speech

Zehua Chen, Yihan Wu, Yichong Leng, Jiawei Chen, Haohe Liu, Xu Tan, Yang Cui, Ke Wang, Lei He, Sheng Zhao, Jiang Bian, Danilo Mandic

2212.14518

AudioDec: An Open-source Streaming High-fidelity Neural Audio Codec

Yi-Chiao Wu, Israel D. Gebru, Dejan Marković, Alexander Richard

2305.16608

Non-intrusive speech quality assessment using neural networks

Anderson R. Avila, Hannes Gamper, Chandan Reddy, Ross Cutler, Ivan Tashev, Johannes Gehrke

1903.06908

Multimodal Self-Supervised Learning of General Audio Representations

Luyu Wang, Pauline Luc, Adria Recasens, Jean-Baptiste Alayrac, Aaron van den Oord

2104.12807

NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers

Kai Shen, Zeqian Ju, Xu Tan, Yanqing Liu, Yichong Leng, Lei He, Tao Qin, Sheng Zhao, Jiang Bian

2304.09116

FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis

Rongjie Huang, Max W. Y. Lam, Jun Wang, Dan Su, Dong Yu, Yi Ren, Zhou Zhao

2204.09934

Multi-talker Speech Separation with Utterance-level Permutation Invariant Training of Deep Recurrent Neural Networks

Morten Kolbæk, Dong Yu, Zheng-Hua Tan, Jesper Jensen

1703.06284

Monaural Singing Voice Separation with Skip-Filtering Connections and Recurrent Inference of Time-Frequency Mask

Stylianos Ioannis Mimilakis, Konstantinos Drossos, João F. Santos, Gerald Schuller, Tuomas Virtanen, Yoshua Bengio

1711.01437

Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separation

Dong Yu, Morten Kolbæk, Zheng-Hua Tan, Jesper Jensen

1607.00325

ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech

Rongjie Huang, Zhou Zhao, Huadai Liu, Jinglin Liu, Chenye Cui, Yi Ren

2207.06389