Prosody annotations for Switchboard: https://groups.inf.ed.ac.uk/switchboard/index.html

Neural Text to Speech Synthesis Tutorial

https://github.com/tts-tutorial/icassp2022

Survey paper: https://arxiv.org/abs/2106.15561

Convolutional Pitch Tracker (ICASSP 2018)

https://marl.github.io/crepe/

PyTorch port with lots of usage details: https://github.com/maxrmorrison/torchcrepe

Transformer-based sprocket successor, uses TTS pretraining. Available as egs/arctic/vc1 in ESPnet. Sounds much worse than sprocket.


http://www.kecl.ntt.co.jp/people/kameoka.hirokazu/Demos/vtn/index.html

Dealing with residual vocoder noise:

LogMMSE Speech Enhancement and Noise Reduction

https://github.com/rajivpoddar/logmmse

y_enh = logmmse(y, sr, output_file=None, initial_noise=1, window_size=160, noise_threshold=0.15)

StarGANv2-VC authors mentioned this method as one achieving highest MOS on VCC-2020 🤯

https://github.com/yl4579/StarGANv2-VC

I need to take a closer look at VTN

VTN is T23,

T10 is ASR and prosody encoder fed into speaker-dependent TTS fed into WaveNet with single Gaussian outputs. The alternative system of T10 was an autoregressive LSTM that converted PPG into melspc and was used for two male-male parallel speakers.

On AMP and HiFi-GAN: may need to remove the bias from convolution

https://prml-lab-speech-team.github.io/demo/FreGAN2/

A vocoder that uses discrete wavelet transform in the discriminator and has a progressive generator structure similar to StyleGAN2 that produce iDWT arguments

https://github.com/prml-lab-speech-team/demo/tree/master/FreGAN2/code

ICLR 2022

HiFi-GAN + chunked autoregression trains faster and keeps track of pitch better

https://github.com/descriptinc/cargan

https://serrjoa.github.io/projects/universe/

Score-based diffusion for universal speech enhancement (55 distortion types)

Base model: 49M parameters, 5 days, 2xV100, AMP
The paper goes on to describe improvements to the model
Scaled up model: 189M parameters, 14 days 8xV100

Neural Phonetic Alignment with pretrained models for English:
https://github.com/lingjzhu/charsiu/

Lightweight speech encoder

https://github.com/yl4579/AuxiliaryASR