wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations

Masked latent speech representations across time steps and trained a transformer with contrastive loss over quantized representations, achieving state-of-the-art ASR with 10 minutes of labels.

Serves as qualified reference for implementing task-speech-recognition in production systems.

Assumptions

• Underlying spatio-temporal continuity and domain distributional stability hold

Limitations

• Performance scaling and computational footprint depend on receptive field depth, sequence length, and resolution