speech-representation

Self-Supervised Speech Pre-training and Representation Learning Toolkit

[ICLR 2025] SOTA discrete acoustic codec models with 40/75 tokens per second for audio language modeling

[ACL 2024] Official PyTorch code for extracting features and training downstream models with emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation

Ultra-low bitrate speech codec (0.27-1 kbps) with cross-modal alignment and real-time capabilities

A Survey of Spoken Dialogue Models (60 pages)

A 1.6B causal Transformer audio tokenizer with streaming, variable bitrates, and semantic alignment across speech, sound, and music

A single-layer, streaming codec model providing SOTA audio quality and discrete tokens designed for superior downstream modelability.

[ACL 2026 Main] Training, inference, and testing of the SAC speech codec model.

This is the code for paper: XY-Tokenizer: Mitigating the Semantic-Acoustic Conflict in Low-Bitrate Speech Codecs

LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERT

Survey of audio language models

Official Implementation of Mockingjay in Pytorch

A mini, simple, and fast end-to-end automatic speech recognition toolkit.

Official code for paper:"Speaking Clearly: A Simplified Whisper-Based Codec for Low-Bitrate Speech Coding"

Causal Speech Enhancement Based on a Two-Branch Nested U-Net Architecture Using Self-Supervised Speech Embeddings

DUSTED: Spoken-Term Discovery using Discrete Speech Units

Semi-supervised spoken language understanding (SLU) via self-supervised speech and language model pretraining

Survey spoken dialogue models with speech input and output, and explore their designs, timelines, and intermediate representations