Eric Bezzam retweeted
A small, on-device TTS models is competing with models up to 100× their size, and taking first place on specific benchmarks! @GradiumAI's new Phonon model offers a concrete example. On the Open TTS Leaderboard, Gradium reports: - First place on intelligibility in German and Spanish. - Top-three intelligibility in every language it evaluated. - Leading CPU streaming results on time-to-first-audio. These results also show why it’s useful to compare TTS models on more than a single overall ranking. For the Open TTS Leaderboard, @EricBezzam separates several dimensions: - Intelligibility, measured through word/character error rates. - Generation throughput and time-to-first-audio. - Predicted naturalness scores. - Speaker similarity for voice cloning. Leading on intelligibility in a particular language is a specific result, and so is leading on CPU response latency; but neither establishes that a model is best across every use case. For on-device applications, Phonon’s combination of size, intelligibility, and CPU latency seems really relevant. For other (heavier?) workloads, language coverage, throughput, voice similarity, or delivery may be more important. Leaderboard: huggingface.co/spaces/hf-aud… Gradium’s post: huggingface.co/blog/phonon-t…
1
11
61
4,494
Had the pleasure of hosting @MartinVetterli and @ThomVett at the @huggingface Paris office for a little EPFL reunion 🤖 Fellow alum @cetiennec showed us neat robot demos. Exciting stuff coming on open robotics 🤗
19
1,871
🎸 cue "Good Riddance (Time of Your Life)" by Green Day: "It's something [too predictable] but in the end, it's right. I hope you had the time of your life" We’re phasing out LibriSpeech from the @huggingface Open ASR Leaderboard, meaning it no longer counts toward the default average WER used to rank models. Maybe long overdue, but it’s hard to let go of such an influential resource in the speech community! Kudos to the LibriSpeech team for setting a standard for benchmarking and dataset preparation 👏 leaderboard: huggingface.co/spaces/hf-aud…
4
2
65
4,906
🗣️ Open TTS Leaderboard updates! > 14 new languages via the @MiniMax_AI multilingual eval set. thanks @realmrfakename for helping with the integration 🙏 > NEW models: LFM3-Audio-2B by @liquidai and maya2-native by @mayaresearch_ai > audio.cpp streaming backends for 8 models, thanks @0xShug0 ⚡️
9
6
52
2,880
maya2-native by @mayaresearch_ai is built for speed: lowest TTFA on A100 (19.1 ms p50 / 20.5 ms p95), and it sits on the Pareto frontier of both speed-vs-WER plots. It covers English and 11 Indic languages.
1
130
audio.cpp (C++/ggml) establishes itself on the streaming eval 👏 kudos to @0xShug0! On both H200 and A100, it takes 3 of the top 5 spots.
2
2
103
Eric Bezzam retweeted
Probably the most impactful ASR release of 2026. #1 open-weight model on the Open ASR Leaderboard. Supported day0 in 🤗 Transformers
Meet NVIDIA Nemotron 3.5 Transcribe 🎙️ An open-weight, promptable 30B MoE model with ~4B active parameters that brings transcription in 40 languages, speaker attribution, translation, contextual prompting and audio understanding into one checkpoint. Watch it keep track of who said what when everyone’s talking at once. 1/ 🧵
2
5
39
3,380
BEAST of a model (30B, 4B active) with stellar results on the Open ASR Leaderboard! > #1 among open models on English, #9 overall > #1 among open models on multilingual 🇩🇪🇫🇷🇮🇹🇪🇸🇵🇹🇮🇳🇳🇱, #6 overall Open weights: huggingface.co/nvidia/NVIDIA… Leaderboard: huggingface.co/spaces/hf-aud… Demo: huggingface.co/spaces/smolag…
3
2
30
2,361
Eric Bezzam retweeted
One of the biggest changes in the history of 🤗 Transformers audio is in the works, spearheaded by @eustachelb We’re rebuilding essentially the entire audio preprocessing stack. The goal is to move away from every model carrying its own `FeatureExtractor` implementation, and towards a common `AudioProcessor` architecture where models describe their frontend, while shared backends handle the actual computation. The stack currently covers ~30 audio frontends and 62 model mappings across Whisper, Wav2Vec2, Parakeet, SeamlessM4T, Qwen, Gemma, codecs, TTS models,and many others. A big part of the work is making backends explicit: `backend="torch"` → native PyTorch preprocessing (often super fast!) `backend="numpy"` → portable NumPy preprocessing That means audio preprocessing can increasingly live in the same framework as the model itself, while still keeping compatibility with the numerical behavior existing checkpoints expect. It also gives us a much better foundation for what comes next: new hardware backends, faster kernels, compilation, and optimizations that can benefit many audio models at once instead of being implemented model by model. ~30 frontends migrated. 62 model mappings. One common preprocessing architecture. The entrypoint, common across all architectures: `AutoAudioProcessor.from_pretrained(...)` Audio in Transformers is about to get a much stronger foundation! 🎧🎧🎧
7
12
61
3,938
Open TTS Leaderboard updates 🗣️ > A 31M and a 35B model were added: phonon by @gradium and Drama-3 by @FishAudio. ~1000× apart in size, similar English WER (1.08 vs 1.19) > New metric: UTMOS, for naturalness of English outputs. Thanks @realmrfakename! 🧵
9
14
105
5,697
These recent releases present two interesting "philosophies" for designing TTS systems: > Drama-3: one 35B model, 83 languages, built-in voice cloning, and the 2nd-best UTMOS (4.47) > Phonon: small per-language models, 102 preset voices across 5 languages, built for CPU Both are needed!
1
1
255
Eric Bezzam retweeted
Phonon weights are on the @huggingface Hub, with support in Transformers, and pip install speech-to-speech runs it locally from day 0. Thanks @julien_c @eustachelb @andimarafioti @mervenoyann and team. huggingface.co/gradium/phono…
3
5
94
8,135
Open TTS Leaderboard updates 🗣️ > Paradee-8M by @sahilmdkr, distilled from Kokoro-82M: same WER at 1/10 the params > Now evaluating streaming engines audio.cpp (@0xShug0 ) & photon (@vikhyatk), great for voice agents 🤖 > Eval scripts now open-source ⭐️ 🧵
5
10
61
4,737
audio.cpp and photon bring low-latency streaming to open models: under 35 ms p50 / 35 ms p95 time-to-first-audio on A100. Check them out if you're building voice agents 🤖
2
2
11
2,780