Sudarshan Kamath from SmallestAI on how structure beats scale

https://www.youtube.com/watch?v=14Cb7D8p-C4

Odyssey 2026 on speaker verification starts today

https://odyssey2026.inesc-id.pt/the-full-schedule/

ScenA: Reference-Driven Multi-Speaker Audio Scene Generation from In-the-Wild Priors
https://finmickey.github.io/scena/

Abstract. Existing multi-speaker dialogue systems bind speakers to utterances through structured supervision: per-turn tags, multi-stream transcriptions, or learnable speaker embeddings. These systems operate within speech-only pipelines that produce clean vocal sequences without the ambient texture of real conversations. We take a different approach. Our method, ScenA, conditions a text-to-audio flow-matching foundation model, pretrained on large-scale in-the-wild data, directly on multiple reference voices and a free-form natural language prompt that describes an entire multi-speaker audio scene. Leveraging such a foundational model allows us to inherit its capacity for natural, non-studio audio: background noise, room acoustics, overlapping dialogue, and spontaneous paralinguistic events, while adding multi-speaker control without any per-turn structure. Concretely, reference latents are concatenated into the model's token sequence and distinguished by lightweight identity-aware positional encodings. However, we identify a critical obstacle to this approach: the Reference Shortcut. During training under standard noise schedules, the model can identify the matching reference by acoustic similarity to the noisy target, bypassing the text prompt entirely. We address this with a high-noise-biased timestep distribution that forces the model to rely on the text prompt for speaker assignment. We evaluate ScenA on the CoVoMix2-Dialogue benchmark, showing that it outperforms existing multi-speaker systems on speaker-binding metrics while generating rich conversational audio with overlapping speech, emotional vocalizations, and ambient sound. Our results demonstrate the advantage of using a general-purpose audio model conditioned on a free-form scene description, rather than passing structured dialog scripts through a speech-only pipeline.

https://huggingface.co/marcoyang/spear-xlarge-speech-audio-v2 recently published
  
SPEAR XLarge v2 is the flagship open-source SPEAR encoder for unified speech and general-audio representation learning. This is the ICML 2026 accepted version of SPEAR: A Unified SSL Framework for Learning Speech and Audio Representations. This model is the XLarge v2 release, aligned with the model used in the ICML 2026 paper. Compared with the earlier XLarge v1, v2 is enhanced for complex acoustic scenes through token mixing, improving robustness for overlapped speech, noisy audio, and real-world sound mixtures while keeping SPEAR's unified speech-and-audio design.
SPEAR XLarge v2 uses a Zipformer backbone with about 600M parameters, consisting of 13 Zipformer stacks. It produces 1280-dimensional frame-level representations at approximately 50 Hz from 16 kHz waveforms.

Supposed to be good https://arxiv.org/abs/2605.28139

ARK-ASR-3B is a multilingual automatic speech recognition model. It achieves current state-of-the-art results on the Hugging Face Open ASR Leaderboard English short-form benchmark, with an average WER of 5.04%

https://huggingface.co/AutoArk-AI/ARK-ASR-3B

More or less recent tech from Microsoft. Interesting that accuracy is still more or less the same as Whisper Large v3

https://arxiv.org/abs/2604.00610

Speech LLMs are Contextual Reasoning Transcribers
Keqi Deng, Ruchao Fan, Bo Ren, Yiming Wang, Jinyu Li

Despite extensions to speech inputs, effectively leveraging the rich knowledge and contextual understanding of large language models (LLMs) in automatic speech recognition (ASR) remains non-trivial, as the task primarily involves direct speech-to-text mapping. To address this, this paper proposes chain-of-thought ASR (CoT-ASR), which constructs a reasoning chain that enables LLMs to first analyze the input speech and generate contextual analysis, thereby fully exploiting their generative capabilities. With this contextual reasoning, CoT-ASR then performs more informed speech recognition and completes both reasoning and transcription in a single pass. Moreover, CoT-ASR naturally supports user-guided transcription: while designed to self-generate reasoning,

Rare WER is interesting. It flips things around. Whisper V3 and Cohere (also AED) are still best compared to LLM based systems with better overall WER (Qwen). It actually confirms the intuition that Whisper usually gets all special terms right.

Rare WER actually explored before in papers, for example

End-to-End Speech Recognition Contextualization with Large Language Models
https://arxiv.org/abs/2309.10917

Several interesting NAR / diffusion systems released recently

https://huggingface.co/ibm-granite/granite-speech-4.1-2b-nar

Fast and reasonably accurate. Uses interesting CTC output guidance and bidirectional LLM for postcorrection but kind of hard to adapt because LLM is too special

Tedlium WER 4.39

https://github.com/taeyoun811/Whisfusion

Diffusion ASR with Whisper-Small Encoder and SMDM-170M Decoder. Not very accurate since it doesn't use CTC

Tedlium WER 18.03 (bad)

https://github.com/liuzhan22/Diffusion-ASR

From Cambridge. Uses Whisper Large encoder and LLADA 8B Instruct for correction. Can edit existing AR hypothesis for better accuracy

Tedlium WER 7.05 (not very good yet)

There is competition on African voices going on, still plenty of time to join and get the data

https://afrivoice.github.io/afrivoice_eac_hackathon/

https://www.kaggle.com/competitions/afri-voices-east-africa-asr-hackathon

DCASE2026 Challenge results are out! https://dcase.community/challenge2026/

Some new Persian stuff from @Reza2kn

Nemo models

https://huggingface.co/Reza2kn/Shenava-Koochik-v1.0
https://huggingface.co/Reza2kn/Shenava-Rizeh-v1.0
https://huggingface.co/Reza2kn/Shenava-Rizeh-Pizeh-v1.0

Datasets

https://huggingface.co/datasets/Reza2kn/persian-asr-relabeled-gemini
https://huggingface.co/datasets/Reza2kn/persian-asr-text-2.69M-deduped

Leaderboard

https://huggingface.co/spaces/Reza2kn/persian-asr-double-benchmark

Everyone works on translation these days, here is a nice recent release

https://github.com/cmots/STEB

Official code release for STEB: A Speech-to-Speech Translation Expressiveness Benchmark for Evaluating Beyond Translation Fidelity, an automatic evaluation toolkit for speech-to-speech translation systems.

Claude can rewrite Kaldi into a very tiny rust codebase:

https://github.com/Reza2kn/Vosk-Rust

and even add new features like quantization

https://real-tse.github.io/challenge/ challenge results and reports

For example MERL 1st place in offline target speaker extraction

https://real-tse.github.io/assets/pdf/MERL-SA-Track2.pdf

CARTSE first place in online speaker extraction

https://real-tse.github.io/assets/pdf/CARTSE-Track1.pdf

Some recent audio annotation and TTS finetuning projects from LAION, complicated pipelines

https://github.com/LAION-AI/univeral-audio-annotation-pipeline

Produces structured JSON annotations from any audio file, covering speech transcription, speaker diarization, emotions, vocal bursts, sound effects, and music. Best configuration: Gemma-12B + DiCoW — Nemotron 3.5 words + VibeVoice/Sortformer diarization + DiCoW overlap-aware ASR, fused by a text-only Gemma-4-12B LLM (no audio in the final step). It is the highest-Reward pipeline on SoundScape-Bench (0.253) — rank 3 of all systems, nearly matching Gemini 3.5 Flash (0.256) and ahead of every other pipeline. (It trades precision for that recall: see the tradeoff note.)

https://github.com/LAION-AI/laionbox

LaionBox fine-tunes the DramaBox flow-matching transformer using LoRA (rank=128) with 6 differentiable auxiliary losses that push generated audio toward higher naturalness, quality, and voice cloning fidelity:
CLAP Naturalness — Maximizes perceptual naturalness via VoiceCLAP text similarity
Quality MLP — Binary classifier trained to distinguish real from synthetic audio
Centroid Real/Fake — Distribution matching toward real speech embeddings
Speaker Similarity — WavLM-SV voice identity preservation
Comb Filter Detector — Latent-space CNN detecting interference artifacts
Artifact Detector V2 — Residual CNN for general artifact detection

So Huggingface still have trouble to put ensemble model on leaderboard while there is a pull request. At the same time they put Modulate immediately after release

Modulate CTO claims they trained the model on 500M hours of speech

https://www.linkedin.com/feed/update/urn:li:activity:7481395636882444288/

Modulate wins just 0.01 in WER over azure and only place #4 on private leaderboard

Scaling doesn't work it seems

Tanel keeps winning challenges

https://betrac.github.io/

https://www.linkedin.com/feed/update/urn:li:activity:7482403480888975360/

the winners of the Beyond Transcription Challenge! 🏆

Lightweight track (<6B params, no tools)
1. TalTech
2. NTT-HI-CS
3. KUSLP

Heavyweight track (<36B params)
1. TalTech
2. KUSLP
3. NTT-HI-CS

🧪 The Challenge
Teams were given 1,100 hours of fully synthetic doctor-patient conversations with reference SOAP notes (conversations roleplayed by Gemma 3, notes generated by Kimi K2, from our Interspeech paper), a list of allowed open-weight models and datasets, and one goal: build the best end-to-end audio-to-SOAP-note system possible.

📊 The Results
Systems were evaluated with an automated medical Concept F1 scorer on held-out conversations from the same distribution.
All three top teams converged on the same recipe: supervised fine-tuning on the references, followed by reinforcement learning with Concept F1 as the reward. Their systems are remarkable.
Crushing hallucinations: the best baselines and cascaded systems we evaluated — built from Qwen 3 and Whisper components — hallucinate on more than 20% of claims. The top competition systems brought that below 1%.

❓ But does it generalize?
The obvious objection: isn't this overfitting to synthetic data? And isn't Concept F1 a very limited metric?
So we tested it. During evaluation, teams also generated notes for 272 human-acted medical dialogues — not permitted for training, and with no reference SOAP notes. Across n = 19 submitted systems, we asked two questions:
1. Does synthetic performance predict real performance? Yes, almost exactly. Real-data Concept F1 tracks held-out synthetic Concept F1 with a slope of 0.89 (lightweight) and 0.94 (heavyweight) — essentially the identity line (r = 0.97–1.00).
2. Does Concept F1 predict LLM-as-a-judge quality? (judge pipeline using Gemma 4) Yes — r = 0.83–0.87. The agreement is tightest among the strongest systems and fans out below ~0.35 Concept F1, so the metric is most trustworthy exactly where it matters.
The synthetic data approach looks like a genuinely promising way forward. Plenty of open questions remain — but "train on synthetic, deploy on real" held up here.

Just Turkish TTS so not very applicable to wide audience but interesting design (only 200M DiT + 25 Hz VAE from VoxCPM2)

https://github.com/freyavoiceai/FreyaTTS

FAD optimization during training

https://github.com/voidful/fd-speech

Great part from MERL paper above

https://real-tse.github.io/assets/pdf/MERL-SA-Track2.pdf

VII. METRIC ATTACK

Many speech separation and target speech extraction systems optimize evaluation metrics either explicitly or implicitlyduring training, including metrics such as SI-SDR and PESQ. From the perspective of Goodhart’s Law, this practice
fundamentally compromises the validity of such metrics as evaluation tools: once a metric becomes an optimization target,
it ceases to function as an independent measure of system quality.

.....

Given the fragility of non-intrusive metrics as demonstrated by our attack and also shown in [37], [38], we suggest that
the Challenge Organizers either remove DNSMOS and spksim when calculating the official ranking or replace them
with alternative speech quality and speaker similarity metrics that were not attacked either advertently or inadvertently by
submitted systems

We've just released Zipformer Tajik model (you can use it with sherpa-onnx)

https://huggingface.co/alphacep/vosk-model-tg

Somewhat initial one, we will work more on it.