COT-TTS: Audio Context-Aware Text-to-Speech with Chain-of-Thought Reasoning

https://luckybian.github.io/COT-TTS/

Similar ideas, TTS properties could be more predictable and explainable

https://huggingface.co/WhissleAI/steer-tts

Recently a good Arabic ASR was released:

https://huggingface.co/IFM/Jais-ASR

Good results on telephony, just a little behind the public leader (Openai GPT Audio 1.5) but still significantly behind custom finetuned models. For huge volume and smaller domain a simple conformer/zipformer can do much better than any advanced model but might fail on proper names and complex vocabulary.

Some other leaderboard results (not complete) here:

https://huggingface.co/spaces/elmresearchcenter/open_universal_arabic_asr_leaderboard

https://lnkd.in/p/eh-i56jw

https://opensvbench.xyz/

https://arxiv.org/abs/2610.13760

OpenSVBench: Diagnosing Speaker Verification Generalization Across Deployment Scenarios
Qituan Shangguan, Bosen Xu, Shuai Wang

Speaker verification (SV) systems now achieve very low error rates on standard public protocols, but those results do not fully describe deployment generalization. VoxCeleb-style evaluations remain useful anchors for English in-the-wild media speech. Scenario-specific evaluations and challenges are also valuable. However, neither setting directly shows whether the same system remains reliable across diverse deployment scenarios. We introduce OpenSVBench, a public benchmark for cross-scenario SV evaluation. Its fixed public core contains 26 trial sets organized into 11 scenario groups, with 9.15M verification trials, frozen target/non-target keys, and metadata for scenario-wise reporting. Around this core, the online leaderboard acts as a diagnostic evaluation interface rather than only a ranked list. It accepts score packages from new systems, evaluates them with frozen keys and common scoring, and reports trial-set breakdowns, scenario-group metrics, macro EER/minDCF, model-wise profiles, and scenario-wise comparisons. We evaluate 18 public checkpoints spanning x-vector, ECAPA-TDNN, CAMPPlus, ResNet and ERes variants, ReDimNet2, SAM-ResNet, and pretrained Transformer speaker recipes. The results show clear cross-scenario gaps: systems with similar macro EER can have different scenario profiles, strong performance in one scenario does not guarantee strong performance in another, and same-scenario trial sets can still expose source and protocol gaps.

Recently released MingTTS supposed to be good and just 0.5B params. And some other models too:

https://x.com/AntLingAGI/status/2112210811997540475

Today, we're open-sourcing 5 Ming-3.0 models across 3 speech families.

- Ming-3.0-tts: 0.5B voice design & paralinguistic cues; Tiny 7.9B A1.3B long-form speech.
- Ming-3.0-audio: Flash 126B A6.5B understanding & tools; Tiny 9.3B A2.7B efficient dialogue.
- Ming-3.0-asr: Tiny 8.6B A2B dialects & context.

Voice is somewhat robotic, not sure how it got good score

https://x.com/ArtificialAnlys/status/2112389524466929979/video/1

Not that the TTS is great but I find that using two personas for assistant is psychologically interesting thing

https://x.com/JonPTaylor/status/2112629213211828652

Lately there is a boom in tiny models, feels like everyone tired of growth. For example

https://github.com/yinruiqing/tiny-transducer

1M params transducer based on https://arxiv.org/abs/2101.06856

or this 1M TTS

https://huggingface.co/VantoraLabs/Vocetta2-1m

What is interesting the UTMOS reported is above 4.0 which assumes a good quality while in practice there is a clear metallic noise

https://huggingface.co/VantoraLabs/Vocetta2-1m/blob/main/samples/sample_3.wav

Feels like guys found a way to hack UTMOS getting good metrics without actual sound quality. No more UTMOS in our evaluation anymore then

https://huggingface.co/AuroraAI-Research/AuroraTTS-100K is another example

samples here

https://huggingface.co/AuroraAI-Research/AuroraTTS-100K/tree/main/samples

Encoder might be useful, not public though

https://arxiv.org/abs/2610.31292

This paper introduces Spark-Audio 1.0, a speech foundation model trained on domestically produced computing infrastructure. The model accepts text and speech inputs and generates text outputs. It supports multilingual (99 languages) and multi-dialect (202 dialects) speech recognition, speech translation, audio perception and understanding, and both text- and speech-based dialogue, marking a shift from merely ``hearing clearly'' to genuinely ``understanding'' speech. Architecturally, Spark-Audio 1.0 couples a 0.65B-parameter dense audio encoder with a 30B-A3B Mixture-of-Experts (MoE) large language model (LLM). The audio encoder is pretrained with representation-enhanced BEST-RQ and then refined through multi-task supervised training. The audio--LLM is trained through staged pretraining, followed by post-training that combines supervised fine-tuning with on-policy distillation. In total, the model is trained on 13 million hours of audio data together with large-scale text data. We evaluate Spark-Audio 1.0 on mainstream audio and text benchmarks, as well as on in-house test sets drawn from real-world application scenarios. Despite using substantially less audio training data than Qwen3.5-Omni-Flash, Spark-Audio 1.0 achieves comparable performance on most audio and text tasks, outperforms it in multilingual speech recognition, spoken dialogue, and real-world application scenarios, and approaches the performance of larger closed-source models. Nevertheless, there remains room for improvement in instruction following and complex audio reasoning. Finally, the successful training of Spark-Audio 1.0 on domestic hardware, including Huawei Ascend 910B and Hygon BW1000 chips, demonstrates the feasibility of a domestic technical pathway for building speech foundation models.

Some new Turkish models (and datasets/etc)

ASR

https://huggingface.co/erayyapagci/balbal-v1
https://huggingface.co/atasoglu/seda-v0.2

TTS

https://huggingface.co/kasparovabi/drama-3-tr-speaker

Time to go multimodal, great dataset for audio/video things

https://github.com/FreedomIntelligence/TalkVid

https://github.com/higurashi-ai/FastDialogueSidon

FastDialogueSidon is a fast one-step model for full-duplex two-speaker dialogue separation and joint restoration: a raw 16 kHz mono mixture (telephone / conversational audio, up to 120 s per window) goes in, two separated 24 kHz speaker tracks come out. Every module — encoder, DiT head and decoder — is trained for FastDialogueSidon. A single DiT step per window replaces the 30 diffusion steps of DialogueSidon, and a lightweight decoder (a frame-rate SAN-WaveNeXt2 (WaveFitNeXt) latent decoder, 30 M parameters) replaces its DAC decoder. Running on fused GPU kernels, it is ~100x faster than the official DialogueSidon release run as-is, at matching intelligibility.

Deepgram is acquiring Daily (company behind Pipecat)

https://x.com/kwindla/status/2115791491586986432

https://huggingface.co/spaces/Edge0/Audio8-TTS-Realistic-0.6b

Introducing Audio8-TTS-Realistic, @Edge0_AI’s new open-weight speech model: realistic, zero-shot voice cloning in just 0.6B parameters.

• Human-like prosody and expressive delivery
• Clones a voice from one short clip, in 23 languages
• 601M params: small enough to fine-tune and serve affordably

Currently #1 on the Hugging Face TTS Arena, where people vote blind on which voice sounds more human.

Open weights are available now on @huggingface. Try it free.

1. How it learns to sound human:

TTS is usually trained on two things: the right words (WER) and the right voice (similarity).

But neither captures rhythm, pauses, or timing.

So we trained a realism critic that scores each output against a real person saying the same sentence in the same voice—ranked via a blind listening study.

Then: SFT → GRPO on accuracy + similarity → GRPO on the critic.

That last stage only saw Chinese and English prompts, yet it improved all 13 lower-resource languages.

2. How it stacks up at 0.6B:

• CV3 (9 languages): 4.86% error — ahead of CosyVoice3-1.5B (6.89%), Qwen3-TTS-1.7B (7.73%), and VoxCPM2 (5.35%)
• MiniMax (24 languages): 5.89% error — vs. 27.41% for Qwen3-TTS-1.7B
• Runs in ~1.1 GB RAM on an Apple M2 CPU (INT4)

Larger 4–8B models still lead on some sets, but we’re within reach at a fraction of the size.

3. Audio8-TTS-Realistic is now open source.

Quick adaptation to a domain is still a big problem, an interesting concept from Whissle from our Reddit:

https://huggingface.co/WhissleAI/arc-v1.1

Given the list of names it biases output to them and returns them already annotated in speech. However,the algorithm itself is a simple trie biasing during beam search and it doesn't always work as required tokens disappear in lower layers.

A good approach is to inject bias deeper in the layers, however, it is not very standard yet. An interesting recent work on optimizing NanoGPT training with ngrams might be the right direction

https://brunborg.com/n-gram-hinted-lm

Overall there is an interesting trend in improvement of CTC architectures with some LLM features. A good Turkish model
released recently:

https://huggingface.co/erayyapagci/balbal-v2-manas
https://erayy.com/balbal/

No publication for that, so quite unclear what the features bring but the features look interesting. Claude lists
the following related publications:

Lample, Sablayrolles, Ranzato, Denoyer, Jégou. Large Memory Layers with Product Keys. NeurIPS 2019.
https://arxiv.org/abs/1907.05242
The original PKM: two sub-key banks, top-k over k*k pairs, query BatchNorm, multi-head reads of one shared value table.

Kim, Jung. Large Product Key Memory for Pretrained Language Models. Findings of EMNLP 2020.
https://arxiv.org/abs/2010.03881
Memory slots going stale during training; adding PKM to a pretrained model, alongside the FFN rather than replacing it.

Berges, Oğuz, Haziza, Yih, Zettlemoyer, Ghosh. Memory Layers at Scale. ICML 2025 (PMLR 267).
https://arxiv.org/abs/2412.09764
Memory layers up to 128B memory parameters; largest gains on factual tasks.

Huang, Min, et al. (ByteDance Seed). Ultra-Sparse Memory Network (UltraMem). ICLR 2025.
https://arxiv.org/abs/2411.12364
PKM extended to 20M slots; faster inference than MoE of the same size.

DeepSeek-AI, Peking University. Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models (Engram). 2026.
https://arxiv.org/abs/2601.07372
Hashed N-gram lookup. Gate sigmoid(RMSNorm(h) . RMSNorm(W_K e) / sqrt(d)) on W_V e: the same structure as balbal's gate
(balbal uses LayerNorm and a learned bias starting at -2).

Nozaki, Komatsu. Relaxing the Conditional Independence Assumption of CTC-Based ASR by Conditioning on Intermediate Predictions
(Self-Conditioned CTC). Interspeech 2021.
https://www.isca-archive.org/interspeech_2021/nozaki21_interspeech.html
Intermediate CTC predictions fed back into the next layer's input; compare balbal's soft_feedback.

Chi, Salazar, Kirchhoff. Align-Refine: Non-Autoregressive Speech Recognition via Iterative Realignment. NAACL 2021.
https://arxiv.org/abs/2010.14233
Iterative Transformer refinement of CTC alignments in three passes.

Another model is Japanese

https://github.com/third-intelligence/Haas-ASR

Its prediction network uses Kimi Delta Attention from Kimi Linear, run on the kernels of flash-linear-attention

https://arxiv.org/abs/2510.26692
Kimi Linear: An Expressive, Efficient Attention Architecture

Massive effort. Datasets are not complete but often interesting finds

https://huggingface.co/spaces/kadirnar/stt-arena