Voice is somewhat robotic, not sure how it got good score

https://x.com/ArtificialAnlys/status/2112389524466929979/video/1

Not that the TTS is great but I find that using two personas for assistant is psychologically interesting thing

https://x.com/JonPTaylor/status/2112629213211828652

Lately there is a boom in tiny models, feels like everyone tired of growth. For example

https://github.com/yinruiqing/tiny-transducer

1M params transducer based on https://arxiv.org/abs/2101.06856

or this 1M TTS

https://huggingface.co/VantoraLabs/Vocetta2-1m

What is interesting the UTMOS reported is above 4.0 which assumes a good quality while in practice there is a clear metallic noise

https://huggingface.co/VantoraLabs/Vocetta2-1m/blob/main/samples/sample_3.wav

Feels like guys found a way to hack UTMOS getting good metrics without actual sound quality. No more UTMOS in our evaluation anymore then

https://huggingface.co/AuroraAI-Research/AuroraTTS-100K is another example

samples here

https://huggingface.co/AuroraAI-Research/AuroraTTS-100K/tree/main/samples

Encoder might be useful, not public though

https://arxiv.org/abs/2610.31292

This paper introduces Spark-Audio 1.0, a speech foundation model trained on domestically produced computing infrastructure. The model accepts text and speech inputs and generates text outputs. It supports multilingual (99 languages) and multi-dialect (202 dialects) speech recognition, speech translation, audio perception and understanding, and both text- and speech-based dialogue, marking a shift from merely ``hearing clearly'' to genuinely ``understanding'' speech. Architecturally, Spark-Audio 1.0 couples a 0.65B-parameter dense audio encoder with a 30B-A3B Mixture-of-Experts (MoE) large language model (LLM). The audio encoder is pretrained with representation-enhanced BEST-RQ and then refined through multi-task supervised training. The audio--LLM is trained through staged pretraining, followed by post-training that combines supervised fine-tuning with on-policy distillation. In total, the model is trained on 13 million hours of audio data together with large-scale text data. We evaluate Spark-Audio 1.0 on mainstream audio and text benchmarks, as well as on in-house test sets drawn from real-world application scenarios. Despite using substantially less audio training data than Qwen3.5-Omni-Flash, Spark-Audio 1.0 achieves comparable performance on most audio and text tasks, outperforms it in multilingual speech recognition, spoken dialogue, and real-world application scenarios, and approaches the performance of larger closed-source models. Nevertheless, there remains room for improvement in instruction following and complex audio reasoning. Finally, the successful training of Spark-Audio 1.0 on domestic hardware, including Huawei Ascend 910B and Hygon BW1000 chips, demonstrates the feasibility of a domestic technical pathway for building speech foundation models.

Some new Turkish models (and datasets/etc)

ASR

https://huggingface.co/erayyapagci/balbal-v1
https://huggingface.co/atasoglu/seda-v0.2

TTS

https://huggingface.co/kasparovabi/drama-3-tr-speaker

Time to go multimodal, great dataset for audio/video things

https://github.com/FreedomIntelligence/TalkVid

https://github.com/higurashi-ai/FastDialogueSidon

FastDialogueSidon is a fast one-step model for full-duplex two-speaker dialogue separation and joint restoration: a raw 16 kHz mono mixture (telephone / conversational audio, up to 120 s per window) goes in, two separated 24 kHz speaker tracks come out. Every module — encoder, DiT head and decoder — is trained for FastDialogueSidon. A single DiT step per window replaces the 30 diffusion steps of DialogueSidon, and a lightweight decoder (a frame-rate SAN-WaveNeXt2 (WaveFitNeXt) latent decoder, 30 M parameters) replaces its DAC decoder. Running on fused GPU kernels, it is ~100x faster than the official DialogueSidon release run as-is, at matching intelligibility.

Deepgram is acquiring Daily (company behind Pipecat)

https://x.com/kwindla/status/2115791491586986432

https://huggingface.co/spaces/Edge0/Audio8-TTS-Realistic-0.6b

Introducing Audio8-TTS-Realistic, @Edge0_AI’s new open-weight speech model: realistic, zero-shot voice cloning in just 0.6B parameters.

• Human-like prosody and expressive delivery
• Clones a voice from one short clip, in 23 languages
• 601M params: small enough to fine-tune and serve affordably

Currently #1 on the Hugging Face TTS Arena, where people vote blind on which voice sounds more human.

Open weights are available now on @huggingface. Try it free.

1. How it learns to sound human:

TTS is usually trained on two things: the right words (WER) and the right voice (similarity).

But neither captures rhythm, pauses, or timing.

So we trained a realism critic that scores each output against a real person saying the same sentence in the same voice—ranked via a blind listening study.

Then: SFT → GRPO on accuracy + similarity → GRPO on the critic.

That last stage only saw Chinese and English prompts, yet it improved all 13 lower-resource languages.

2. How it stacks up at 0.6B:

• CV3 (9 languages): 4.86% error — ahead of CosyVoice3-1.5B (6.89%), Qwen3-TTS-1.7B (7.73%), and VoxCPM2 (5.35%)
• MiniMax (24 languages): 5.89% error — vs. 27.41% for Qwen3-TTS-1.7B
• Runs in ~1.1 GB RAM on an Apple M2 CPU (INT4)

Larger 4–8B models still lead on some sets, but we’re within reach at a fraction of the size.

3. Audio8-TTS-Realistic is now open source.

Quick adaptation to a domain is still a big problem, an interesting concept from Whissle from our Reddit:

https://huggingface.co/WhissleAI/arc-v1.1

Given the list of names it biases output to them and returns them already annotated in speech. However,the algorithm itself is a simple trie biasing during beam search and it doesn't always work as required tokens disappear in lower layers.

A good approach is to inject bias deeper in the layers, however, it is not very standard yet. An interesting recent work on optimizing NanoGPT training with ngrams might be the right direction

https://brunborg.com/n-gram-hinted-lm

Overall there is an interesting trend in improvement of CTC architectures with some LLM features. A good Turkish model
released recently:

https://huggingface.co/erayyapagci/balbal-v2-manas
https://erayy.com/balbal/

No publication for that, so quite unclear what the features bring but the features look interesting. Claude lists
the following related publications:

Lample, Sablayrolles, Ranzato, Denoyer, Jégou. Large Memory Layers with Product Keys. NeurIPS 2019.
https://arxiv.org/abs/1907.05242
The original PKM: two sub-key banks, top-k over k*k pairs, query BatchNorm, multi-head reads of one shared value table.

Kim, Jung. Large Product Key Memory for Pretrained Language Models. Findings of EMNLP 2020.
https://arxiv.org/abs/2010.03881
Memory slots going stale during training; adding PKM to a pretrained model, alongside the FFN rather than replacing it.

Berges, Oğuz, Haziza, Yih, Zettlemoyer, Ghosh. Memory Layers at Scale. ICML 2025 (PMLR 267).
https://arxiv.org/abs/2412.09764
Memory layers up to 128B memory parameters; largest gains on factual tasks.

Huang, Min, et al. (ByteDance Seed). Ultra-Sparse Memory Network (UltraMem). ICLR 2025.
https://arxiv.org/abs/2411.12364
PKM extended to 20M slots; faster inference than MoE of the same size.

DeepSeek-AI, Peking University. Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models (Engram). 2026.
https://arxiv.org/abs/2601.07372
Hashed N-gram lookup. Gate sigmoid(RMSNorm(h) . RMSNorm(W_K e) / sqrt(d)) on W_V e: the same structure as balbal's gate
(balbal uses LayerNorm and a learned bias starting at -2).

Nozaki, Komatsu. Relaxing the Conditional Independence Assumption of CTC-Based ASR by Conditioning on Intermediate Predictions
(Self-Conditioned CTC). Interspeech 2021.
https://www.isca-archive.org/interspeech_2021/nozaki21_interspeech.html
Intermediate CTC predictions fed back into the next layer's input; compare balbal's soft_feedback.

Chi, Salazar, Kirchhoff. Align-Refine: Non-Autoregressive Speech Recognition via Iterative Realignment. NAACL 2021.
https://arxiv.org/abs/2010.14233
Iterative Transformer refinement of CTC alignments in three passes.

Another model is Japanese

https://github.com/third-intelligence/Haas-ASR

Its prediction network uses Kimi Delta Attention from Kimi Linear, run on the kernels of flash-linear-attention

https://arxiv.org/abs/2510.26692
Kimi Linear: An Expressive, Efficient Attention Architecture

Massive effort. Datasets are not complete but often interesting finds

https://huggingface.co/spaces/kadirnar/stt-arena

More complete information about memory experiments from Erayy

https://erayy.com/balbal-blog/

Honestly, my impression is that architecture is not well tested, a dataset and tests could be better. Maybe if he trained English model first it would be easier to evaluate it. Optimizing from 38% to 37.5% on Turkish doesn't make much sense.

LFM3 Omni is cool but I don't understand something. It claims 140ms TTFA, but the actual delay in the video between input end and the response is 1.3 seconds.

https://x.com/xenovacom/status/2115923280079286543

Arabic gonna have some boost soon

https://huggingface.co/datasets/speechcolab/gigaspeech3

https://arxiv.org/abs/2611.02476

GigaSpeech 3: Scaling Multi-Dialect Arabic Speech Recognition with Weakly Labeled Data
Junzhe Liu, Yujie Tu, Yifan Yang, Peng Wang, Haoran Wang, Guodong Lin, Leyan Liao, Chengquan Zhang, Zhifu Gao, Xiangang Li, Qinyuan Cheng, Xipeng Qiu, Wei-Qiang Zhang, Guoguo Chen, Kai Yu, Xie Chen

Dialectal Arabic speech recognition remains constrained by the scarcity of dialect-resolved transcribed speech despite the abundance of web audio. We introduce GigaSpeech 3, a corpus of 39,000 hours and 10 million weakly transcribed speech segments covering Algerian, Egyptian, Iraqi, Saudi, and Syrian Arabic. Starting from 60 hours of manual transcription per dialect, we train labeling models using speech-recognition-biased self-supervised learning and apply them to dialect-matched web audio. Data selection and four rounds of noisy-student refinement for Saudi and Syrian Arabic yield the final corpus. We validate its utility through dialect-specific and multi-dialect speech recognition training and cross-domain evaluation. On GigaSpeechBench, the final dialect-specific models outperform their labeling models across all five dialects and achieve a macro-average word error rate of 23.86%, a 16.6% relative reduction from the strongest third-party system covering all five dialects, while outperforming the best third-party result on three dialects. A multi-dialect model using self-predicted dialect identities achieves a macro-average word error rate of 25.51% while also supporting Modern Standard Arabic. Across eight conditions from three external corpora, a GigaSpeech 3-trained Zipformer achieves the lowest word error rate among six evaluated models in every condition, while adapting the Qwen3 speech recognizer on GigaSpeech 3 improves performance across all eight conditions. We release the corpus and recipes as a foundation for dialect-aware Arabic speech processing.

Beside GPRO

A recursive self-improvement loop
We also use a recursive self-improvement (RSI) loop: our agent system interacts with simulated environments, reviews its failures, synthesizes targeted data, and trains the model on it before repeating the cycle. The simulation must reproduce the timing and variety of real interaction—interruptions, overlapping speech, side conversations, background noise—realistic enough to teach natural behavior and hard enough to expose weaknesses. The engineering target is the whole harness of simulators, tools, and judges, not an isolated model. For a small team, the loop turns edge cases into training data and lets us iterate with far less manual data work.

https://www.boson.ai/blog/higgs-realtime-duplex

160M TTS, singing feature is nice

https://github.com/lowkeytea/scyllasband

Hugging Face: https://huggingface.co/spybyscript/scyllasband

Try it in your browser: https://lowkeytea.dev/scyllasband/demo/

https://www.youtube.com/watch?v=GHkj44MaB9Y