MBZUAI researchers constrain speech decoding to restore Arabic diacritics
MBZUAI researchers developed a constrained CTC decoder that uses speech to restore Arabic diacritics without altering the supplied transcript. In their tests, it beat a text-and-speech baseline, though both datasets consisted of carefully read rather than conversational speech.
Researchers at Mohamed bin Zayed University of Artificial Intelligence have developed a speech-based method for restoring omitted Arabic diacritics without allowing the system to rewrite the underlying transcript. The method presented at Interspeech 2026 narrows the speech recognizer’s job to choosing the pronunciation marks that belong after each fixed Arabic letter.
Arabic writing commonly omits short vowels and other pronunciation details that can be represented by marks above or below letters. Without those marks, a written word can have more than one pronunciation or meaning. Speech supplies acoustic cues that can resolve some of that ambiguity, while the provided transcript supplies the base characters that the output must preserve.
In their research paper, Rufael Marew, Amr Keleg and Hanan Aldarmaki describe a non-autoregressive system built on connectionist temporal classification, or CTC. At inference time, the system turns the undiacritized transcript into a linear-chain character lattice. Each original letter stays fixed in sequence and is followed by a wildcard position. There, the decoder can select either one composite diacritic label or the CTC blank symbol, meaning no diacritic. The resulting graph admits only output strings compatible with the original letter sequence. It can be combined with the CTC decoding graph or used to limit beam-search expansions to the labels allowed at each position.
That constraint operates during decoding; the acoustic model still has to be trained on speech paired with diacritized transcripts. In the reported comparisons, the proposed method and a Text+ASR baseline used the same speech-recognition model. The baseline generated a diacritized ASR hypothesis, then fused it with the original undiacritized text using two Transformer encoders and cross-attention. It also benefited from text-model pretraining on the Tashkeela corpus, while the constrained system used the ASR model directly.
The largest gap appeared in the authors’ cross-dataset tests. A model trained on the Classical Arabic ClArTTS corpus and tested on the Modern Standard Arabic ArVoice corpus recorded a 12.04% diacritic error rate, compared with 19.21% for Text+ASR. When the training and test corpora were reversed, the rates were 11.86% and 76.61%, respectively. With both corpora used for training, the constrained system recorded 3.80% on ClArTTS and 8.69% on ArVoice, versus 9.05% and 9.93% for Text+ASR. Diacritic error rate measures the share of character positions whose predicted mark differs from the reference, including positions that should have no mark and marks at word endings.
The figures are author-reported and have not been independently replicated in the sources reviewed. For the combined-training comparison, the authors used 2,000 bootstrap samples and say the 95% confidence intervals for the error-rate difference remained below zero on both datasets. The paper does not include a direct runtime, latency, memory, parameter-count or energy comparison to support its broader efficiency claim.
The evaluation also covers a narrow speech setting. ClArTTS contains 12 hours of carefully read, single-speaker Classical Arabic from a classical religious book, including a 0.3-hour test set. The selected ArVoice portions contain six hours of multi-speaker, read Modern Standard Arabic news, including a 0.9-hour test set. In its announcement of the work, MBZUAI said the method has not been tested on more conversational datasets. Performance on spontaneous, dialectal, noisy or disfluent speech therefore remains unestablished.
More news

MBZUAI and Inception42 launch six Jais 3 models and Jais-ASR

MBZUAI releases PAN 2 weights and code, with key training stages missing

MBZUAI introduces MedRoundsQA benchmark for diagnostic conversations
