MBZUAI study finds speech-model updates leak age and accent signals

Researchers led by Mohamed bin Zayed University of Artificial Intelligence found that updates from three federated speech-recognition models could reveal a speaker’s age group or accent even when the attacker could not access the target’s audio. In the controlled white-box experiments, age-group inference reached 100% accuracy for Wav2Vec2, 97% for HuBERT and 94% for Whisper; native-versus-non-native English classification reached 100%, 80% and 93%, respectively, according to the study.

The results demonstrate feasibility under a narrow threat model; they do not show how common such attacks are in deployed products. The researchers assumed a passive server-side attacker that followed the federated-learning protocol but could inspect both the global model and a target speaker’s locally fine-tuned model. Each target model was fine-tuned on one utterance from one speaker. Neither source reports an attack against a real service.

Federated learning keeps training data on a user’s device while sending model updates to a server. To simulate the attack, the researchers fine-tuned “shadow” copies of Wav2Vec2-Base, HuBERT-Large and Whisper-Small on public speech samples with known speaker attributes. They reduced each model’s parameter tensors to summary statistics, then classified an unseen update by comparing it with the average update signature for each group. The method did not require the target’s raw audio, transcript or metadata.

The age test used 70 male native-English speakers from the Speech Accent Archive, split evenly between ages 18–24 and 35–44, with five-fold cross-validation. The binary accent test used 200 speakers, evenly divided between native and non-native English speakers, with a 75/25 train-test split. The speakers read the same scripted English passage, limiting what the results show about broader populations or conversational speech.

A finer-grained Wav2Vec2 test used 15 training samples and 20 unseen test speakers for each of 10 accents. The attack scored at least 90% for every tested accent. Performance varied across other attributes: gender accuracy ranged from 46% to 64% across the three models, while emotion and dysarthria results varied by model and task.

The researchers also tested whether broader accent exposure could reduce leakage. After further fine-tuning Wav2Vec2 on 137 samples spanning the 10 frequent accents, no covered accent exceeded 20% attack accuracy. The experiment did not test differential privacy, secure aggregation, update clipping, privacy noise, multi-client aggregation or training with many local utterances. Neither the paper nor MBZUAI’s account establishes that this form of attribute inference is occurring in deployed federated speech systems.