Apple researchers introduce DiscoSign for discourse-aware ASL gloss translation

Apple researchers have introduced DiscoSign, an LLM-based framework designed to preserve discourse information while translating English into American Sign Language glosses. Rather than translate each sentence in isolation, the system carries decisions about spatial references, question-answer clauses and recurring concept-to-gloss mappings from one sentence to the next.

The paper reports stronger discourse-metric results than two baselines, but it describes research, not a product launch. It does not announce a public API, Apple product integration, deployment or end-to-end sign-video system. Readers tracking Apple’s production AI work can compare the research with Apple Intelligence in Apple’s redesigned Health app, but the DiscoSign paper makes no connection between the two.

Using Gemini 2.5 Pro, DiscoSign scored 0.84 on Spatial Coreference Accuracy and 0.97 on Concept-Gloss Consistency on Aesop’s Fables, the authors report. The sentence-level baseline scored 0.29 and 0.59 on those measures. DiscoSign’s Question-Answer Clause Appropriateness score was 0.76, compared with 0.70 for the sentence-level baseline and 0.72 for a context-only baseline. These results come from the researchers’ discourse-specific evaluation suite, not independent measures of real-world translation accuracy.

Glosses are written labels for individual signs and can serve as an intermediate representation before downstream sign production. DiscoSign processes one English sentence per model call and retains three registries. The spatial registry records where entities have been assigned, allowing later pronouns, repeated mentions, possessives and directional verbs to reuse the same indices. The concept registry records an established ASL gloss for a recurring concept or synonym. A third registry tracks whether the system used a Question-Answer Clause, in which a signer poses and answers a question as one discourse unit. Independent linguistic research on ASL describes these clauses as topic-comment structures that depend on discourse conditions.

For each new sentence, DiscoSign places the accumulated registries and earlier sentence-translation pairs into the prompt as constraints. The model returns a gloss translation and structured metadata. A deterministic post-processor then checks spatial indices and concept-gloss choices, replacing violations without another model call. The verified metadata updates the registries for the next sentence. All systems in the reported tests were restricted to 2,859 glosses from ASLLRP SignBank.

The discourse evaluation covered 284 Aesop’s Fables passages averaging five sentences. Separate single-sentence tests included 500 ASL STEM Wiki examples and 2,121 examples from a licensed dataset. Because the fables had no expert-authored ASL gloss references, the authors used their own discourse metrics and back-translation for that part of the evaluation. That limits what the reported scores establish about translation quality. Conventional metrics also changed little across the Gemini configurations: chrF ranged from 39.4 to 41.7 and COMET from 0.76 to 0.78.

The authors repeated the discourse test with the open-weight Qwen3.6-35B-A3B backbone. DiscoSign again led both baselines on the three discourse measures, scoring 0.61 for spatial coreference, 0.86 for concept-gloss consistency and 0.75 for question-answer clause appropriateness. Its spatial result remained below the Gemini-based system’s 0.84.

The human evaluation was narrower: two ASL-fluent evaluators rated 32 stories containing 152 sentences. Automated spatial and concept-consistency measures showed significant combined correlations with human ratings. Correlations for question-answer clause appropriateness were not significant, and the evaluators differed in their agreement about whether a QAC was present.

The authors list several limitations: dependence on changing LLM behavior, no gold discourse-level ASL annotations, and deterministic QAC rules that cannot capture every optional use.