Jun 06, 2026
Findings of the WMT26 Multilingual Instruction Shared Task: Small Models Are Not Yet Polyglots
We present the findings of the WMT26 Multilingual Instruction Shared Task (MIST), evaluating small (≤10B parameter) open-weight language models across 24 languages on three subtasks: context-based QA, cross-lingual summarization, and open-ended generation.
Authors
Patrícia Schmidtová, Katia Artemova, Seth Aycock, Niyati Bafna, Shobhit Banga, Manmeet Kaur, Tom Kocmi, Philipp Koehn, Danni Liu, Nam Luu, Sara Papi, Beatrice Savoldi, Mariya Shmatova, Hanuman Sidh, Evfrosiniya Zerminova, Vilém Zouhar, Maike Züfle, Pinzhen Chen
Abstract
We present the findings of the WMT26 Multilingual Instruction Shared Task (MIST), evaluating small (≤10B parameter) open-weight language models across 24 languages on three subtasks: context-based QA, cross-lingual summarization, and open-ended generation. Submissions from eight teams explore fine-tuning, knowledge distillation, and prompt engineering. Our findings demonstrate that small models are not yet polyglots: while leading systems perform well on high-resource languages, performance drops sharply on low-resource varieties, and comprehension degrades significantly when transferring across non-English language pairs. In open-ended generation, human evaluation and automatic rule-based verification exhibit strong overall rank correlation (ρ=0.79) but diverge locally at the top: human evaluators favor distilled pipelines with superior fluency and cross-lingual equity, whereas automatic verifiers reward concise promptengineered systems. Furthermore, multi-stage pipelines suffer from prompt-language inertia on cross-lingual instructions, and summarization exhibits severe English leakage unless penalized by language gating. Finally, while baseline fluency is largely achieved by modern small models, multi-constraint instruction following and unanswerable question refusal remain the primary performance bottlenecks. We release all test sets, system outputs, human judgments, and evaluation code under a permissive license. 1 Introducti
Related works
