Sep 10, 2026
Building Multilingual Bridges
Data Mixing as the Pillar of Generalization for In-Language Reasoning Optimized data mixtures including multilingual reasoning, non-reasoning and English reasoning data yield superior in-language reasoning. We build Tiny Aya L2-Thinker, a RLM supporting in-language reasoning on over 60 languages without sacrificing performance, the broadest coverage to-date for multilingual RLMs.
Authors
Mehrnaz Mofakhami, Ananya Sahu, Alejandro R. Salamanca, Daniel D'souza, Alexandre Bérard, Thomas Euyang, Marzieh Fadaee, Julia Kreutzer
Abstract
Reasoning language models have substantially advanced on a variety of complex tasks, yet the capability remains overwhelmingly English-centric: even when prompted in another language, models predominantly reason in English. This is inaccessible for non-English-speaking users, risks losing the intent of the original question, and forgoes knowledge more readily expressed in the target language. In this work, we advance L2 reasoning, the ability of a model to reason consistently in the language of the user's prompt, thus building an in-language bridge between the prompt and the answer. We approach this problem from a data-centric angle, investigating how to optimize data composition and scheduling in SFT for reasoning generalization. Building on top of a massively multilingual 3.35B base model, we achieve above 95% L2 reasoning rate across 60 languages on 5 benchmarks spanning math, commonsense reasoning, instruction following and open-ended generation, while keeping performance strong. We show the path to generalizing L2 reasoning to held-out languages goes through broader language coverage, readily available multilingual non-reasoning data, and a sufficient English reasoning backbone. These findings indicate that reasoning is a language-agnostic behavior that can be transferred across typologically diverse languages through careful data mixing and without requiring reasoning supervision in every target language. We release our model weights and multilingual reasoning data to support further research on accessible, in-language reasoning.
Related works
Research
The IOL-AI Challenge: An Open Challenge towards Advancing Linguistic Reasoning
Read
Research
Findings of the WMT25 Multilingual Instruction Shared Task: Persistent Hurdles in Reasoning, Generation, and Evaluation
Read
Research
CALIBER: Calibrating confidence before and after reasoning in language models
Read