This policy primer outlines some of the challenges around measuring AI model efficiency systematically, and the techniques being developed to improve model efficiency. It focuses on work that can be done at the model developer layer, as opposed to the hardware or energy supply layers.

In this research, we explore the possibility of achieving greater fairness by using an imbalanced dataset instead of a balanced one, and how the use of ensembles could further amplify this impact. We carry out the same analysis on real datasets, such as CheXpert and CelebA.

Progress in AI is driven largely by the scale and quality of training data. Despite this, there is a deficit of empirical analysis examining the attributes of well-established datasets beyond text. In this work we conduct the largest and first-of-its-kind longitudinal audit across modalities.

We introduce the Aya Expanse model family, a new generation of 8B and 32B parameter multilingual language models, aiming to address the critical challenge of developing highly performant multilingual models that match or surpass the capabilities of monolingual models.

This Policy Primer summarises several promising avenues to addressing the language gap in AI safety and identifies five recommendations for researchers and policymakers to consider in their efforts to improve AI safety for everyone.

Optimizing Merging at Scale Mitigates Performance Tradeoffs

Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation

This paper provides an analysis of existing available research surrounding two AI and biorisk threat models: 1) access to information and planning via large language models (LLMs), and 2) the use of AI-enabled biological tools (BTs) in synthesizing novel biological artifacts.

In this work, we construct an evaluation suite of 197,243 QA pairs from local exam sources to measure the capabilities of multilingual LLMs in a variety of regional contexts.