NeurIPS

The Neural Information Processing Systems Foundation is a non-profit corporation that aims to foster the exchange of research advances in Artificial Intelligence and Machine Learning, principally by hosting an annual interdisciplinary academic conference with the highest ethical standards for a diverse and inclusive community.

Join Cohere and Cohere For AI at our booth, 1109!

Check out our papers, posters, speaking engagements, and appearances by Cohere and Cohere For AI.

Maintrack Publications

Tuesday, December 12

15:40, Scaling Data-Constrained Language Models, Niklas Muennighoff, Alexander M. Rush, Boaz Barak, Teven Le Scao, Aleksandra Piktus*, Nouamane Tazi, Sampo Pyysalo, Thomas Wolf, Colin Raffel. Location: Oral 2A

17:15, Scaling Data-Constrained Language Models, Niklas Muennighoff, Alexander M. Rush, Boaz Barak, Teven Le Scao, Aleksandra Piktus*, Nouamane Tazi, Sampo Pyysalo, Thomas Wolf, Colin Raffel. Location: Great Hall & Hall B1+B2 #813

17:15, The Goldilocks of Pragmatic Understanding: Fine-Tuning Strategy Matters for Implicature Resolution by LLMs, Laura Ruis*, Akbir Khan, Stella Biderman, Sara Hooker*, Tim Rocktäschel, Edward Grefenstette. Location: Great Hall & Hall B1+B2 #312

17:15, The Grand Illusion: The Myth of Software Portability and Implications for ML Progress, Fraser Mince, Dzung Dinh, Jonas Kgomo, Neil Thompson and Sara Hooker*. Location: Great Hall & Hall B1+B2 #1501

Wednesday, December 13

10:45, Intriguing Properties of Quantization at Scale, Arash Ahmadian*, Saurabh Dash*, Hongyu Chen*, Bharat Venkitesh*, Stephen Gou, Phil Blunsom*, Ahmet Üstün*, Sara Hooker*. Location: Great Hall & Hall B1+B2 #541

10:45, MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers, Lili Yu, Daniel Simig*, Colin Flaherty, Armen Aghajanyan, Luke Zettlemoyer, Mike Lewis. Location: Great Hall & Hall B1+B2 #541

17:00, Repetition In Repetition Out: Towards Understanding Neural Text Degeneration from the Data Perspective, Huayang Li, Tian Lan, Zihao Fu, Deng Cai, Lemao Liu, Nigel Collier, Taro Watanabe, Yixuan Su*. Location: Great Hall & Hall B1+B2 #421

17:00, On Imitation in Mean-field Games, Giorgia Ramponi, Pavel Kolev, Olivier Pietquin, Niao He, Mathieu Laurière, Matthieu Geist

Thursday, December 14

10:45, Improving Language Plasticity via Pretraining with Active Forgetting, Yihong Chen, Kelly Marchisio*, Roberta Raileanu, David Ifeoluwa Adelani, Pontus Stenetorp, Sebastian Riedel, Mikel Artetxe. Location: Great Hall & Hall B1+B2 #328

17:00, D4: Improving LLM Pretraining via Document De-Duplication and Diversification, Kushal Tirumala, Daniel Simig*, Armen Aghajanyan, Ari S. Morcos. Location: Great Hall & Hall B1+B2 #622

Workshop Papers

Friday, December 15

9:00, FAIR-Ensemble: Homogeneous Deep Ensembling Naturally Attenuates Disparate Group Performances, Wei-Yin Ko*, Daniel Dsouza, Karina Nguyen, Randall Balestriero, Sara Hooker*. Workshop on Algorithmic Fairness through the Lens of Time. Location: Room 252-254

9:00, When Less is More: Investigating Data Pruning for Pretraining LLMs at Scale, Max Marion, Ahmet Üstün*, Luiza A Pozzobon*, Alex Wang*, Marzieh Fadaee*, Sara Hooker*. Workshop on Attributing Model Behaviour at Scale (ATTRIB). Location: Room 271-273

Saturday, December 16

15:00, Generalizable Agents for Neural Network Optimisation, Kale-ab Tessera, Callum Tilbury, Sasha Abramowitz, Ruan de Kock W, Omayma Mahjoub, Benjamin Rosman, Sara Hooker*, Arnu Pretorius. Workshop on Advancing Neural Network Training (WANT): Computational Efficiency, Scalability, and Resource Optimization. Location: Room 243-245.

Research Talks

Monday, December 11

10:30-11:00, Beyond the Research Paper, Sara Hooker*. Latinx in AI Research.

Friday, December 15

9:30-10:00, What does scale give us: Why we are building a ladder to the moon. The Future of Attribution in ML. Sara Hooker*. ATTRIB Workshop. Location: Room 271-273

11:00-11:45, The Future of Attribution in ML. Panelists: Sara Hooker*, Pang Wei Kon. ATTRIB Workshop. Location: Room 271-273

12:00-12:30, Accelerating Multilingual AI Progress with AYA, Ahmet Üstün. Location: Hilton Garden Inn Convention Center

14:00-14:30, AYA: Multilingual Instruction Tuning and Open Challenges, Sara Hooker*. Instruction Tuning and Instruction Following.

16:15-17:15, Funding and institutional support for research at the intersection of machine learning and physical sciences. Panelists: Sara Hooker*, Jesse Thaler, Max Welling. ML and the Physical Sciences Workshop. Location: Room 275 - 277

Saturday, December 16

10:30-11:30, Panel on "Socially Responsible Language Modelling Research”. Moderator: Sara Hooker*, Panelists: Roger Grosse, Vinodkumar Prabhakaran, Stella Biderman, David Bau. SoLaR workshop

*Cohere and/or Cohere For AI team member