Meet up with Cohere and learn from top NLP experts at industry and research conferences, webinars, and hackathons. 

Events - AI Conferences, NLP Webinars, LLM Hackathons | Cohere

Events

Title: Limits of Deep learning: Sequence modeling through the lens of complexity

Nikola Zubić is a PhD student with the Robotics and Perception Group, supervised by Professor Davide Scaramuzza (University of Zurich) and Associated Researcher at ETH AI Center. His research focuses on Sequence Modeling and Efficient Neural Network Architectures for Event-based Vision.

Cohere For AI - Nikola Zubić, PhD Candidate, University of Zurich

Limits of Deep learning: Sequence modeling through the lens of complexity

Abstract: Quantization techniques are widely used to improve inference speed and deployment of large language models. While a wide body of work examines the impact of quantized LLMs on English tasks, none have examined the effect of quantization across languages. We conduct a thorough analysis of quantized multilingual LLMs, focusing on their performance across languages and at varying scales. We use automatic benchmarks, LLM-as-a-Judge methods, and human evaluation, finding that (1) harmful effects of quantization are apparent in human evaluation, and automatic metrics severely underestimate the detriment: a 1.7% average drop in Japanese across automatic tasks corresponds to a 16.0% drop reported by human evaluators on realistic prompts; (2) languages are disparately affected by quantization, with non-Latin script languages impacted worst; and (3) challenging tasks such as mathematical reasoning degrade fastest. As the ability to serve low-compute models is critical for wide global adoption of NLP technologies, our results urge consideration of multilingual performance as a key evaluation criterion for efficient models.

Cohere For AI - Kelly Marchisio, Technical Staff Member, Cohere

How Does Quantization Affect Multilingual LLMs?

The future of AI Infrastructure: training models while preserving data privacy



 AI systems are trained on massive amounts of publicly accessible web data. However, the public web doesn't necessarily contain data about all domain areas (e.g., medical data, financial services data, and other signals of human well being), and the data that does exist is increasingly being clawed back through paywalls, rate limiters, and more restrictive licenses and terms of use[

]. In this talk, I describe a novel type of AI infrastructure which can facilitate training AI models on data which remains non-public, with the potential to unlock access to datasets which are not scrapable on the open internet. In doing so, I describe what I believe to be the future of AI training and evaluation infrastructure, tools and techniques which I believe will become essential to the AI community over the next 1-3 years.



 Andrew Trask leads the OpenMined community, a group of over 17,000 researchers, engineers, and developers who build and deploy open source infrastructure for accessing non-public datasets, AI models, and APIs. OpenMined has worked in this capacity with the UK AI Safety Institute, the United Nations, Reddit, Microsoft, Twitter/X, and was featured at the UN General Assembly in 2022, the Whitehouse PETs report in 2023, and the Royal Society's latest report on privacy enhancing technologies (also 2023). Andrew is also a PhD Candidate at the University of Oxford, Senior Research Scientist at Google DeepMind, and has taught multiple online courses (25,000+ students) and authored an introductory textbook called Grokking Deep Learning (20K copies sold).

Cohere For AI - Andrew Trask, Sr. Research Scientist, DeepMind

The future of AI Infrastructure: training models while preserving data privacy

Liliang Ren is a Senior Researcher in Microsoft GenAI, and a graduating PhD candidate in Computer Science at University of Illinois Urbana-Champaign, advised by Prof. ChengXiang Zhai. His current research interests lies in efficient long-context modeling and mixture of experts. He has published six first-authored papers at NeurIPS, ACL and EMNLP. He was a former research intern at Microsoft Research and Amazon Science. Website: 

Efficiently modeling sequences with infinite context length has been a long-standing problem. Past works suffer from either the quadratic computation complexity or the limited extrapolation ability on length generalization. 

In this work, we present Samba, a simple hybrid architecture that layer-wise combines Mamba, a selective State Space Model (SSM), with Sliding Window Attention (SWA). Samba selectively compresses a given sequence into recurrent hidden states while still maintaining the ability to precisely recall memories with the attention mechanism. We scale Samba up to 3.8B parameters with 3.2T training tokens and show that Samba substantially outperforms the state-of-the-art models based on pure attention or SSMs on a wide range of benchmarks. When trained on 4K length sequences, Samba can be efficiently extrapolated to 256K context length with perfect memory recall and show improved token predictions up to 1M context length. As a linear-time sequence model, Samba enjoys a 3.73x higher throughput compared to Transformers with grouped-query attention when processing user prompts of 128K length, and 3.64x speedup when generating 64K tokens with unlimited streaming. A sample implementation of Samba is publicly available in this https URL.

Cohere For AI - Liliang Ren 

Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling  

Join us for a deep dive into the simplicity and effectiveness of Byte Pair Encoding (BPE). We’ll break down the core concepts of BPE and show you how it works. BPE at its core is about finding patterns in text and figuring out which patterns are worth remembering. We will then take a brief overview of the implementation process of BPE from scratch in C.

Currently a final year undergrad , working as a Machine Learning Intern at Pibit.ai. His interests lie in Generative Models and Continual Learning.

Cohere For AI - Ashvanth S, Machine Learning & Research Intern

Simplicity & Effectiveness of Byte Pair Encoding (BPE)

The discussion will explore Common-Sense Bias Discovery (CSBD), a novel method leveraging image descriptions to identify and mitigate biases in image classification by adjusting sampling weights, significantly outperforming current bias mitigation techniques without needing sensitive labels.

The associated paper for the session is @ 



Miao's Bio: Miao is a a CS Ph.D. student at New York University (NYU) and a Graduate Researcher at the The Visualization and Data Analytics Research Center at NYU. (

) Her research direction lies at the intersection of computer vision and fairness/bias mitigation, applied on real-world environmental and healthcare problems.

Cohere For AI - Guest Speaker: Miao Zhang, CS Ph.D. student at NYU

Cohere for AI - Miao Zhang - Common-Sense Bias Discovery and Mitigation for Classification Tasks

About the session: We reformulate the problem of supervised learning as learning to learn with two nested loops (i.e. learning problems). The inner loop learns on each individual instance with self-supervision before final prediction. The outer loop learns the self-supervised task used by the inner loop, such that its final prediction improves. Our inner loop turns out to be equivalent to linear attention when the inner-loop learner is only a linear model, and to self-attention when it is a kernel estimator. For practical comparison with linear or self-attention layers, we replace each of them in a transformer with an inner loop, so our outer loop is equivalent to training the architecture. When each inner-loop learner is a neural network, our approach vastly outperforms transformers with linear attention on ImageNet from 224×224 raw pixels in both accuracy and FLOPs, while (regular) transformers cannot run.

Cohere For AI - Karan Dalal, EECS Student UC Berkeley 

We will have Charles Hernandez from the architectural optimization team at PyTorch talk about his work on TorchAO quantization. Charles also worked on developing the quantized kernels for gpt-fast and other projects with the PyTorch team.

Cohere For AI - Charles Hernandez, ML Research Scientist, Meta 

Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models



Abstract: Recent progress in large language models (LLMs) has enabled the deployment of many generative NLP applications. At the same time, it has also led to a misleading public discourse that “it’s all been solved.” Not surprisingly, this has, in turn, made many NLP researchers – especially those at the beginning of their careers – worry about what NLP research area they should focus on. Has it all been solved, or what remaining questions can we work on regardless of LLMs? To address this question, this paper compiles NLP research directions rich for exploration. We identify fourteen different research areas encompassing 45 research directions that require new research and are not directly solvable by LLMs. While we identify many research areas, many others exist; we do not cover areas currently addressed by LLMs, but where LLMs lag behind in performance or those focused on LLM development. We welcome suggestions for other research directions to include: 



Bio: Oana is a Postdoctoral Researcher at the University of Michigan and incoming Assistant Professor at Santa Clara University. Her research interests are at the intersection of language and vision, where she aims to develop equitable models that work equally well across demographics such as income, race, gender, language, and culture. She is passionate about research applications for social good impact, and the use of AI to attract and retain minorities in CS. Her work resulted in several publications in top conferences such as ACL and EMNLP. She is a co-organizer of the NLP for Positive Impact (NLP4PI) workshop at EMNLP 2024 and the SemEval 2024 Task on Emotion Recognition in Low Resource Languages. Oana is also involved in several outreach programs, including co-organizing the ACL Mentorship global panel sessions and many other events and workshops centered on improving diversity in CS.

Cohere For AI - Dr. Oana Ignat, CS Postdoctoral Researcher

Join us to mark the end of Expedition Aya, our six-week global open-build challenge designed to jumpstart multilingual research initiatives. This event will feature presentations from all participating teams, with updates on their progress. Tops projects will be awarded exclusive Expedition Aya swag, and Cohere API credits. Let’s celebrate multilingual research and look ahead to projects to come.

Expedition Aya - Closing Ceremony

Join us to mark the end of Expedition Aya, our six-week global open-build challenge designed to jumpstart multilingual research initiatives.

The discussion will explore training CLIP models with synthetic data, focusing on the paper's methodologies, their effectiveness versus real data, and future research and application impacts in AI and computer vision.

The associated paper for the session is @ 



Fabio's Bio: Fabio is a postdoctoral researcher in the Torr Vision Group (

) at the University of Oxford, working with Phil Torr. Previously, Fabio was a co-tutelle PhD at Vislab/Ambarella, located in Parma (Italy) and Inria Astra-vision team, in Paris (France), where he was supervised by Raoul de Charette. Fabio is currently working on large scale diffusion models for image generation, and continual learning.

Cohere For AI - Guest Speaker: Fabio Pizzati,Researcher Univ of Oxford

Cohere for AI - [C4AI] Fabio Pizzati - SynthCLIP: Are We Ready for a Fully Synthetic CLIP Training?

Nahid Alam is the Staff Machine Learning Engineer, Computer Vision at Cisco Meraki. Nahid has a MS in Computer Science from Clemson University.


Join Nahid Alam as she dives into her daily work with cutting-edge Computer Vision tools at Cisco Meraki. Discover the entire lifecycle of a Computer Vision project from planning to deployment, the role of team collaboration, and case studies demonstrating Computer Vision's versatility in solving complex industry challenges

Cohere For AI - Guest Speaker: Nahid Alam, ML Engineer

Cohere For AI - Guest Speaker: Nahid Alam - Behind the Screens 



Speaker Bio: Gwanghyun (Bradley) Kim is a Ph.D. candidate in Electrical and Computer Engineering (ECE) at Seoul National University (SNU), under the supervision of Prof. Se Young Chun. He completed his M.S. degree at KAIST, where he was advised by Prof. Jong Chul Ye.This year, he is interning at NVIDIA Research, working with Umar Iqbal, Xueting Li, and Ye Yuan. Last year, he interned at Google Research with Alonso Martinez, Krishna Somandepalli, and Yu-Chuan Su.His research focuses on artificial intelligence (AI), particularly in computer vision (CV) and its intersection with Generative AI. His work emphasizes multimodal, high-dimensional, and human-centric generative AI. Gwanghyun has received the Qualcomm Innovation Fellowship and the Yulchon AI Star Scholarship.

Title: 

BeyondScene: Higher-Resolution Human-Scene Generation With Pretrained Diffusion




Abstract: We propose BeyondScene, a novel framework that overcomes prior limitations, generating exquisite higherresolution (over 8K) human-centric scenes with exceptional text-image correspondence and naturalness using existing pretrained diffusion models. BeyondScene employs a staged and hierarchical approach to initially generate a detailed base image focusing on crucial elements in instance creation for multiple humans and detailed descriptions beyond token limit of diffusion model, and then to seamlessly convert the base image to a higher-resolution output, exceeding training image size and incorporating details aware of text and instances via our novel instance-aware hierarchical enlargement process that consists of our proposed high-frequency injected forward diffusion and adaptive joint diffusion. BeyondScene surpasses existing methods in terms of correspondence with detailed text descriptions and naturalness, paving the way for advanced applications in higher-resolution human-centric scene creation beyond the capacity of pretrained diffusion models without costly retraining. Project page:

Cohere For AI - Gwanghyun (Bradley) Kim, PhD Candidate in Gen AI 

We will explore an unsupervised method for analyzing large aerial scans without annotations, enhancing environmental monitoring and mapping, and improving decomposition accuracy and interpretability.Romain's Bio: Romain completed his PhD at École des Ponts, working with the IMAGINE team (

). His research focuses on optimization and machine learning for 3D data, with an emphasis on unsupervised learning, interpretability, and real-time applications.


Elliot's Bio: Elliot is a third-year PhD student (expected graduation in 09/2024) in the Imagine research group at École des Ponts ParisTech and the Willow team at Inria Paris (

). His research focuses on machine learning and computer vision for remote sensing, particularly satellite image time series analysis for tasks such as land use classification and change detection.

Cohere For AI - Romain Loiseau & Elliot Vincent

Learnable Earth Parser: Discovering 3D Prototypes in Aerial Scans

We tackle the challenge of improving factual accuracy in Medical Large Vision Language Models (Med-LVLMs) with our novel approach, RULE. Med-LVLMs often generate responses misaligned with medical facts. RULE uses two strategies: 1) Calibrated selection of retrieved contexts to control factuality risk, and 2) Fine-tuning models with a preference dataset to balance inherent knowledge and retrieved contexts. Our method achieves a 20.8% improvement in factual accuracy across three medical VQA datasets.

Peng works with Prof. Huaxiu Yao at UNC-Chapel Hill. Previously, Peng was a Ph.D. student in the Monash Medical AI Group at Monash University (2023.09-2024.05), advised by A/Prof. Zongyuan Ge. Peng earned a B.Eng. degree in AI Experimental Class from the School of Computer Science and Technology at Soochow University in 2023. Peng is deeply intrigued by large-scale vision-language models and their applications in medical imaging, focusing on enhancing their performance and resilience in various challenges.

Cohere For AI - Peng Xia, Research Intern, Univ. of North Carolina

RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language Models

Will explore Goldfish, a vision-language model for long video understanding, featuring an efficient retrieval mechanism and TVQA-long benchmark evaluation.


Kirolos is currently a Computer Vision Engineer at CreativAI. His research interests lie in computer vision and video analysis, specifically through the use of Multimodal Large Language Models.Kirolos spent about a year as a research intern at KAUST University.In 2023, He completed my master’s degree in AI and Data Science at Ottawa University in Canada. In 2020, he earned his bachelor’s degree in Computer Engineering from Shoubra Faculty of Engineering, Benha University in Egypt.

Cohere For AI - Kirolos Ataallah, Computer Vision Researcher 

Goldfish: Vision-Language Understanding of Arbitrarily Long Video

We will explore a data-driven feature tracker for event cameras, leveraging low-latency events to enhance tracking accuracy and robustness in high-speed motion and challenging conditions.

Nico's Bio: Nico Messikommer holds an M.Sc. in Robotics, Systems, and Control from ETH Zürich, Switzerland, which he obtained in 2019 following his B.Sc. in Mechanical Engineering in 2017. Currently pursuing a Ph.D. at the University of Zurich under the supervision of Prof. Davide Scaramuzza, his research centers on the intersection of computer vision and robotics, exploring topics such as unsupervised domain adaptation, event cameras, and reinforcement learning. His contributions were recognized by the nomination as an Award Candidate at the 2023 Conference on Computer Vision and Pattern Recognition (CVPR).

The associated paper for the session is 

Cohere For AI - Nico Messikommer 

Data-driven Feature Tracking for Event Cameras with and without Frames

Session Topic: We find neural scaling laws apply to galaxy images, and use this to build adaptable foundation models for new telescopes and new science questions.

Bio: Mike is an astrophysicist at the University of Toronto. He works on combining crowdsourcing and deep learning to measure the appearance of millions of galaxies, helping to understand why galaxies look the way they do. His galaxy foundation models have been used on every major telescope. He is interested in the "full stack" of practical problems with using models for science, from model-in-the-loop annotation to quantifying prediction biases.

The associated paper for the session is @ 

Cohere For AI - Nikola Zubić, PhD Candidate, University of Zurich

Upcoming Events

Cohere’s past events