Cohere For AI - Guest Speaker: Hailey Schoelkopf, Research Scientist

Abstract / Description: Hailey will be presenting on two of her recent works on LLM evaluation , Lessons From the Trenches on Reproducible Evaluation of Language Models and Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive? She will be talking about her work maintaining the LM Evaluation Harness, challenges faced and lessons learned for effective evaluation of LLMs, and discuss building a science of effective evaluation.

Paper:https://arxiv.org/html/2405.14782v1

Bio: Hailey Schoelkopf is a research scientist at EleutherAI, a non-profit AI research lab. Hailey's main research focuses include reproducible and reliable LLM evaluation, and large-scale efficient training and inference of DL models. Her work has often focused on open access to models and tooling for open science, having released projects to serve as research infrastructure such as the Pythia and Llemma language models, and maintaining software tools such as the LM Evaluation Harness.