Cohere For AI - Guest Speaker: Yifei Wang, Postdoctoral Researcher

Short bio: Yifei Wang is a postdoc at MIT CSAIL, working with Prof. Stefanie Jegelka. Prior to that, he obtained his PhD from Peking University. He is broadly interested in machine learning and representation learning, with a focus on bridging the theory and practice of self-supervised learning. His work is recognized by the Best ML Paper Award of ECML-PKDD 2021 and the Silver Best Paper Award of ICML 2021 AdvML workshop. He served as an Area Chair and a Session Chair for ICLR 2024, as well as a regular reviewer for main machine learning conferences. His homepage is https://yifeiwang77.com/.

Abstract: Non-negative Contrastive Learning (NCL), a renaissance of Non-negative Matrix Factorization (NMF) aimed at deriving interpretable features. The power of NCL lies in its enforcement of non-negativity constraints on features, reminiscent of NMF's capability to extract features that align closely with sample clusters. NCL not only aligns mathematically well with an NMF objective but also preserves NMF's interpretability attributes, resulting in a more sparse and disentangled representation compared to standard contrastive learning (CL). Theoretically, we establish guarantees on the identifiability and downstream generalization of NCL. Empirically, we show that these advantages enable NCL to outperform CL significantly on feature disentanglement, feature selection, as well as downstream classification tasks. At last, we show that NCL can be easily extended to other learning scenarios and benefit supervised learning as well.