RI: Small:Exploring Efficient Bayesian Model-Augmentation Techniques for Decomposible Contrastive Representation Learning

Modern deep learning models that trains large on web-scale data with a heavy computational load are achieving state-of-the-art performance on many real-world problems. The ability to learning good representations from the large-scale unlabeled data is the key to the success. Although many methods have been developed, their underlying properties and limitations are still not well understood. Contrastive Representation Learning (CRL) is a method of learning to represent the data such that similar data are close to each other while dissimilar data are far apart. This project investigates the limitations of CRL in order to make it more scalable to big data and be appropriately applied to real-world problems such as computer vision. The project will also support the continued development of machine-learning related courses for undergraduate and graduate students at University at Buffalo as well as various outreach activities to expose K-12 students to the field of computer science. This research develops a scalable conditional decomposable contrastive learning framework from the Bayesian principle and extend it to the federated learning and multi-model learning. First, an augmentation technique will be applied to decouple the entanglement of positive-negative samples, leading to a conditional decomposable loss that can be optimized with unbiased stochastic gradients. Second, the technique will be further refined to develop a communication-efficient distributed training framework for CRL, where clients no longer need to explicitly communicate with other clients to fetch other negative samples. Third, the improved CRL technique will be further applied to the emerging field of foundation models for vision-and-language modeling. The project will also result in the dissemination of shared data and benchmarks to the broader AI community, for example through Github, presentation and workshop organization. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.