Dr. Gal Vardi - On Implicit Bias and Benign Overfitting in Neural Networks

(The talk will be given in English)

Speaker:     Dr. Gal Vardi

TTI-Chicago and the Hebrew University

hall, Electrical Engineering-Kitot Building‏ 011

Monday, January 15^th, 2024

15:00 - 16:00

On Implicit Bias and Benign Overfitting in Neural Networks

Abstract

When training large neural networks, there are typically many solutions that perfectly fit the training data. Nevertheless, gradient-based methods often have a tendency to reach those which generalize well, namely, perform well on test data, and understanding this “implicit bias” has been a subject of extensive research. Surprisingly, trained networks often generalize well even when perfectly fitting noisy training data (i.e., data with label noise), a phenomenon called “benign overfitting”. In the first part of the talk, I will discuss the implicit bias and its implications. I will show how the implicit bias can lead to good generalization performance, but also has negative implications in the context of susceptibility to adversarial examples and privacy attacks. In the second part of the talk, I will discuss benign overfitting and the settings in which it occurs in neural networks.

Short Bio

Gal is a postdoctoral researcher at TTI-Chicago and the Hebrew University, hosted by Nati Srebro and Amit Daniely as part of the NSF/Simons Collaboration on the Theoretical Foundations of Deep Learning. Prior to that, he was a postdoc at the Weizmann Institute, hosted by Ohad Shamir, and a PhD student at the Hebrew University, advised by Orna Kupferman. His research focuses on theoretical machine learning, with an emphasis on deep-learning theory.

השתתפות בסמינר תיתן קרדיט שמיעה = עפ"י רישום שם מלא + מספר ת.ז. בטופס הנוכחות שיועבר באולם במהלך הסמינר