(The talk will be given in English)

Speaker:     Dr. Yossi Gandelsman

                        UC Berkeley and Meta

Monday, January 13^th, 2025

12:00 - 13:00

Interpreting the Inner Workings of Vision Models

Abstract

In this talk, I will present an approach for interpreting the internal computation in deep vision models. I will show that these interpretations can be used to detect model bugs and to improve the performance of pre-trained deep neural networks (e.g., reducing hallucinations from image captioners and detecting and removing spurious correlations in CLIP) without any additional training. Moreover, the obtained understanding of deep representations will be utilized to unlock new model capabilities (e.g., novel identity editing techniques in diffusion models and faithful image inversion in GANs). I will demonstrate how to find common representations across different models (discriminative and generative) and how deep representations can be adapted at test time to improve model generalization without any additional supervision. Finally, I will discuss future work on automation of the presented interpretation techniques and their application to continual model correction and scientific discovery.

Short Bio

Yossi is a EECS PhD at UC Berkeley, advised by Alexei Efros, and a visiting researcher at Meta. Before that, he was a member of the perception team at Google Research (now Google-DeepMind). He completed his M.Sc. at Weizmann Institute, advised by Prof. Michal Irani. His research centers around deep learning, computer vision, and mechanistic interpretability.

השתתפות בסמינר תיתן קרדיט שמיעה לתלמידי תואר שני ושלישי = עפ"י רישום שם מלא + מספר ת.ז. בטופס הנוכחות שיועבר באולם במהלך הסמינר