EE Seminar: On the Stability of Deep Networks and its Relationship with Compressed Sensing and Metric Learning

~~
Speaker:   Dr. Raja Giryes
                        EE, Tel Aviv University

Monday, April 11th, 2016
15:00 - 16:00
Room 011, Kitot Bldg., Faculty of Engineering

On the Stability of Deep Networks and its Relationship with Compressed Sensing and Metric Learning

Abstract
This lecture will address the fundamental question: What are deep neural networks doing to metrics in the data? We know that two important properties of a classification machinery are: (i) the system preserves the important information of the input data; (ii) the training examples convey information for unseen data; and (iii) the system is able to treat differently points from different classes. We show that these fundamental properties are inherited by the architecture of deep neural networks. We formally prove that these networks with random Gaussian weights perform a distance-preserving embedding of the data, with a special treatment for in-class and out-of-class data. Similar points at the input of the network are likely to have the same output. The theoretical analysis of deep networks presented exploits tools used in the compressed sensing and dictionary learning literature, thereby making a formal connection between these important topics. The derived results allow drawing conclusions on the metric learning properties of the network and their relation to its structure; and provide bounds on the required size of the training set such that the training examples would represent faithfully the unseen data. The results are validated with state-of-the-art trained networks.

Bio: 
Raja Giryes is a faculty member in the school of electrical engineering at Tel Aviv University. His research interests lie at the intersection between signal and image processing and machine learning, and in particular, in deep learning, inverse problems, sparse representations, and signal and image modeling. More details in web.eng.tau.ac.il/~raja

 

11 באפריל 2016, 15:00 
חדר 011, בניין כיתות-חשמל 
אוניברסיטת תל אביב עושה כל מאמץ לכבד זכויות יוצרים. אם בבעלותך זכויות יוצרים בתכנים שנמצאים פה ו/או השימוש
שנעשה בתכנים אלה לדעתך מפר זכויות, נא לפנות בהקדם לכתובת שכאן >>