סמינר המחלקה להנדסת תעשייה

Estimating Information-Theoretic Measures with Classification Algorithms

Yuval Shalev

  Ph.D candidate at the Department of Industrial Engineering in the Tel-Aviv University, 

  under the guidance of Prof. Irad Ben-Gal and Dr. Amichai Painsky.
 
https://us02web.zoom.us/j/88556814331?pwd=REJvREpGNzVJUlQ4Y0lRSFczTHJkUT09
 
   

Abstract:
 
Estimating the information-theoretic measures of discrete random variables is a fundamental problem in information theory and related fields. This problem has many applications in various domains, including machine learning, statistics and data compression. Over the years, a variety of estimation schemes have been suggested. However, despite significant progress, most methods still struggle when alphabet size is large, random variables are from high dimensional space or when time dimension is involved. In this work, we introduce a holistic solution for this problem that is based on classifiers models. A set of these models is trained on the joint distribution of the random variables ,to obtain estimators for fundamental measures such as entropy, mutual information and transfer entropy. We focus on estimating these measures in scenarios where other methods usually fail. We  provide an experimental study of several use cases that demonstrates the advantages of our proposed scheme over state-of-the-art methods.
 
Bio:
 
Yuval Shalev is a Ph.D candidate at the Department of Industrial Engineering in the Tel-Aviv University,   under the guidance of Prof. Irad Ben-Gal and Dr. Amichai Painsky. Previously, he completed his M.Sc studies in physics in the Hebrew University. His research interests include Machine learning, deep learning and their connection to information theory. In parallel to its academic studies, Yuval has been working in the financial industry since 2008 as a quant and a data scientist.  Today, He leads the research team in the data science group of Citi innovation lab in Israel.