(The talk will be given in English)

Speaker:     Dr. Amichai Paisky
                   Massachusetts Institute of Technology

Monday, December 25^th, 2017
15:00 - 16:00

Room 011, Kitot Bldg., Faculty of Engineering

Universal Loss and Gaussian Learning Bounds

Abstract

In this talk I address two fundamental predictive modeling problems: choosing a universal loss function, and how to approach non-linear learning problems with linear means.

A loss function quantifies the difference between the true values and the estimated fits, for a given instance of data. Different loss functions correspond to a variety of merits, and the choice of a "correct" loss could sometimes be questionable. Here, I show that for binary classification problems, the Bernoulli log-likelihood loss (log-loss) is universal with respect to practical alternatives. In other words, I show that by minimizing the log-loss we minimize an upper bound to any smooth, convex and unbiased binary loss function. This property justifies the broad use of log-loss in regression, in decision trees, as an InfoMax criterion (cross-entropy minimization) and in many other applications.

I then address a Gaussian representation problem which utilizes the log-loss. In this problem we look for an embedding of an arbitrary data which maximizes its "Gaussian part" while preserving the original dependence between the variables and the target. This embedding provides an efficient (and practical) representation as it allows us to consider the favorable properties of a Gaussian distribution. I introduce different methods and show that the optimal Gaussian embedding is governed by the non-linear canonical correlations of the data. This result provides a primary limit for our ability to Gaussianize arbitrary data-sets and solve complex problems by linear means.

Bio
Amichai Painsky is a Post-Doctoral Fellow, co-affiliated with the Israeli Center for Research Excellence in Algorithms (I-CORE ALGO) at the Hebrew University, and the Signals, Information and Algorithms (SIA) Laboratory at MIT. Amichai received his B.Sc. in Electrical Engineering from Tel Aviv University (2007), his M.Eng. in Electrical Engineering from Princeton University (2009) and his Ph.D. in Statistics from Tel Aviv University (2016). He is a recipient an outstanding Ph.D. students award from the school of Mathematical Sciences, the Weinstein Institute of Signal Processing and the Marejn Foundation. Previously, he received a Brain Return Ph.D. Scholarship from the Israeli Center for Returning Scientists.

(The talk will be given in English)

Speaker:     Dr. Roi Livni
                   Department of Computer Science, Princeton University

Wednesday, December 20^th, 2017
15:00 - 16:00

Room 011, Kitot Bldg., Faculty of Engineering

Overcoming Intractability in Learning

Abstract

Machine learning has recently been revolutionized by the introduction of Deep Neural Networks. However, from a theoretical viewpoint these methods are still poorly understood. Indeed the key challenge in Machine Learning today is to derive rigorous results for optimization and generalization in deep learning. In this talk I will present several tractable approaches to training neural networks. At the second part I will discuss a new sequential algorithm for decision making that can take into account metric structure in the action space and avoids erratic behavior which often makes MAB algorithm impractical.

I will present our work that provides some of the first positive results and yield new, provably efficient, and practical algorithms for training certain types of neural networks. In a second work I will present a new online algorithm that learns by sequentially sampling random networks and asymptotically converges, in performance, to the optimal network. Our approach improves on previous random features based learning in terms of sample/computational complexity, and expressiveness. In a more recent work we take a different perspective on this problem. I will provide sufficient conditions that guarantee tractable learning, using the notion of refutation complexity. I will then discuss how this new idea can lead to new interesting generalization bounds that can potentially explain generalization in settings that are not always captured by classical theory.

In the setting of reinforcement learning I will present a recently developed new algorithm for decision making in a metrical action space. As an application, we consider a dynamic pricing problem in which a seller is faced with a stream of patient buyers. Existing MAB algorithms often ignore the structure in the action space, and they are led to an erratic behavior that leads to sub-optimal regret in face of a strategic environment. Our algorithm achieves an optimal regret, and improves on previously known regret bound.

Bio
Roi Livni is a research instructor at the computer science department in Princeton University. He is a recipient of the Eric and Wendy Schmidt fellowship for strategic innovation, and a Yad Hanadiv fellow for postdoctoral position.
He completed his PhD at the Hebrew University, during which he was a recipient of a Google Europe fellowship in learning theory. He also served, during his graduate studies, as a long term intern at Microsoft Research. His graduate work won a Best paper award in ICML'13, and a best student paper award in COLT'13.

School of Mechanical Engineering Seminar
Monday, January 8, 2018 at 14:00
Wolfson Building of Mechanical Engineering, Room 206

Suspension-Colloidal Transport in Porous Media: Petroleum and Environmental Applications

Prof. Pavel Bedrikovetsky

Australian School of Petroleum

University of Adelaide

Flow of suspensions and colloids in porous media with particle capture, detachment and consequent permeability alteration occurs in aquifers and subterranean basins during exploitation of artesian wells, disposal of industrial wastes in aquifers and consequent contamination, fresh and hot water storage in aquifers and geothermal reservoirs, ocean water invasion into aquifers, industrial filtering, as well as exploitation of oil and gas production and injection wells.

We discuss the particulate transport in rocks with fines attachment and detachment accounting for particle- and pore size distributions. The basic equations form a stochastic population-balance system. The system can be upscaled in the cases of mono-sized particles and of small-concentrations. 1D linear and axi-symmetric problems for suspension injection or detachment of natural reservoir fines allow for exact solutions. The solutions allow analysing the propagation of concentration waves of injected colloids or lifted suspensions, its effects on well injectivity and productivity. The exact solutions allow also for downscaling, i.e. restauration of the micro-scale behaviour. Another application of the exact solutions is regularisation of inverse problems, allowing interpreting laboratory experiments and tuning the model parameters. We show simple lab and field devices for complete characterisation of suspension-colloidal system and lab-based reservoir-scale predictions.

The talk is completed by exploration of random-walk and Boltzmann’s models, and of two-phase suspension-colloidal transport.

Bio: Pavel is a Professor of Petroleum Engineering at the University of Adelaide. His research covers suspension and multiphase transport in porous media, including exact integration, upscaling, inverse problems, and technologies of enhanced gas and oil recovery. He authors a seminal book on reservoir engineering and 230 papers in academic journals and Society of Petroleum Engineers (SPE). He is 2008-2009 and 2016-2017 SPE Distinguished Lecturer.