EE Seminar: Space-Time Layout using a Neural Network

~~Speaker:   Shahar Ben Ezra, 
M.Sc. student under the supervision of Prof. Daniel Cohen-Or and Nahum Kiryati

Wednesday, January 6th, 2016 at 15:00
Room 011, Kitot Bldg., Faculty of Engineering

Space-Time Layout using a Neural Network

Abstract

Cameras are now ubiquitous in our lives. We carry them everywhere and record daily and casual activities without hesitation.

A given activity is often captured by multiple people from different viewpoints resulting in a sizable collection of photo footage even for a single event. We present a method that effectively organizes and summarizes this spatio-temporal content.

Given an unorganized collection of photos taken by a number of photographers, capturing some dynamic event at a number of time-steps roughly simultaneously, we would like to organize the collection into a Space-Time table. The organization is an embedding of the photos into an order-preserving view-point and time-steps clusters.

Our method relies on a self-organization map (SOM), which is a neural network that embeds the training data (the set of images) into a discrete domain. We introduce BiSOM, which is a variation of SOM that considers two features (space and time) rather than one, to organize the given photo collection in the table.