EE Seminar: Coordinate Descent for Network Linearization
Electrical Engineering Systems ZOOM Seminar
Speaker: Vlad Rakhlin
M.Sc. student under the supervision of Prof. Shai Avidan
Sunday, 8th September 2024, at 15:00
Join Zoom Meeting
https://zoom.us/j/96616415160?pwd=6HxwuIEBriJVMIgqaRVwMntidNsUZM.1
Meeting ID: 966 1641 5160
Passcode: 0Wpex3
Coordinate Descent for Network Linearization
Abstract
Private Inference lets a service provider and a client conduct inference without revealing information on either side. PI is known to be slow in practice, in large part because computing ReLU securely is expensive in terms of communication bandwidth. Reducing the number of ReLUs is termed Network Linearization. Reducing ReLU count is a discrete optimization problem and there are two common ways to approach it. Most current state-of-the-art methods are based on a smooth approximation that jointly optimizes network accuracy and ReLU budget at once. However, the last hard thresholding step of the optimization usually introduces a large performance loss. We take an alternative approach that works directly in the discrete domain by leveraging Coordinate Descent in ReLU space as our optimization framework. In contrast to previous methods, this yields a sparse solution by design. We demonstrate, through extensive experiments that our method is State of the Art on common benchmarks.
השתתפות בסמינר תיתן קרדיט שמיעה = עפ"י רישום בצ'ט של שם מלא + מספר ת.ז.
