(The talk will be given in English)

Speaker:     Prof. Or Ordentlich

                                School of Computer Science and Enginnering, Hebrew University of Jerusalem

Monday, March 24^th, 2024

13:00 - 14:00

Optimal Quantization for Matrix Multiplication

Abstract

The main building block of large language models is matrix multiplication, which is often bottlenecked by the speed of loading these matrices from memory. A possible solution is to trade accuracy for speed by storing the matrices in low precision (“quantizing” them). In recent years a number of quantization algorithms with increasingly better performance were proposed (e.g., SmoothQuant, Brain compression, GPTQ, QuIP, QuIP#, QuaRot, SpinQuant). In this work, we prove an information theoretic lower bound on achievable accuracy of computing matrix product as a function of compression rate (number of bits per matrix entry). We also construct a quantizer (based on nested lattices) achieving this lower bound. Applying our nested lattice scheme for quantizing weights, KV-cache, and activations of Llama-3-8B to 4 bits, yields smaller perplexity than state-of-the-art quantization schemes.

Based on joint work with Yury Polyanskiy, and with Semyon Savkin and Eitan Porat  

https://arxiv.org/pdf/2410.13780

https://arxiv.org/pdf/2502.09720 

Short Bio

Or Ordentlich is an associate professor in the School of Computer Science and Engineering at the Hebrew University of Jerusalem. His research focuses on information theory, and its application to modern problems in communication, compression and data science. Or received the B.Sc. (cum laude), M.Sc. (summa cum laude), and Ph.D. degrees from Tel Aviv University, Israel, in 2010, in 2011, and 2016, respectively, all in electrical engineering. During the years 2015-2017 he was a postdoctoral fellow in the Laboratory for Information and Decision Systems at the Massachusetts Institute of Technology (MIT), and in the Department of Electrical and Computer Engineering at Boston University. He has been serving as an associate editor for Signal Processing and Source Coding in the IEEE Transactions on Information Theory since 2021.

השתתפות בסמינר תיתן קרדיט שמיעה = עפ"י רישום שם מלא + מספר ת.ז. בטופס הנוכחות שיועבר באולם במהלך הסמינר

הפקולטה להנדסה מזמינה את קהילת אוניברסיטת תל אביב ליריד "עוטף ישראל", שכבר הפך למסורת. ביריד יתארחו עסקים קטנים מהדרום ומהצפון, של מילואימניקים ושל עקורים, ואתם.ן תוכלו ליהנות מהמוצרים ומהסחורה הכי טובה, לקראת חג הפסח.

האירוע יתקיים ביום רביעי | 2.4 | 14:00-10:00 | ברחבת הכניסה לבניין הפקולטה

ביריד תוכלו לקנות:

• עוגות ועוגיות ממושב עלמה 
• קונפיטורת פטל מנתיב העשרה
• תמרים מדרום ים המלח
• גבינות של מילואימניק
• שמן זית מרמת הגולן
• יין מנתיב העשרה
• בירה לזכרו של סמל תומר נגר ז"ל
• עציצים משדרות
• זרי פרחים מיובשים מכוכב מיכאל
• שמן חוחובה מכפר רתמים
• מוצרי עץ של נגר מילואימניק
• תכשיטים מקריית שמונה
• קרמיקה מאלוני הבשן
• מארזי עיצוב ותוכן ממושב יבול
• ספרי מתח ודרמה של סופר מנתיב העשרה
• סימניות מתכת מעוצבות מאופקים
• ירקות ופירות
• סוקלנטים מנתיב העשרה
• סיכות, מגנטים, חולצות, כובעים ושלל מוצרים של מטה משפחות החטופים

The Tel Aviv University community is invited to the "Israel Envelope" fair, which has already become a tradition at the Faculty of Engineering. The fair will host small businesses from the south and the north, reservists and displaced persons, and you will be able to enjoy the best products and merchandise, in preparation for Passover.

The event will take place on Wednesday | 2.4 | 10:00-14:00 | at the entrance to the faculty building

Here’s what you’ll find available for purchase:

Cakes and cookies from Alma

Raspberry jam from Netiv HaAsara

Dates from the southern Dead Sea

Cheese from a reservist

Olive oil from the Golan Heights

Wine from Netiv HaAsara

Beer in memory of Sgt. Tomer Nagar, z"l

Potted plants from Sderot

Dried flower bouquets from Kochav Michael

Jojoba oil from Kfar Retamim

Wooden products from a reservist carpenter

Ceramics from Alonei Habashan

Design and content gift boxes from Yevul

Thriller and drama books by an author from Netiv HaAsara

Decorative metal bookmarks from Ofakim

Fresh vegetables and fruits

Succulents from Netiv HaAsara

Pins, magnets, T-shirts, hats, and a variety of products from the Families of the Hostages Organization

We look forward to your presence at the fair and to supporting these communities together.

  -סמינר זה יחשב כסמינר שמיעה לתלמידי תואר שני ושלישי-  This Seminar Is Considered A Hearing Seminar For Msc/Phd Students

Electrical Engineering Systems Seminar

Speaker: Sagi Ben Itzhak

M.Sc. student under the supervision of Prof. Nahum Kiryati

Wednesday, 26^th March 2025, at 15:00

Room 011, Kitot Building, Faculty of Engineering

Localization-Guided Supervision for Robust Medical Image Classification by Vision Transformers

Abstract

A major challenge in developing data-driven algorithms for medical imaging is the limited size of available datasets. Furthermore, these datasets often suffer from inter-site heterogeneity caused by the use of different scanners and scanning protocols. These factors may contribute to overfitting, which undermines the generalization ability and robustness of deep learning classification models in the medical domain, leading to inadequate performance in real-world applications. To address these challenges and mitigate overfitting, we propose a framework which incorporates explanation supervision during training of Vision Transformers (ViT) models for image classification. Our approach leverages foreground masks of the class object during training to regularize attribution maps extracted from ViT, encouraging the model to focus on relevant image regions and make predictions based on pertinent features. We introduce a new method for generating explanatory attribution maps from ViT-based models and construct a dual-loss function that combines a conventional classification loss with a term that regularizes attribution maps. Our approach demonstrates superior performance over existing methods on two challenging medical imaging datasets, highlighting its effectiveness in the medical domain and its potential for application in other fields.

השתתפות בסמינר תיתן קרדיט שמיעה לתלמידי תואר שני ושלישי = עפ"י רישום שם מלא + מספר ת.ז. בדף הנוכחות שיועבר באולם במהלך הסמינר