סמינר מחלקה של מקסים מונסטירסקי - למידת זריקה מתוך דוגמאות בודדות באמצעות שימוש ב-Decision Transformers

School of Mechanical Engineering Seminar
wednesday, November 16, 2022, at 14:00
Wolfson Building of Mechanical Engineering, Room 206

Learning to Throw with a Handful of Samples using Decision Transformers

Max Monastirsky

MSc Avishay Sintov

Throwing objects by a robot extends its reach and has many industrial applications; providing better efficiency to many tasks such as packaging in warehouses, object transfer, conveyor belt management and recycling. While analytical models can provide efficient performance, they require accurate estimation of system parameters. Reinforcement Learning (RL) algorithms can provide an accurate throwing policy without prior knowledge. However, they require an extensive amount of real world samples which may be time consuming and, most importantly, pose danger. Training in simulation, on the other hand, would most likely result in poor performance on the real robot.

In this work, we explore the use of Decision Transformers (DT) and their ability to transfer from a simulation-based policy into the real-world. Contrary to RL, we re-frame the problem as sequence modeling and train a DT by supervised learning. The DT is trained off-line on data collected from a far-from-reality simulation through random actions without any prior knowledge on how to throw. Then, the DT is fine-tuned on a handful (~5) of real throws. Results on various objects show accurate throws reaching an error of approximately 4cm. Also, the DT can extrapolate and accurately throw to goals that are out-of-distribution to the training data. We additionally show that few expert throw samples, and no pre-training in simulation, are sufficient for training an accurate policy.

Join Zoom Meeting

https://tau-ac-il.zoom.us/j/4962025174?pwd=bVJUeElXRUUya3BERisyNllLOE9EZz09