School of MechanicalEngineering Seminar

Wednesday, March 16, 2022 at 14:00
Wolfson Building of Mechanical Engineering, Room 206

Learning human intention recognition through natural motion for intuitive human-robot interaction

Nadav D. Kahanowich

M.Sc. student of  Dr. Avishai Sintov

     Many tasks performed by two humans require mutual interaction between arms such as handing over tools and objects. In order for a robotic arm to interact with a human in the same way and for other applications of Human-Robot Collaboration (HRC), it must reason about the location of the human arm and hand-held object in real-time. Furthermore and to acquire interaction in a timely manner, the robot must be able predict the final target of the human and detect if and what type of object is held by the human, in order to plan and initiate motion beforehand.

     In this research, we explore the use of two low-cost wearable devices. The first device is equipped with an array of Force sensitive resistor (FSR) sensors, to classify hand-held objects [1]. The second is equipped with two inertial measurement units (IMU) for learning reaching motion [2]. The wearable devices can replace or be complementary to visual perception in cases of bad lighting or occlusions in a cluttered environment. For each of the systems, we have collected large datasets from real human motions. For  grasped object classification system, we have trained and optimized an ANN and proposed a novel algorithm which increases prediction robustness. For the second system, we have trained and optimized two separate models (ANN and RNN) to estimate the wrist position and predict user’s reach, with promising results.

[1]    N. D. Kahanowich and A. Sintov, "Robust Classification of Grasped Objects in Intuitive Human-Robot Collaboration Using a Wearable Force-Myography Device," in IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 1192-1199, April 2021, doi: 10.1109/LRA.2021.3057794.

[2]     N. D. Kahanowich and A. Sintov, "Learning Human-arm Reaching Motion Using IMU in Human-Robot Collaboration", Collaborative robotics, in submission

Join Zoom Meeting https://us02web.zoom.us/j/82108132163?pwd=Z2h4UzNzUS9mbXplT0lMU1pZenFEQT09