At ICRA 2009, the Rollin' Justin humanoid robot (the lovable robot that "Danced Like in Pulp Fiction") demonstrated some impressive teleoperation capabilities. The man-machine interface (MMI) consists of two components. The first component comprises two DLR-III lightweight arms, the same type employed by the robot, terminated with force-torque sensing load cells to command the omnidirectional base or the arms / hands. Meanwhile, the second component, a fully-immersive heads-up display with vicon (optical) head tracking, constantly streams robot-mounted camera images to the heads-up display while simultaneously panning and tilting the robot's head in concert with the user's head movements. All-in-all, this is a very impressively engineered system. Be sure to check out the pictures and video below.
The video below comes from the ICRA 2009 video session, entitled "Multimodal Telepresent Control of DLR Rollin' JUSTIN".
For the sake of convenience, I've made a few choice screen captures. On the left is a picture of the dual DLR-III Lightweight arms and the user donning the heads-up display with vicon tracking markers (the white dots), while on the right is the teleoperated Rollin' Justin humanoid robot.
As was shown in the video, mobility is controlled via one of the joysticks (shown below left), while the heads-up display showed camera images streaming from Justin (below right).
The video shows Justin manipulating an "umbilical-type" connector -- an impressive feat of dexterity. However, I'm not entirely sure how the MMI data is mapped into Justin's arm positions, and more importantly, to the dexterous finger motions / positions. For the arm control, a straight forward mapping of (position) changes in the MMI end-effectors could cause corresponding changes in Justin's end-effectors (via inverse kinematics); however, those precise finger movements would seem to require a bit more complexity. Perhaps someone can chime in to help clarify?
Anyway, that is a very impressive (and expensive!) system.
Comments (1)