CSAIL Research Abstracts - 2005 link to http://publications.csail.mit.edu/abstracts/abstracts05/index.html link to http://www.csail.mit.edu
bullet Introduction bullet Architecture, Systems
& Networks
bullet Language, Learning,
Vision & Graphics
bullet Physical, Biological
& Social Systems
bullet Theory bullet

horizontal line

A Behavior Based Approach to Humanoid Robot Manipulation

Aaron Edsinger


Current approaches to humanoid robot manipulation typically rely upon detailed models of the manipulator and the object being manipulated. For robots to gain general utility in areas such as space exploration, small-parts assembly, agriculture, end even in our homes, they must be able to intelligently manipulate unknown objects in unstructured environments. A behavior based approach to manipulation can provide the tight visual and sensorimotor coupling to the world that is required during engagements in unknown and unstructured environments. To this end, we are developing an approach to humanoid robot manipulation which integrates force sensing and compliant manipulators into a behavior based control decomposition.

The manipulation platform Domo being developed for this work has 29 active degrees of freedom (DOF), 58 proprioceptive sensors, and 24 tactile sensors. 22 DOF use force controlled and compliant actuators . There are two 6 DOF force controlled arms, two 4 DOF force controlled hands, a 2 DOF force controlled neck, and a 7 DOF active vision head. The real-time sensorimotor system is managed by an embedded network of five DSP controllers. The vision system includes two FireWire CCD cameras. The cognitive system runs on a small, networked cluster of PCs.


We have developed an initial experiment with Domo involving visually guided reaching towards and grasping of an object. The experiment utilizes a single fixed camera coordinated with one arm and one hand. A rudimentary vision system designed to segment a specialized set of objects using motion cues and appearance based features provides a visual target for reaching. We use a kinematic approach to hand localization in order to predict the occurrence of the hand in the visual field. We then apply a set of force based manipulator behaviors to guide the robot hand to the target while avoiding collisions with the body. Finally, a set of grasping behaviors preshape the grasp before contact and execute the grasp based on the visual servoing error. This experiment is illustrated below.


A video sequence of the robot performing visually guided preshaping, reaching, and grasping.(A) The robot preshapes the grasp for a vertically oriented object. (B) The robot preshapes the grasp for a horizontally oriented object. (C,D) The start of a reaching sequence. (E)Reaching towards the target with a preshaped hand. (F) Grasp completion.

Research Support

This research is supported in part by Toyota Motor Corporation.


[1] Aaron Edsinger-Gonzales and Jeff Weber. Domo: A Force Sensing Humanoid Robot for Manipulation Research. In Proceedings of the 2004 IEEE International Conference on Humanoid Robots, Santa Monica, Los Angeles, CA, USA. November, 2004.

[2] Robert Platt, Oliver Brock, Andrew H. Fagg, Deepak R. Karuppiah, Michael T. Rosenstein , Jefferson A. Coelho Jr., Manfred Huber, Justus H. Piater, David Wheeler, Roderic A. Grupen. A Framework For Humanoid Control and Intelligence, In Proceedings of the 2003 IEEE International Conference on Humanoid Robots,2003

[3] Lorenzo Natale, Linking Action to Perception in a Humanoid Robot: A Developmental Approach to Grasping. Doctoral Thesis. LIRA-Lab, DIST, University of Genoa. 2004

[4] J. Pratt, M. Chew, P. Dilworth and G. Pratt, Virtual Model Control: An Intuitive Approach for Bipedal Locomotion. In International Journal of Robotics Research. 20:2.192-143. 2001

horizontal line

MIT logo Computer Science and Artificial Intelligence Laboratory (CSAIL)
The Stata Center, Building 32 - 32 Vassar Street - Cambridge, MA 02139 - USA
tel:+1-617-253-0073 - publications@csail.mit.edu
(Note: On July 1, 2003, the AI Lab and LCS merged to form CSAIL.)