Research Abstracts - 2007

Figure 1: Robot grasping an object [1].

Introduction

Finding a way to provide intelligent humanoid robots with the ability to grasp objects has been a question of great interest. Figure 1, a robot grasping an object, is an example of one such effort [1]. In the project, we explore an efficient and robust method to decide grasps given new objects using example-based learning. The method finds grasps for objects geometrically similar to those the system has seen before, in which object shapes are modeled as 3-D polygon meshes. For example, if the system has been shown a cup being grasped by the handle, it should now be able to grasp any new cup.

There are two problems to be solved in order to adapt example grasps to new objects. First, the system should be able to retrieve objects that are geometrically similar to the given object from the database storing previously seen objects. After collecting objects the system knows how to grasp successfully, it needs to adapt example grasps to new object. In the next section, we describe our basic approach and current progress.

Approach and Progress

To retrieve similar objects from the database, we use a spherical harmonics representation [2]. The descriptor in [2] is invariant under rotation and object retrieval using this descriptor is known to be a high-performance 3D object retrieval method. Once a set of promising objects is found, the system adapts the corresponding demonstration grasps to generate grasps for the new object. The basic idea is to transfer grasps by finding a direct correspondence between vertices on the example object and vertices on the new object. Based on this correspondence information, the example grasp's contact points are directly transferred onto the other object. This transfer, together with the retrieval method described above, completes the system that chooses an appropriate grasp when given a new object.

We formulate the problem of finding the correspondence between two meshes as the problem of finding an embedding of the data mesh into the demonstration mesh. The same framework was originally used in a correspondence algorithm for unsupervised registration of nonrigid surfaces [4]. Each data mesh point is associated with a correspondence variable that specifies the corresponding demonstration mesh point. The embedding is encoded as an assignment to all correspondence variables of the data mesh. We define a joint probability distribution over the correspondence variables, which is represented as a Markov network.

The model enforces preservation of local and global features such as local mesh geometry and geodesic distance between corresponding point pairs. The system optimizes the joint probabilistic model over all point-to-point correspondences. This is done by performing probabilistic inference to find the most likely joint assignment to the entire set of correspondence variables. Currently, loopy belief propagation is used for performing inference.

The framework of the algorithm described above has been implemented. Figure 2 is an example of running the algorithm on two cup meshes. The patches with the same color are matched using partial correspondence information. Currently, it only gives a coarse matching. This can be improved by achieving a finer matching from the correspondence.

Figure 2: Cup meshes (Left: data mesh, Right: demonstration mesh). Corresponding patches of two objects are painted with the same color using the correspondence information after 10 iterations of loopy belief propagation.

Future Work

The spherical harmonics descriptor mentioned above is designed for retrieving similar shapes. For our problem, the grasp area is more important than the overall geometry. By eliminating or loosening the constraint that the overall shapes of two objects be similar, we can make the system more general. For example, if the handles of cups and suitcases are similar, it would be possible to grasp suitcases based on grasps of cup handles. In this case, the shape retrieval method should be changed to emphasize only parts of shapes. In a similar vein, [5] provides an efficient shape matching algorithm based on the sparse shape information associated with hand pose that is represented as contact points and normals, which are important for grasp function.

Using only two types of signatures as described above is not enough to make the system robust. For example, if objects are symmetric, geodesic distance does not give enough information for belief propagation to converge. It is necessary to include more signatures that capture the features of shapes. In the future, different types of potentials other than shape context potential and geodesic distance potential will be explored. Also, we plan to emphasize the geometric constraints of the example grasp's contact points when searching for a correspondence, to make the grasp more likely to succeed.

In addition, methods to transfer the example grasp onto the data mesh will be studied. Once the entire system is implemented, it will work without any human interaction. Transferred grasps will be evaluated to check if the grasp is valid and stable. This evaluation will involve the use of inverse kinematics as done in [1]. If the generated grasp is evaluated to be a successful grasp in a physics-based simulation, the data object and the generated grasp can be added to the database for future use as a demonstration grasp.

References:

[1] K. Hsiao, Imitation Learning of Whole-Body Grasps, Proceedings of IROS, Beijing, China, October 2006.

[2] M. Kazhdan, T. Funkhouser, and S. Rusinkiewicz. Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors. Proceedings of the 2003 Eurographics/ACM SIGGRAPH symposium on Geometry processing, 2003.

[3] S. Belongie, J. Malik, and J. Puzicha. Shape matching and object recognition using shape contexts. IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 24, pg 509-522, Apr 2002.

[4] D. Anguelov, P. Srinivasan, D. Koller, S. Thrun, H. Pang and J. Davis. The Correlated Correspondence Algorithm for Unsupervised Registration of Nonrigid Surfaces. Proceedings of the Neural Information Processing Systems (NIPS) Conference, 2004.