Robotics Laboratory

Department of Computer Science | Iowa State University

Nonlinear Observability & Observers

Geometric sensing has several drawbacks. First, it always requires enough geometric constraints, which may result in high sensor expense. Second, manipulation often involves dynamic motions, which sometimes make purely geometric sensing approaches inapplicable for pose estimation. Third, by nature geometric sensing cannot estimate the motion of an object. Fourth, it is not feedback driven and therefore pliable to noises and uncertainties. Finally, geometric sensing is often performed at the beginning of a task and hence not well integrated into manipulation.

In the second and main part of my Ph.D. thesis I studied sensing approaches that take advantage of not only known geometry but also task mechanics as well. Part of my motivation came from blind grasping exemplified by the task of picking up a pen on the table while keeping your eyes closed. Your fingers fumble on the table until one of them touches the pen and (inevitably) starts pushing it for a short distance. While feeling the contact move on the fingertip, you can almost tell which part of the pen is being touched. Assume the pushing finger is moving away from you. If the contact remains almost stable, then the middle of the pen is being touched; if the contact moves counterclockwise on the fingertip, then the right end of the pen is being touched; otherwise the left end is being touched. Immediately, a picture of the pen configuration has been formed in your head so you coordinate other fingers to quickly close in for a grip.

(a) Pose and Motion from Contact

The example to the left tells us that the pose of a known shape may be inferred from the contact motion on a finger pushing the shape. The figure on the right shows two motions of a quadrilateral in different initial poses pushed by an ellipse under the same motion. Although the initial contacts on the ellipse were the same, the final contacts are quite far apart. Thinking in reverse leads to these questions:

I gave affirmative answers to the above questions in the general case. To accomplish this, I derived a dynamic analysis of pushing which yields a nonlinear system that relates through contact the object pose and motion to the finger motion. The contact motion on the fingertip thus encodes certain information about the object pose. Nonlinear observability theory was employed to show that such information is sufficient for the finger to “observe” not only the pose but also the motion of the object. Therefore a sensing strategy can be realized as an observer of the nonlinear dynamical system. Two observers were subsequently introduced. The first observer, based on the result of Gauthier, Hammouri, and Othman 1992, has its “gain” determined by the solution of a Lyapunov-like equation; it can be activated at any time instant during a push. The second observer, based on Newton’s method, solves for the initial (motionless) object pose from three intermediate contact points during a push.

To see how the observer works, the figure on the left shows a disk of radius 1cm at constant velocity 5cm/s pushing a 7-gon while observing its pose and motion. The snapshots were taken every 0.1s. Contact friction between the polygon and the disk was assumed to be large enough to allow only the rolling on the disk edge. The edge of the polygon in contact was assumed to be known. The coefficient of support friction was 0.3. (a) The scene of pushing for 0.71s. (b) The imaginary scene as “perceived” by the observer during the same time period. The observer constantly adjusted its estimates of the polygon’s pose and motion based on the moving contact on the disk boundary until they converged to the real pose and motion. Although the real contact and its estimate were about 4.5cm apart on the contact edge at the start of estimation, the error became negligible in 0.56s.

Besides conducting extensive observer simulations, I also implemented a force sensor for contact sensing using strain gauges. The sensor is composed of a horizontal disk with diameter 3cm and a cylindrical stainless steel beam erected vertically on the disk and attached to the gripper of an Adept robot at the top. Two pairs of 350 Ohm strain gauges are mounted on the upper end of the beam where they would be most sensitive to any force exerted on the disk. They would measure the contact force along two orthogonal directions, respectively. When contact friction is small enough, the contact force measured by the gauges would point along the disk normal at the contact, thereby indicating the contact location on the disk boundary.

The results on pose and motion from contact were presented in the following two award-nominated papers in the IEEE International Conferences on Robotics and Automation:

A refined version subsuming the above two papers appears as

(b) Local Observability of Rolling

I continued my study of local observability, focusing on three-dimensional tasks. I intended to look into the same type of information: contacts between objects and manipulators. As we would see, surface geometry around the contact plays the main role in kinematics. It affects not only how fast an object moves relative to a manipulator but also how fast it rotates, in response to a relative motion between the object and the manipulator. Meanwhile, dynamics deal with the rate of change of such relative motion under gravity and contact force (or simply, under the controlled manipulator motion).

The task chosen for our study, illustrated on the right, involves a horizontal plane, or a palm, that can translate in arbitrary directions, and a smooth object that can only roll on the palm. As the object rolls, its contact traces out a curve as function of time in the plane. Suppose the palm is covered by an array of tactile cells that are able to detect the contact location at any time instant. We would like to know if this curve of contact contains enough information for the palm to know about the configuration of the object. Our approach was to study local observability of rolling from the contact curve. Through cotangent space decomposition, we obtained a sufficient condition on local observability of the system. This condition depends only on the differential geometry of contact and on the object’s angular inertia matrix; it is satisfied by all but some degenerate shapes such as a sphere. We refer the interested reader to the relevant paper