Manipulation by Feel: Touch-Based Control with Deep Predictive Models

I Introduction

Imagine picking up a match stick and striking it against a matchbox to light it, a task you have performed with ease many times in your life. But this time, your hand is numb. In 2009, Johansson et al. performed this experiment, studying the impact of anesthetizing the fingertips of human subjects on their ability to perform this task. The results were striking: human subjects could barely manage to pick up a match stick, let alone light it. Videos of this experiment show human clumsiness that is strikingly reminiscent of the faltering, lurching struggles of modern robots .

Why did taking away the sensation of touch have such an impact on these subjects? Touch is unique among sensory modalities in that it is physically immediate and permits direct measurement of ongoing contact forces during object interactions, from which it is possible to infer friction, compliance, mass, and other physical properties of surfaces and objects. This knowledge is critical for manipulation tasks like matchstick striking. Visual sensing is a poor substitute: not only is it physically remote, but it is also usually occluded by the actuators at the points of contact. Manipulation without touch is perhaps akin to navigation without vision.

The importance of tactile sensing has long been acknowledged in the robotics community , but exploiting touch in robots has proven exceedingly challenging for three key reasons: (i) tactile sensing technology has largely been limited to sparse pointwise force measurements, a far cry from the rich tactile feedback of biological skin, (ii) accurate physics models of contacts and forces have remained elusive, and (iii) it is unclear how to even specify a desired tactile goal, such as holding a matchstick, or striking it obliquely against the matchbox.

In this paper, we show the promise of robotic control with rich tactile feedback, by exploiting recent advances to tackle these difficulties. Firstly, deformable elastomer-based tactile sensing (e.g., GelSight) has proven to be an extremely versatile, high-bandwidth, high-spatial resolution alternative to traditional tactile sensing . This provides our first stepping stone.

Secondly, while high-resolution touch sensing such as GelSight (see Figure 1 and 2) provides us with adequate sensory signals, we also need a control mechanism that can handle such high-dimensional observations and choose intelligent actions . Our second stepping stone comes from the deep learning literature, where deep models of high-dimensional data have been successfully employed to enable a variety of vision-based robotic skills .

Finally, as we will show, the combination of these two advances makes it feasible to plan towards tactile goals specified directly in the raw tactile observation space. Goal specification in this manner is not only much more informative than, say, in the space of forces at sparse contact points, but is also often much more natural for a user to specify.

Concretely, our contributions are as follows. We train deep model-based control policies that operate directly on observed raw high-dimensional tactile sensing maps. We show that such policies may be learned entirely without rewards, through diverse unsupervised exploratory interactions with the environment. Finally, we show how the desired manipulation outcomes for our policies may be specified as goals directly in the tactile observation space. We demonstrate and evaluate these contributions on a high-precision tactile ball rolling task, a joy-stick re-positioning task and a die rolling task: a robot arm with three linear axes is equipped with a tactile sensor at its end effector. Its goal is to move the end-effector into a configuration so that a desired tactile goal-pattern is measured with the sensor.

These tasks are designed to exhibit two key difficulties that are shared by a wide range of manipulation tasks: (i) An external object must be in contact with the robot end-effector in a specific desired configuration. This is a common feature of many manipulation and tool use problems. For example, when trying to light a match, a specific sequence of contact states needs to be attained. (ii) While performing the task, the object of interest, such as the ball bearing, becomes occluded from view, therefore controllers or policies that only have access to a remote visual observation are unable to solve the task. In the experiments for these three distinct manipulation tasks, our method outperforms hand-designed baselines for each task. We see these results as an important step towards integrating touch into solutions for general robotic manipulation tasks.

II Related Work

Prior work on touch-based control has proposed methods ranging from manual design of control laws to extracting and controlling high-level features from touch sensors . In contrast to these methods, our approach does not rely on pre-specified control laws and features. We learn a general-purpose predictive model that can be used to accomplish a variety of tasks at test time. Furthermore, we use a high-resolution touch sensor based on the GelSight design , which provides detailed observations of the contact surface in the form of a camera image.

Prior work has also used reinforcement learning to learn to stabilize an object with touch sensing and explored learning forward predictive models for touch sensors . While this prior work used low-dimensional readings from a BioTac sensor, our model operates directly on the raw observations of a GelSight sensor, which in a our case is an RBG image downsampled to 48x64 pixels. We demonstrate that comparatively high-resolution tactile sensing in conjunction with the proposed tactile MPC algorithm allows to reposition freely-moving objects according to user-specified goals, a more complex task than those demonstrated in prior work . To address this high-dimensional prediction problem, we build on recent work on control via video prediction . Prior work in this area has used video prediction in combination with model-predictive control to perform non-prehensile object repositioning from RGB camera images. To our knowledge, no prior work has used video prediction models together with touch sensing for touch-based object repositioning. Concurrent work learned a two dimensional latent space and dynamics model to perform control for following human demonstrations given in the tactile space, however handling of objects has not been shown yet.

A variety of different touch sensor designs have been proposed in the literature , though affordability, sensitivity, and resolution all remain major challenges. The BioTac sensor has been widely used in robotics research, particularly for grasping , but it provides only a limited number of measuring channels (i.e., 19 or 22 for different configurations). The GelSight design, which consists of a camera that observes deformations in a gel, offers good resolution, though at the cost of latency . In our case, this tradeoff is worthwhile, because the resolution of the sensor allows us to precisely reposition objects in the finger. GelSight-style sensors have been used in a number of prior works for other applications, including tracking , inserting USB connectors , estimating object hardness , and grasping . To our knowledge, our work is the first to employ them for object repositioning with learned predictive models.

III Tasks and Hardware Setup

While our aim is to develop control approaches for general manipulation, we focus on three representative tactile control tasks: rolling a ball to a specified position, manipulating an analog stick from a game controller, and rolling a die to a specified face, see Figure 2. Each of these tasks presents unique challenges, making them well-suited for evaluating our approach. In this section, we describe the tasks, the sensor, and the experimental hardware setup.

The ball repositioning task requires the robot to move a ball bearing to a target location on the sensor, which requires careful modulation of the contact force between the ball and the finger without the ball slipping out of the finger. Since the ball is underactuated, properly balancing the forces is critical.

Analog stick deflection task

The second task requires the robot to deflect an analog stick. This task presents an additional challenge: the robot must intentionally break and reestablish contact to deflect the analog stick in the desired direction, since sliding the finger along the stick will deflect it in undesirable ways. We encourage the reader to view the supplementary video for an illustration. The only way to perform the task successfully is to lift the finger off of the stick by moving vertically, repositioning it, bringing it back down onto the stick, and then deflecting the stick in the desired direction. This adds the difficulty of making the system only partially observable (when the finger loses contact with the joystick), and requires the model to acquire a more fine grained understanding of contact dynamics.

Rolling a 20-sided die

This third task requires the robot to roll a 20-sided die so that a specified die face is facing upwards. This task was chosen for two reasons: First this task has a comparably high-level of difficulty due to slippage and undesired rolling of the die. Second this task allows us to use a simple and intuitive success metric — the fraction of trails where the desired face ended up on top.

Tactile sensor

Hardware setup

Autonomous data collection

To train our deep predictive model, we need to collect training data of the robot interacting with its environment. We autonomously collected 7400 trajectories for the ball, around 3000 trajectories for the analog stick, and 4500 trajectories for the die experiment. Each trajectory consists of 15 to 18 time steps, depending on the experiment. These training trajectories were collected by applying random movements along each of the three axes. More details about the data collection process are provided in Appendix VII-A.

IV Deep Tactile Model-Predictive Control

The use of a high-resolution sensor such as the GelSight enables fine control, but also presents a major challenge: the high-dimensional observation space makes modeling and control substantially more difficult. To allow performing a variety of different manipulation tasks at test-time using a large dataset collected beforehand, we explore a model-based method for touch-based control. Our method builds on prior work on control via visual prediction . In this class of methods, a deep recurrent convolutional network is trained to predict future video frames conditioned on the most recent observations and a sequence of future actions. More information about the model are provided in Appendix 11. At test time, this model can be used to perform a variety of manipulation tasks by optimizing over the actions until the model produces the predictions that agree with the user’s goal, and then executing those actions on the robot. Prior work has applied this approach to vision-based object repositioning .

Goal specification

At test-time, the user specifies a goal by providing a goal tactile image: a reading from the GelSight sensor for the desired configuration, which we denote as $I_{g}$. While a number of methods could be used to specify goals, this approach is simple and general, and allows us to evaluate our method on a “tactile servoing” task.

Tactile MPC control

Once the predictive model has been trained, we may use it to plan to achieve any user-specified goal configuration $I_{g}$ in the tactile observation space. For this, we employ model-predictive control with the learned predictive model. We use an optimization-based planner to optimize over the action sequence at each time step to determine the actions for which the predicted outcome is closest to goal tactile image $I_{g}$, as illustrated in Figure 3. The planning problem is formulated as the minimization of a cost function $c_{t}(I_{g},\hat{I}_{t})$ which provides a distance metric between the predicted image $\hat{I}_{t}$ and the goal image $I_{g}$. In this work we set $c(\cdot,\cdot)$ to the mean squared error (MSE) in pixel-space between $I_{g}$ and $\hat{I}_{t}$, such that the optimization is given by

Implementation details

We use three CEM iterations for optimization, with 100 samples each. The prediction horizon for the video-prediction model is between 15 and 18 depending on the task. Each action is repeated three times, such that the plan consists of five or six actions. This planning horizon is usually sufficient to reach the goal from most configurations considered in our experiments. Using time-correlated actions helps to reduce the search space for the optimizer, thus reducing the number of required samples and increasing the control rate.

V Experimental Results

We now experimentally validate our deep tactile MPC approach on three real-world tactile manipulation tasks: moving a ball bearing, positioning an analog joystick, and rolling a die. For video results, see the project webpagehttps://sites.google.com/view/deeptactilempc.

Evaluating the performance of the tactile policy is challenging, since the target is provided directly in the space of tactile observations, and the ground truth pose of the object is unknown. However the advantage of using tactile images as goals is that this is highly general, since a user can easily specify any goal by manually positioning the sensor in the desired configuration.

For our evaluation, we use three different metrics that quantify different aspects of control performance: 1) mean squared error (MSE) of the difference between the goal-image $I_{g}$ and the tactile-image $I_{t}$ observed at the end of an episode, and 2) manually annotated distance in pixel-space between the pressure centroid of the object at the end of the trjectory and the location of the pressure centroid in the goal image. 3) For the die-rolling task we have a more intuitively meaningful success metric — the fraction of trials in which the die could be rolled so that it has the desired face on top.

While the MSE metric can be automatically evaluated and exactly captures the objective we optimize for in tactile MPC, mean squared errors in image space do not necessarily reflect actual distances between object positions and poses. It is for this reason that we use the additional manually annotated distance measure.

V-B Tactile Control Baseline

In order to provide a comparative baseline for our tactile MPC method, we designed an alternative method that uses hand-engineered image features to solve each of our three evaluation tasks. It accomplishes this by first detecting the pressure centre in the imprint of the ball, joy-stick or die and then moving in a straight line towards the target position. To estimate the coordinates of the pressure center in the current image and in the goal image $I_{g}$, it computes the weighted centroid of the pixels in the current image, where weights are squared pointwise differences between the current image and a blank “background” image from the sensor when it is not in contact with any object. This difference image outlines the contact region, and detecting its centroid roughly localizes the object of interest.

Having detected an approximate position of the object in both the current and goal image, the baseline commands an action along the vector from the current estimated contact position to its estimated position in the goal image. The step length is tuned to achieve the maximum control performance in terms of the estimated distance between the current and desired object position. Note that this is a fairly strong baseline for the ball bearing task, since localizing the centroid provides a good indication for the location of the spherical ball. For the joystick and die-rolling task, this baseline fails frequently, yet it is hard to design a better baseline. In contrast, deep tactile MPC is more general and does not require manual tuning or knowledge about specific object mechanics. The deep dynamics model used in tactile MPC learns a variety of basic properties about the world purely from data – such as that ball bearings remain in one piece and move opposite to the direction of movement of the sensor.

V-C Manipulating a Ball, an Analog Stick, and a 20-sided Die

We find that our method enables a robot to achieve all three manipulation tasks through only touch sensing, without other sensory feedback. For example, the robot is able to maneuver a ball, manipulate a die to new faces, control a joystick, all entirely by feel. For qualitative examples of these results, see Figures 5, 6 and 9, as well as the supplementary videohttps://sites.google.com/view/deeptactilempc.

For the ball repositioning task, on the left of Figure 7 we show a plot that illustrates the fraction out of 30 test trajectories that obtained distances between the pressure centroid at the final time-step and goal image which are lower than a certain threshold. The positions were hand-labeled, the distances are measured in terms of distances in the 64x48 tactile image. The right sight of Figure 7 shows the same graph for the mean-squared error between the final image and the goal-image. Note that for both metrics our method (in orange) consistently dominates the baseline approach (in blue) by a substantial margin.

The results for the analog stick repositioning task are shown in Figure 8, using the same metrics. Again, we see that our method (in orange) substantially outperforms the baseline.

As shown in Table I, our deep tactile MPC method achieves a significantly lower median distance than the baseline in both the ball-rolling and analog-stick task. In the die rolling experiments the difference between tactile MPC and the baseline is even larger. We conjecture that this is because of the fact that the dynamics in this die rolling task are too complex to be handled well by a simple hand-tuned controller, since it involves complex slipping, sliding and rolling motions on multiple surfaces. Figure 9 shows a qualitative example (more in supplementary video).

Based on these results we conclude that using sampling-based planning in a combination with a deep dynamics model is powerful method for solving a range of challenging manipulation tasks solely based on tactile information. We expect that the gap between hand-design methods and learning-based method will be even greater for more complex robotic manipulation scenarios such as multi-fingered manipulation.

VI Discussion and Future Work

Precise in-hand manipulation in humans heavily relies on tactile sensing. Integrating effective touch sensing into robotic control can enable substantially more dexterous robotic manipulation, even under visual occlusion and for small objects that are otherwise difficult to perceive. In this paper, we presented a touch-based control method based on learning forward predictive models for high-bandwidth GelSight touch sensors. Our method can enable a robotic finger to reposition objects and reach user-specified goals.

While our results indicate that deep convolutional recurrent models can effectively model future touch readings conditioned on a robot’s actions, our method still has a number of limitations. First, we explore short-horizon control, where the goal can be reached using only tens of time steps. While this is effective for simple servoing tasks, it becomes limiting when tasks require rearranging multiple objects or repeatedly executing more complex finger gaits. However, as video prediction models improve, we would expect our method to improve also, and to be able to accommodate more complex tasks.

Another limitation of our work is that, with a single finger, the range of manipulation behaviors that can be executed is limited to simple rearrangement. A more dexterous arm or a multi-fingered hand could perform more complex manipulation tasks. An exciting direction for future work would be to extend our results with multiple fingers equipped with touch sensors, with a joint predictive model that can predict the dynamics of object interaction with all of the fingers at once. Such setting could perform complex in-hand object repositioning, assembly, and other manipulation skills.

Acknowledgements

We would like to thank Chris Myers from the CITRIS Invention Lab at UC Berkeley for help with building the custom GelSight sensor and the 3 axis test-rig.

References

VII Appendix

When performing autonomous data collection we either need to reset the environment to a well-defined set of starting conditions after each trajectory or the set of reachable states in the environment needs to be confined. In the case of the ball-rolling task we use slightly curved surface so that upon completion of a trajectory the ball automatically rolls back to a location close to the middle of the arena. For the analog-stick task a reset mechanism was provided by the springs embedded into the analog stick. In the die rolling task we used a thread fastened to the die wound onto a motor that resets the die to an approximately fixed starting pose at the beginning of each trial. For each trial, the sensor makes contact with the surface of the die and moves it in an arbitrary direction resulting in different die faces. At the end of each trial, the die is reset as described above.

VII-B Deep Recurrent Visual Dynamics Model

The video prediction model is implemented as a deep recurrent neural network. Future images are generated by applying transformations to previous images. A schematic is shown in Figure 11. More details on the video-prediction architecture can be found in and . Note that depending on the experiment during the first 3 time-steps of unrolling the RNN prediction model we feed the most recent ground truth observations, we call these images context frames.