Chapter 8: Contact Dynamics — Reading Touch as a Control Input

Overview

Part I explained tactile sensors and tactile data. Part II covered robot-hand hardware and human demonstration. The next question is operational: when a controller receives fingertip force, taxel maps, slip signals, and contact poses, how should the hand move? This chapter starts from contact dynamics and tactile feedback control.

The sequential multi-object grasping problem in the Cosmax meeting material is a useful anchor. The hand already holds one object. To grasp the next one, it must roll, shift, or support the first object while freeing fingers for a second grasp. The central question is therefore not simply whether a grasp succeeded, but whether the current contact set allows the next contact transition ^[6].

After reading this chapter, you should be able to... - Explain contact state, normal force, friction cones, and complementarity constraints as control variables. - Distinguish how tactile feedback enters impedance control, admittance control, and MPC. - Explain why model-based control and learned residuals are often combined. - Describe why contact dynamics comes before grasp planning in Cosmax-style multi-object handling.

8.1 Treating Contact as State

In robot-hand control, contact is not a binary event. It changes the state space of the hand-object system. Without contact, the object is a free rigid body. With contact, finger pose, contact normal, friction, object mass, contact compliance, and actuator limits jointly determine what motion is possible.

A tactile controller needs at least four layers of state:

Without these layers, the controller only observes joint positions. That reduces grasping to "the fingers closed, so the object is probably held." Tactile control asks a stricter question: where is the object supported, in which direction, and with what margin?

8.2 Complementarity and Contact-Implicit Planning

Contact-implicit trajectory optimization often uses complementarity. If the signed distance at a contact candidate is positive, contact force must be zero. If contact force is positive, the signed distance should be zero. In plain terms: if two bodies are apart, they cannot push each other; if they push, they must be touching.

The Cosmax 2026-05-12 material reviewed quasi-dynamic contact models and contact-implicit MPC as candidates for in-hand rearrangement ^[1]. The attraction is clear: instead of hand-designing a contact sequence, the optimizer can select contact creation and contact breaking over the whole trajectory.

The limits are equally clear:

• complementarity constraints make optimization hard;
• object mesh, mass, inertia, stiffness, and friction are required;
• global plans can take tens of seconds or minutes;
• real tactile force, object pose, and joint state arrive with noise and delay.

Thus model-based contact planning is rarely the whole solution. It is better viewed as a structural layer. Low-level impedance control maintains compliant contact, tactile feedback estimates the actual contact state, and residual learning compensates for modeling errors.

8.3 Impedance, Admittance, and Force Control

The low-level workhorse of tactile control is impedance or admittance control. The goal is not to track a rigid position target, but to maintain a desired relationship between force and displacement. When wiping a board, constant contact force matters more than zero position error. When inserting a bulb or tightening a cap, the system must provide enough axial force while avoiding collision and breakage.

UMI-FT illustrates this separation ^[5]. A high-level multimodal policy produces slow references from vision and F/T signals. Faster lower-level controllers regulate arm compliance and gripper force. The same separation applies to multi-finger hands: a high-level planner proposes object reference motion or grasp phases, while finger-level controllers maintain contacts with tactile force.

In this architecture tactile sensing is not a passive observation. It is the feedback channel that keeps contact stable.

8.4 Model-Based Control and RL: A Practical Division of Labor

The Cosmax 2026-06-05 material sharpened the distinction between model-based and RL-based in-hand manipulation ^[7]. Model-based methods need less data and are more interpretable, but require object geometry, physical parameters, and explicit object pose. RL handles abstraction and geometry variation more naturally, but depends on simulation and training resources.

For sequential multi-object grasping, the practical split is:

1. A model-based planner proposes a coarse object reference motion.
2. A tactile feedback controller maintains stable contact and slip margins.
3. Residual RL or a diffusion policy absorbs friction, occlusion, compliance, and finger-interference errors.
4. Failure logs return to simulation and real replay for policy updates.

This is not a choice between MPC and RL. The model gives structure; learning absorbs the residuals of reality.

8.5 Sequential Multi-Object Grasping as a Contact-Control Problem

The Cosmax task can be restated as a contact-control problem. One object is already in the hand. Some fingers must be freed for a second object. The first object must not be dropped. The controller must track:

• stability of the first object: force-closure margin, slip margin, palm/finger support;
• rearrangement feasibility: which finger can be released safely;
• second-grasp feasibility: which finger and approach direction are required;
• sensing uncertainty: how tactile contact compensates for visual occlusion.

This chapter gives the grammar for the rest of Part III. Chapter 9 focuses on slip and shear, Chapter 10 on multi-contact in-hand rearrangement, and Chapter 11 on force closure and tactile grasp planning.

Summary

The first step in tactile control is turning sensor output into contact state. Once contact state exists, force control, impedance control, MPC, and residual learning can operate on the same problem. In manufacturing tasks where the hand already holds an object and must grasp another, grasp planning without contact dynamics is not enough.

Manufacturing-Cell Checkpoint

Deploying contact dynamics requires turning contact state into a shared language for logs and controllers. Each attempt should record object id, grasp candidate, actual contact patch, normal and shear force, slip event, operator override, cycle time, and product-damage flags. A success/failure bit is not enough to tell whether the failure came from perception, contact acquisition, insufficient force closure, or execution-time slip.

The control stack should also be separated by time scale. Vision and task policy reason slowly about object identity and process order. Contact MPC updates finger references at an intermediate rate. Impedance and tactile reflex loops close quickly on force derivatives and slip. Without this split, the high-level policy misses fast contact events, while the low-level reflex has no understanding of the next object to grasp. Sequential multi-object grasping makes this separation unavoidable.

Operational Reading Note

The practical value of this chapter is not only the concept of contact-dynamics control; it is the set of engineering decisions that the concept changes. A deployable robot-hand project should start by asking what state becomes observable after this chapter is applied. The answer should be concrete: contact existence, contact patch, normal force, shear direction, slip margin, object pose, task phase, operator override, or product-damage risk. If a variable cannot be logged or consumed by a controller, it remains an explanatory idea rather than a system capability.

The second decision is the unit of evidence. Research demos often report one success metric, but tactile manipulation improves through failure records. A useful attempt record contains the object or SKU, the selected grasp candidate, the robot hand and sensor configuration, calibration version, task phase, tactile summary, policy action, safety intervention, and final outcome. This record is what connects the sensor chapters to the data chapter, the control chapters to the learning chapters, and the manufacturing chapters to QA.

The third decision is where the chapter sits in the control stack. Some ideas belong in fast reflex loops, some in contact MPC, some in policy inputs, and some only in offline diagnosis. Mixing these time scales creates brittle systems: a VLA cannot react to millisecond slip, and a low-level force controller cannot infer the next process step. The right architecture separates fast contact stabilization, mid-level grasp or rearrangement control, and slow task reasoning.

Finally, the chapter should be evaluated by the failure modes it removes. A method that improves benchmark success but leaves the team unable to distinguish perception failure, contact-acquisition failure, force-closure failure, execution-time slip, or maintenance drift is not yet production-ready. A method with slightly lower headline performance but better logs, safer force limits, and clearer recovery hooks may be the stronger basis for manufacturing Physical AI.

Chapter-Specific Implementation Framework

Turning contact dynamics into a working system begins with state definition. The concept should not remain an abstract performance claim; it should become a variable that a controller and a logger can both read. For this chapter, the relevant state may include contact state, contact patch, normal force, shear direction, object pose, task phase, safety margin, operator override, and product-damage risk. Each variable needs a coordinate frame, a timestamp convention, a calibration version, and an owner in the control stack. Without this discipline, a successful trial is hard to explain and a failed trial is almost impossible to diagnose.

The second step is time-scale separation. A fast loop at hundreds of hertz or 1 kHz should handle motor current, force derivatives, shear spikes, and slip reflexes. A mid-level loop at tens of hertz should update contact pose, grasp phase, and reference finger motion. A slower task loop should reason over object identity, SKU, fixture state, instruction, and the next grasp candidate. contact dynamics must be assigned to the right layer. A VLA cannot react to millisecond slip. A low-level force controller cannot infer the next process step. A robust architecture lets these layers communicate through compact state summaries rather than forcing every signal into one monolithic policy.

The third step is a record schema. A useful attempt record should contain attempt id, robot-hand model, sensor layout, calibration version, task phase, object or SKU id, selected grasp, measured contact patch, normal and shear force summary, slip event, policy output, safety intervention, operator note, and final outcome. In a manufacturing cell this record is also a QA trace. A research demo can be persuasive with a video, but a production experiment needs replayable evidence. For that reason, the result table for contact dynamics should include failure-type distribution, retry count, product-damage rate, cycle-time variance, and intervention frequency alongside success rate.

The fourth step is a small test protocol. Starting with every object and every hand motion makes failures uninterpretable. A better protocol begins with atomic tasks: contact acquisition, stable hold, controlled release, contact switch, recovery after slip, and force-limited correction. The next stage composes two or three atoms into sequential manipulation. Only after that should the system attempt a Cosmax-style first grasp, in-hand rearrangement, and second grasp sequence. This staged protocol reveals whether contact dynamics actually removes a failure mode or merely shifts the failure later in the trajectory.

The fifth step is to treat hardware and maintenance as experimental variables. The same algorithm can behave differently when gel surfaces wear, pads become contaminated, cable tension changes, a sensor is replaced, calibration drifts, backlash grows, or surface humidity changes. The log therefore needs software version, pad age, cleaning state, calibration time, replacement event, and fault code. These fields are not administrative details. They determine whether a performance drop comes from the learned policy, the contact model, the sensor, the hand mechanics, or the production environment.

The sixth step is failure-driven decision making. The team should ask which failure class improves after adding contact dynamics: perception before contact, contact acquisition, force-closure insufficiency, execution-time slip, collision, product damage, or operator override. If the answer is unclear, the method is not yet actionable. If the answer is clear, the next investment becomes much easier to choose. A contact-state problem suggests better sensing or calibration. A closure-margin problem suggests hand geometry or force control. A replay mismatch suggests simulation fidelity. A repeated intervention suggests task design, fixture design, or operator workflow.

Validation Protocol: From Demonstration to Repeatable Evidence

The method in this chapter should be validated as a repeatable experiment, not as a single successful demonstration. The first step is to lock the reset condition. Object pose, hand initialization, sensor calibration, pad condition, lighting, fixture state, and software version should be recorded before every trial. If those variables drift silently, the team cannot tell whether tactile feedback improved the behavior or whether the experiment simply became easier.

The second step is planned disturbance. Rotate the object slightly, vary surface friction, delay one fingertip contact, perturb the grasp candidate, or introduce a mild occlusion. A tactile method should degrade gracefully under these disturbances. More importantly, the log should show which signal was used for recovery: normal force, shear direction, slip event, contact patch migration, motor current, or a learned latent state. Without planned disturbance, the system may look robust while only succeeding in the narrow reset condition.

The third step is ablation. Compare no tactile input, normal force only, normal plus shear, slip-event tokens, and the full tactile summary. If performance improves only when the full high-dimensional stream is used, the method may be powerful but expensive. If a compact contact summary gives most of the gain, it may be the better manufacturing design because it is easier to log, debug, and transmit across control layers.

The fourth step is recovery-oriented metrics. A contact-rich system will still fail. The question is whether it notices earlier, recovers faster, retries safely, or leaves a clearer diagnosis. Useful metrics include time from slip onset to correction, force overshoot, contact reacquisition time, number of safe retries, intervention frequency, and product-damage near misses. These metrics often matter more than the final binary success rate.

The final step is deployment rehearsal. A researcher-adjusted experiment and an operator-run procedure are different systems. The operator should replace the sensor or pad, run calibration, start the task, stop after a fault, and export logs using the intended procedure. If performance collapses during this rehearsal, the bottleneck is not the policy alone; it is the integration and maintenance workflow. Passing this rehearsal is what turns a tactile manipulation method into a candidate for a manufacturing cell.

Control Design Pattern: Turning Tactile Signals into Actions

The four chapters in Part III return to the same operational question: when the hand receives tactile evidence, what should the fingers do next? A practical answer is a three-stage pattern. First, do not map the raw tactile signal directly to action. Convert it into contact belief: where the contact is, which direction force is applied, how much margin remains, whether slip is likely, and whether the next contact transition is feasible. Second, split that belief into a safety gate and a reference update. The safety gate prevents excessive force, slip, collision, and product-damage risk. The reference update changes target finger pose, target force, internal force, or release timing. Third, feed these results back to the higher-level policy so it can choose the next mode.

This pattern matters most in manufacturing multi-object tasks. Declaring that the first object is stable does not only mean that grip force is high enough. It means that the remaining contacts can support the object if one finger is released, that palm support is real rather than assumed, that the approach path for the second object is open, and that the slip margin is still acceptable. Tactile sensing updates this judgment continuously. Therefore the contact controller's output should not be a single command such as "grip harder." It should include a discrete mode such as hold, release, shift, roll, regrasp, or abort, plus finger-level references for position, force, or torque.

Experiments should evaluate the quality of these mode transitions. At each transition, check whether force spikes appear, whether the contact patch moves in the expected direction, whether shear remains inside the friction cone, whether object-pose estimates agree with tactile evidence, and whether the controller recovers without operator intervention. These logs reveal whether the problem is controller design, hand morphology, sensor placement, calibration, or the task fixture. Without transition-level evidence, a high success rate can hide a brittle policy that only works because the reset condition is narrow.

The same pattern also clarifies the relationship between model-based and learned control. A contact model can propose which mode transition should be possible. A learned residual can compensate for friction, compliance, and unmodeled geometry. Tactile feedback decides whether the proposed transition is actually happening. If the three disagree, the system should slow down, increase observation, or abort rather than blindly completing the motion. This is the control-level meaning of using touch as a first-class signal.

Operator Handoff and Safe-Stop Criteria

In a manufacturing cell, tactile control must end in an operator-understandable procedure. The system should expose when it continues, when it slows down, and when it stops. If slip margin drops, the controller may increase grip force within the allowed envelope. If the contact patch leaves the expected region, it may attempt a regrasp. If force limits or product-damage risk are exceeded, it should enter abort mode immediately. These conditions should not be hidden inside the policy. The operator interface and QA log should use the same names as the controller.

A useful handoff record contains three fields. The first is phase: acquire, hold, shift, release, regrasp, or abort. The second is stop reason: slip, over-force, lost contact, pose uncertainty, collision risk, hardware fault, or calibration drift. The third is next action: automatic retry, operator confirmation, sensor cleaning, recalibration, fixture reset, or object removal. This makes tactile control a shared operating procedure rather than a black-box behavior. It also gives the engineering team a direct path from field failures back to controller changes, sensor maintenance, or task redesign.

References

1. Pang, T., Suh, H. J., Yang, L., & Tedrake, R. (2023). Global planning for contact-rich manipulation via local smoothing of quasi-dynamic contact models. IEEE Transactions on Robotics. source [Pang et al., 2023]
2. Jiang, Z., et al. (2024). Contact-implicit model predictive control for dexterous in-hand manipulation: A long-horizon and robust approach. IROS 2024. source [Jiang et al., 2024]
3. Jiang, Z., et al. (2025). Robust model-based in-hand manipulation with integrated real-time motion-contact planning and tracking. arXiv:2505.04978. source [Jiang et al., 2025]
4. Chen, T. G., et al. (2025). DexForce: Force-informed action for contact-rich manipulation. arXiv preprint. source [Chen et al., 2025]
5. Choi, H., et al. (2026). UMI-FT: Force-torque-aware universal manipulation interface. Research manuscript and seminar material. source [Choi et al., 2026]
6. Cosmax Robotics Meeting. (2026a). Sequential multi-object grasping and active in-hand rearrangement problem statement. Internal meeting PDF, 2026-05-12. private source [Cosmax, 2026a]
7. Cosmax Robotics Meeting. (2026b). Model-based approach vs RL-based approach for in-hand manipulation. Internal meeting PDF, 2026-06-05. private source [Cosmax, 2026b]