MoiréTac: A Dual-Mode Visuotactile Sensor for Multidimensional Perception Using Moiré Pattern Amplification

Kit-Wa Sou1*, Junhao Gong1*, Shoujie Li1†, Chuqiao Lyu1, Ziwu Song1, Shilong Mu2, Wenbo Ding1†
*Indicates Equal Contribution; †Indicates Corresponding Authors.
1Tsinghua University; 2Xspark Ai
Paper

Abstract

Visuotactile sensors typically employ sparse marker arrays that limit spatial resolution and lack clear analytical force-to-image relationships. To solve this problem, we present MoiréTac, a dual-mode sensor that generates dense interference patterns via overlapping micro-gratings within a transparent architecture. When two gratings overlap with misalignment, they create moiré patterns that amplify microscopic deformations. The design preserves optical clarity for vision tasks while producing continuous moiré fields for tactile sensing, enabling simultaneous 6-axis force/torque measurement, contact localization, and visual perception.

We combine physics-based features (brightness, phase gradient, orientation, and period) from moiré patterns with deep spatial features. These are mapped to 6-axis force/torque measurements, enabling interpretable regression through end-to-end learning. Experimental results demonstrate three capabilities: force/torque measurement with R²>0.98 across tested axes; sensitivity tuning through geometric parameters (threefold gain adjustment); and vision functionality for object classification despite moiré overlay. Finally, we integrate the sensor into a robotic arm for cap removal with coordinated force and torque control, validating its potential for dexterous manipulation.

Video Demonstration

Key Contributions

  1. Moiré interferometry for visuotactile sensing: We apply moiré pattern amplification to transform marker-based sensing into continuous interference fields, achieving R²>0.98 across six force/torque axes. Unlike sparse marker methods that interpolate between discrete points, our continuous fields directly encode mechanical deformations at every pixel.
  2. Physics-informed force/torque framework: We establish analytical mappings from four moiré observables (I, ∇φ, θ, Λ) to 6-axis wrench, enabling sensitivity adjustment through geometric parameters. This interpretable approach permits analytical sensitivity analysis and systematic calibration transfer, which are difficult to obtain with purely black-box models.
  3. Transparent dual-mode operation: We maintain optical transparency for object recognition while measuring forces, demonstrated in automated cap removal tasks. The preserved visual channel operates concurrently with tactile sensing, unlike opaque sensors that lose visual information upon contact.

Overview

Overview

Overview of MoiréTac. The dual-grating optical system generates moiré observables, which are mapped to force/torque measurements. The top-left panels show how press and rotation affect stripe density and orientation. The application demonstrates dual-mode visual and tactile sensing during robotic manipulation.

Structure Illustration

Structure

Structure illustration. (a) Mapping from Moiré observables (intensity I, phase ∇φ, angle θ, period Λ) to 6-axis force/torque sensing. (b) Photo of prototype. (c) Exploded view of layered architecture. (d) Cross-section showing compression-to-fringe coupling. (e) Responses under normal, shear, and twist loading; a waveguided LED produces a contact rim that delineates the boundary and maintains fringe visibility.

Sensor Fabrication

Fabrication

Fabrication process of MoiréTac sensor showing assembly steps. (a-c) Optical base assembly with LED ring positioning on baffle cap, lower reference grid installation, and acrylic cover bonding. (d-f) Elastomer layer preparation including silicone casting at 60°C for 3 hours, demolding, and thermal bonding to acrylic substrate. (g-h) Sensing layer completion with upper grid placement and final assembly. (i) Fully integrated sensor with camera.

System Workflow

Workflow

Overview of the MoiréTac processing pipeline. The system employs a GateER (Gated Energy Ratio) module for contact detection and automatic mode switching between vision and tactile sensing. In tactile mode, physics-based features (I, ∇φ, θ, Λ) are extracted and fused with ResNet34 deep features for 6-axis force/torque regression.

Six-Axis Force/Torque Calibration

Six-axis Calibration

Six-axis force/torque calibration. (a) Experimental setup with robotic arm, commercial F/T reference, and MoiréTac prototype. (b) Force calibration showing linearity for normal force Fz and shear forces Fx, Fy. (c) Torque calibration demonstrating Tz and Tx, Ty performance. Results show R²≥0.99 for forces and R²>0.98 for torques, with tight clustering along identity lines indicating low cross-talk between axes.

Dual-Mode Manipulation Demonstration

Demonstration

Hand-assisted cap removal demonstrating dual-mode operation. (a) Task sequence: (1) vision-guided approach to locate cap, (2) press to establish grip with force feedback, (3) rotate to unscrew with torque monitoring, (4) release after completed removal. (b) Corresponding sensor views showing vision-tactile mode transitions. Green box indicates visual target detection, while moiré patterns reveal force/torque during manipulation.

Citation

@article{sou2025moir,
  title={Moir\'eTac: A Dual-Mode Visuotactile Sensor for Multidimensional Perception Using Moir\'e Pattern Amplification},
  author={Sou, Kit-Wa and Gong, Junhao and Li, Shoujie and Lyu, Chuqiao and Song, Ziwu and Mu, Shilong and Ding, Wenbo},
  journal={arXiv preprint arXiv:2509.12714},
  year={2025}
}