Overview
LingBot-Touch is an open-source tactile foundation model from Robbyant, Ant Group's embodied AI company, released in November 2026 together with a technical report, inference code and Tiny and Small weights under Apache 2.0.
It starts from a specific nuisance. Tactile signals are sparse and noisy, sitting on a near-zero background most of the time. Carry self-supervised methods over from vision unchanged and the model can drive the loss down simply by outputting zero, learning nothing about real contact. The team went the JEPA route, predicting the latent representation of masked signals instead of reconstructing them. On the same data the gap is stark: tactile-to-video top-5 recall at 1,000 hours reaches 32.1% for JEPA against 5.6% for MAE and 4.4% for VAE; at 4,000 hours JEPA climbs to 49.3% while MAE stays under 9% and VAE actually falls to 3.1%. Reconstruction fails here for a reason. Most of a tactile clip is near-zero background, so reconstruction loss keeps falling while contact features never form.
Sensor heterogeneity is the second obstacle. Magnetic, piezoresistive and visuotactile sensors work on different principles and produce very different signals. LingBot-Touch gives each type its own tokenizer, maps readings into one shared token space, and feeds a shared Transformer, with 5D RoPE encoding time, finger identity and 3D position together. During training the mask ratio warms up from 40% to 80%, because starting high would hide the only informative contact points in a sparse signal and send gradients exploding.
Results were checked on real hardware. Across 26 real robot tasks, four policies and 104 combinations, frozen encoder features scored highest in 89 of them, and adding touch to vision raised success rates by 14 to 26 percentage points. There is also a counterintuitive control: feeding raw tactile signals straight into a policy made results worse in 22 of the 104 groups, and swapping in pretrained features recovered most of those cases. Touch data needs a shared representation before it helps. On a magnetic dexterous hand never seen in pretraining, LingBot-VLA went from 57.21 to 75.04.
Key Features
- JEPA latent prediction: Predicts the latent representation of masked tactile signals instead of reconstructing them. Tactile-to-video top-5 recall reaches 49.3% at 4,000 hours of data, while same-size MAE stays under 9% and VAE at 3.1%.
- Unified encoding across three sensor types: Magnetic, piezoresistive and visuotactile readings each pass through their own tokenizer into one shared token space, so transfer does not depend on a single hardware vendor.
- 5D RoPE positional encoding: Encodes time, finger identity and 3D position together, aligning fingertip contact in a canonical frame so multi-finger and two-handed tasks share one representation.
- Masking warmup: The mask ratio rises from 40% to 80% over 300 steps, avoiding the early collapse that comes from hiding every informative contact point in a sparse signal.
- Frozen features plug in directly: Pretrained features drop into downstream force control and manipulation policies, transferring across sensors, dexterous hands, tasks and action models, including sensors never seen in training.
- Real-robot validation: Across 26 real tasks, four policies and 104 combinations, frozen features ranked highest in 89; adding touch to vision lifted success by 14 to 26 percentage points.
- Fully open: Apache 2.0, with the technical report, inference code and both Tiny and Small weights public and usable commercially.
Use Cases
- Dexterous hands on assembly, insertion and grasping tasks that require controlled contact force.
- Robotics teams with a working vision policy that want tactile feedback in the force-control loop.
- Cross-sensor transfer when switching tactile hardware without collecting and training from scratch.
- Embodied AI research and teaching, using pretrained features directly in downstream experiments.
Pros
- Latent prediction clearly works on sparse tactile signals: 49.3% recall at 4,000 hours against under 9% for reconstruction approaches.
- Three sensor types share one token space, so swapping hardware does not mean retraining.
- Frozen features are usable as-is, which keeps integration cheap with no full policy finetune.
- The comparison across 26 real tasks and 104 combinations is thorough, including the case where raw signals made things worse.
- Apache 2.0 with weights, code and the technical report released together.
Pricing
Free and open source under Apache 2.0. Code and both Tiny and Small weights are on Hugging Face and GitHub; commercial use needs no license, and local deployment compute is on you.
Summary
LingBot-Touch is Robbyant's open-source tactile foundation model, using JEPA to predict contact in latent space and sidestepping the failure of reconstruction on sparse signals. Three sensor types meet in a shared Transformer, frozen features drop into downstream policies, and touch lifted success by 14 to 26 points across 26 real tasks, with LingBot-VLA moving from 57.21 to 75.04 on an unseen hand. Apache 2.0, weights and code public.
Version History
- LingBot-Touch open-sourced with report and two weight sizes (2026-11-09): Robbyant, Ant Group's embodied AI company, open-sourced the LingBot-Touch tactile foundation model under Apache 2.0, publishing the technical report, inference code and Tiny and Small weights. Built on a JEPA architecture and pretrained on nearly 5,000 hours of real tactile data from data gloves and robot hands, it gives magnetic, piezoresistive and visuotactile sensors separate tokenizers into a shared token space and Transformer, with 5D RoPE positional encoding. Tactile-to-video top-5 recall is 32.1% at 1,000 hours (MAE 5.6%, VAE 4.4%) and 49.3% at 4,000 hours. Across 26 real tasks, four policies and 104 combinations, frozen features ranked best in 89, and touch added 14 to 26 percentage points over vision alone; on an unseen magnetic hand LingBot-VLA moved from 57.21 to 75.04.