Researchers develop robot skin with both touch and vision

Researchers in Japan have developed a soft, vision-based sensor that enables both touch and proximity sensing in flexible robotic arms.

A research team led by Professor Van Anh Ho and Dr Quan Khanh Luu from Japan Advanced Institute of Science and Technology has developed ProTac, a novel vision-based soft sensing skin for robots. The study was published in the journal IEEE Transactions on Robotics, and presents a scalable approach to multimodal robotic perception.

ProTac is designed around a polymer-dispersed liquid crystal (PDLC) layer that can switch between transparent and opaque states when voltage is applied. When transparent, embedded cameras can ‘see’ through the skin to detect nearby objects. When opaque, those same cameras track how the skin deforms, allowing the robot to sense touch, pressure and the location of contact. This dual-mode sensing enables real-time environmental perception using a single soft layer, eliminating the need for complex embedded electronics.

“This innovation with sensor design simplicity allows robots to perceive both contact and nearby obstacles across a large-area skin in real time, which is difficult to achieve with conventional electronic skins,” Ho said.

To validate the design, the team developed a prototype called the ProTac link, a cylindrical robot arm segment wrapped in a soft sensing skin and equipped with stereo cameras at both ends. This prototype can detect approaching objects from multiple angles, estimate distance, and recognise multiple touch points with high accuracy. It also supports adaptive behaviours such as proximity-based speed adjustment and reflexive contact avoidance.

“ProTac can be applied to dexterous robotic manipulation in various domains where safety and delicate physical interaction are critical,” Luu said.

The researchers envision broad applications across collaborative robotics, assistive devices and soft robotic systems in agriculture or domestic care. To accelerate progress in the field, the researchers have made their design files, models and software open source.

Image credit: iStock.com/gorodenkoff