Back to Sensing & Dexterous Manipulation

Sensing & Dexterous Manipulation

Fifteen Years of GelSight: A Brief History of Vision-Based Tactile Sensors

From surface metrology in 2009 to robot fingertips, soft hands and commercial sensors: GelSight’s research history and the opportunities and challenges of camera-based touch.

Zha Dami2024-03-1519 min readWeChat original
Contents

Recently, I have often encountered discussions of tactile perception in robotics, especially the initially puzzling term “vision-based tactile sensor,” or VBTS. I keep asking: is touch essential for general-purpose manipulation robots? Without touch, could vision alone support a sufficiently broad range of manipulation tasks?

One traditional framework describes six dimensions of human perception: eyes, ears, nose, tongue, body and mind, corresponding to sight, hearing, smell, taste, touch and consciousness. If robots ultimately run on electricity rather than food, taste and smell might seem of little use to robots. What progress has robotics made on the remaining four? [Editor’s note: This is a philosophical framing, not an exhaustive scientific taxonomy. Smell and chemical sensing can also be useful to robots, regardless of their energy source.]

Vision mainly uses cameras, often combined with LiDAR and other sensors; autonomous driving and service robotics have advanced visual algorithms considerably. Hearing uses microphones and related components familiar from phones and smart speakers, with increasingly mature speech recognition. ChatGPT led some people to speculate that AI might soon become self-aware, though its appearance does not establish consciousness. Touch, by comparison, remains very immature in both hardware and algorithms, in my view at this article’s March 2024 publication.

Many papers show that tactile perception can improve manipulation accuracy, yet I do not think a perfect tactile solution, or even a mature hardware platform, has emerged. Conversations with manipulation researchers often mention VBTS, particularly GelSight. This article follows GelSight’s academic development, highlighting milestones from its introduction onward. The many videos aim to show intuitively what these sensors are and what they can do.

Original article outline

Contents

👇

An introduction to robot tactile sensing

An introduction to GelSight

GelSight’s research timeline

Commercializing GelSight

Other vision-based tactile sensors

Opportunities and challenges for VBTS

1. An introduction to robot tactile sensing

Tactile sensors imitate aspects of skin, detecting stimuli from the physical environment. Common approaches include piezoresistive, piezoelectric, optical, capacitive, electromagnetic and camera-based tactile sensing. GelSight is a representative vision-based tactile sensor: it uses visual images to measure touch.

When humans manipulate objects, tactile information concerns both the object and the contact. Object information includes texture, shape and softness; contact information includes normal force perpendicular to the skin, shear force parallel to it, relative slip and object pose. Most piezoresistive, piezoelectric, capacitive and electromagnetic tactile sensors, in the comparison made in the original article, mainly sense normal force and cannot collect the other information or have low sensitivity to it. The greatest advantage of VBTS in that comparison is its ability to sense all the dimensions above with high sensitivity, bringing it very close to the range of tactile information available to human hands. [Editor’s note: This comparison is not universal: non-camera tactile sensors can also measure shear or other quantities, and recovering physical properties from VBTS images requires appropriate models and calibration.]

2. An introduction to GelSight

A University of Tokyo team proposed fluorescent-marker-based optical tactile sensing, GelForce, in 2004, although its resolution was lower. In 2009, Edward H. Adelson’s group at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) introduced GelSight, described in the original as the first ultra-high-resolution vision-based tactile sensor. Its name combines “gel,” the compliant contact material, and “sight,” the camera-based sensing mechanism. When an object contacts it, LEDs illuminate the gel and an internal camera observes deformation. Computer-vision algorithms map that deformation to tactile information.

GelSight optical structure: object, elastomer, support plate, LEDs and camera
GelSight optical structure: object, elastomer, support plate, LEDs and camera.

GelSight sensor structure, from the 2017 GelSight paper.

Since 2009, successive researchers in Adelson’s group have spent more than a decade advancing this approach to the forefront of robotic touch. Work based on GelSight has appeared at MIT, Stanford, CMU, UC Berkeley, UIUC, ETH Zurich, Oxford, Tsinghua, Peking University, Shanghai Jiao Tong University, HKUST and the Chinese Academy of Sciences, as well as Meta and Toyota Research Institute. The original highlights the many Best Paper recognitions associated with this research at RSS, ICRA and IROS. Specific wins and finalist selections are distinguished in the timeline below. Let us revisit important moments in its first fifteen years, through March 2024.

3. GelSight’s research timeline

Introducing GelSight—MIT, 2009

GelSight’s story begins with its inventor, MIT professor Edward Adelson. He is a neuroscientist and the John and Dorothy Wilson Professor of Vision Science at MIT, and a member of the US National Academy of Sciences and American Academy of Arts and Sciences. His recent research focuses on artificial touch for robotics. Micah Johnson and Adelson introduced the sensing technique at CVPR 2009.

The initial work observed microscopic surface structure rather than robot touch. The 2009 sensor used a transparent elastomer—a gel-like material—with a coated outer surface to measure the texture and geometry of an object pressed against it. Viewed from behind, the contact surface appears with uniform reflectance, allowing its geometry to be reconstructed in 3D from illumination and shading.

GelSight, 2009. #RobotTouch #VisionBasedTactileSensors (watch in the original WeChat article).

Source: Micah Johnson et al., 2009.

GelSight—MIT, 2011

At SIGGRAPH 2011, Adelson’s team improved the system for microscopic surface geometry, demonstrating spatial resolution as fine as 2 micrometers (Micah Johnson et al., 2011). Geometry capture did not depend on whether an object was matte, shiny or transparent. Hardware could take various forms, including handheld devices for high-resolution measurements in the field. Compared with 2009, the work improved materials, lighting and reconstruction, and suggested applications in surface-defect detection. Its focus was still surface metrology rather than robot manipulation.

GelSight, 2011. #RobotTouch #VisionBasedTouch #GelSight (watch in the original WeChat article).

Source: Micah Johnson et al., 2011.

GelSight evolves—MIT, 2013

At CVPR 2013, Adelson’s group first connected GelSight to robotic tactile perception in the timeline presented here, building a database of 40 tactile texture classes including fabrics, wood and sandpaper (Rui Li and Edward Adelson, 2013). The system classified materials from that database, showing how the sensor could support robotic material recognition.

GelSight material images and reconstructed surface textures across forty classes
GelSight material images and reconstructed surface textures across forty classes.

Tactile images of 40 material classes collected with GelSight.

GelSight—MIT, 2014

At IROS 2014, the team demonstrated precise localization and manipulation of small parts using GelSight (Rui Li et al., 2014). Its compact fingertip sensor achieved approximately 10-micrometer-scale spatial resolution and supported USB insertion and removal, combining vision and touch for fine manipulation and closed-loop control. This helped establish camera-based tactile sensing for robot manipulation. The design shown below resembles the later GelSight Mini. [Editor’s note: The original calls this the world’s first such sensor; that absolute priority claim is not independently established here.]

Compact GelSight fingertip and camera, LEDs, light-guiding plate and elastomer layout
Compact GelSight fingertip and camera, LEDs, light-guiding plate and elastomer layout.

Fingertip GelSight Sensor: design and structure for robot fingertips. Source: Rui Li et al., 2014.

The 2009–2011 devices were large—even the smallest roughly soda-can-sized—and insufficiently real-time for manipulation. From 2011, the team adapted the technology for robotics through hardware and software improvements. The 2014 fingertip version combined compactness and high resolution. The evolving platform subsequently supported normal force, shear and slip measurements; the explicit marker-based shear-and-slip work appears below in 2015.

Baxter USB insertion and removal with fingertip GelSight, from the original article.

A Baxter robot inserts and removes a USB connector using fingertip GelSight sensors. Source: Rui Li et al., 2014.

Compared with many conventional piezoelectric, piezoresistive or magnetic sensors, GelSight offers high-resolution geometry, estimates of normal and shear forces, and softness estimation useful for compliant objects. Its optical mechanism avoids some sensitivities associated with magnetic sensing. With the same basic hardware and software, robots can adapt their grasps to objects with different hardness, size and shape—eggs, chips, strawberries and even leaves, as the video shows. Multidimensional contact information enables closed-loop control reminiscent of human manipulation, valuable for broader generalization. [Editor’s note: Optical tactile sensors are not completely immune to temperature, humidity, lighting or material aging; the original’s absolute immunity claim is too broad.]

Adaptive grasping with GelSight tactile feedback, from the original article.

GelSight—MIT, 2015

At ICRA 2015, the team introduced tracking markers into the fingertip system to measure shear, relative slip and torsional loading (Wenzhen Yuan et al., 2015). Measuring normal force, shear and slip together became a widely recognized advantage in robot touch. The demonstration below shows this work.

GelSight, 2015. #RobotTouch #VisionBasedTouch #GelSight (watch in the original WeChat article).

Source: Wenzhen Yuan et al., 2015.

GelSight’s development accelerates—MIT, 2017

The 2017 Sensors paper GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force (Wenzhen Yuan et al.) played a major role in advancing its robotics applications. It reviews sensing principles, sensor and optical design, algorithms for shape, force and slip, and the design and manufacture of different hardware versions. It systematically addresses hardware and software issues and shows how high-resolution geometry and contact-force information support material perception and recognition.

An IROS 2017 paper presented an improved sensor for geometry and slip measurement (Siyuan Dong et al.). The core designs in that paper and the Sensors review are illustrated below. Together with the 2014 fingertip design, they became reference prototypes for many research laboratories.

Improved GelSight sensor and robot-gripper mounting
Improved GelSight sensor and robot-gripper mounting.

Source: Siyuan Dong et al., 2017.

Across 2017–2018, the group published work on perceiving object and material properties through touch. Two hardness-estimation projects, cited in the source as Wenzhen Yuan et al. and Yuan with Adelson, used fingertip GelSight without requiring precisely controlled contact conditions or known object shapes. Deep convolutional and recurrent networks analyzed the data to estimate hardness across differently shaped objects. [Editor’s note: One cited project, Estimating Object Hardness with a GelSight Touch Sensor, was presented at IROS 2016; the original reference labels it 2017.]

GelSight hardness estimation, 2017. #RobotTouch #GelSight #VisionBasedTactileSensors (watch in the original WeChat article).

Source: Wenzhen Yuan et al., 2017.

At CVPR 2017, the group studied 118 fabric samples, recording RGB and depth images of hanging fabrics and high-resolution tactile images (Wenzhen Yuan et al.). CNNs trained across these three modalities related appearance to touch. Each input produced an embedding representing physical properties, regardless of modality. Comparing embeddings allowed the system to predict how a fabric would feel from an image, and vice versa.

GelSight: Look and Feel, 2017. #RobotTouch #GelSight #VisionBasedTouch (watch in the original WeChat article).

Source: Wenzhen Yuan et al., 2017.

Building on that work, the team developed a robot that actively perceived clothing properties (Wenzhen Yuan et al., 2019). Guided by an external Kinect sensor, it squeezed garments with GelSight and identified eleven attributes: physical properties such as thickness, fuzziness, softness and durability, and semantic attributes such as suitable season and preferred washing method. The dataset contained 153 garments, with CNNs analyzing tactile data.

GelSight grasping clothing. #RobotTouch #GelSight (watch in the original WeChat article). [Editor’s note: The original embedded-video label says 2017; the associated research cited here is the 2019 clothing-perception work.]

Source: Wenzhen Yuan et al., 2019.

GelSlim—MIT, 2018

In 2018, the researchers further miniaturized fingertip sensing, building on the 2014 design, to introduce GelSlim. The left side below shows the earlier GelSight structure; GelSlim rearranges the internal optics to make the finger thinner.

Optical layouts comparing GelSight and the thinner mirrored GelSlim design
Optical layouts comparing GelSight and the thinner mirrored GelSlim design.

Source: Siyuan Dong et al., 2018. The full author list appears in the references.

GelSight—MIT, RSS 2020 Best Paper Award Finalist

A paper on cable manipulation with tactile sensing became an RSS 2020 Best Paper Award finalist (Yu She et al.). GelSight estimates cable pose in the grasp and friction during sliding. Cable following is decomposed into two tactile controllers: one controls the grip and the other the cable’s pose.

GelSight cable following. #GelSight #RobotTouch (watch in the original WeChat article).

Source: Yu She et al., 2020.

GelSight—MIT, IROS 2020 Best Paper Award

At IROS 2020, the MIT team and collaborators from Shanghai Jiao Tong University won the Best Paper Award for SwingBot (Chen Wang et al.). The robot explores an object’s physical properties through in-hand tactile sensing, then performs dynamic swing-up manipulation. Using gravity or arm acceleration makes mass, center of mass and friction especially important.

SwingBot tactile exploration through tilting, shaking and swing-up manipulation
SwingBot tactile exploration through tilting, shaking and swing-up manipulation.

Source: Chen Wang et al., 2020.

GelSight Wedge—MIT, 2021

The team introduced GelSight Wedge in 2021 (Shaoxiong Wang et al.), combining a compact shape suited to robot fingers with high-resolution 3D reconstruction.

GelSight Wedge contacting a plate and tactile contact images
GelSight Wedge contacting a plate and tactile contact images.

Source: Shaoxiong Wang et al., 2021.

GelSlim 3.0—MIT, 2022

Building on GelSlim from 2018, Alberto Rodriguez’s group introduced GelSlim 3.0, sensing high-resolution shape, force and slip in a more compact finger. Optimizing the optical path from illumination to the camera enabled that integration.

GelSlim 3.0 compact finger and tactile image outputs
GelSlim 3.0 compact finger and tactile image outputs.

Source: GelSlim 3.0 paper.

GelSight Baby Fin Ray—MIT, 2023

In 2023, Ted Adelson’s team introduced GelSight Baby Fin Ray, integrating tactile sensing into compliant soft fingers. Earlier fingertip structures were rigid: contact deformed the gel rather than the finger itself. For a compliant Fin Ray finger, the challenge is measuring gel deformation while the finger also bends.

GelSight Baby Fin Ray compliant tactile finger demonstration, from the original article.

Source: GelSight Baby Fin Ray paper.

GelSight EndoFlex—MIT, 2023

Beyond grippers, the group developed a multi-fingered hand. GelSight EndoFlex has three soft fingers with high-resolution sensing along their length, built from soft outer shells and flexible endoskeletons. Two cameras per finger capture contact along the front and sides. Enveloping grasps collect rich tactile information in a single grasp.

GelSight EndoFlex soft three-fingered tactile hand demonstration, from the original article.

Source: GelSight EndoFlex paper.

GelSight Svelte—MIT, IROS 2023 Best Paper Award Finalist

In 2023, Jialiang Zhao and Edward Adelson introduced GelSight Svelte: a curved, human-finger-sized sensor using one camera for broad tactile coverage and proprioception. Curved mirrors provide the required shape and coverage. Bending and twisting loads deform the compliant finger structure, which the camera also observes. A CNN estimates bending and twisting torques from those images. [Editor’s note: The original says the paper won IROS 2023 Best Paper. The authors’ project page identifies it as a Best Paper Award finalist; that distinction is corrected here.]

GelSight Svelte: a finger-shaped vision-based tactile sensor. #RobotTouch (watch in the original WeChat article).

Source: Jialiang Zhao and Edward Adelson, 2023.

The researchers then introduced GelSight Svelte Hand, a three-fingered, two-degree-of-freedom tactile hand. Each finger has a camera covering roughly the full finger length. A semi-rigid endoskeleton beneath soft silicone provides both stiffness and compliance.

GelSight360—MIT, 2023

Where many earlier designs sensed primarily one side, GelSight360 provides fingertip-like omnidirectional tactile sensing. Introduced in 2023, it uses a novel crossed-LED illumination arrangement.

GelSight360 mounted in a gripper with omnidirectional tactile images
GelSight360 mounted in a gripper with omnidirectional tactile images.

Source: GelSight360 paper.

Adelson’s group also collaborated on robot manipulation beyond its sensor designs. Two representative projects involving Tao Chen at MIT appeared in our earlier article, Bringing Robot Hands Closer to Human Hands: A Review of Dexterous Manipulation Research by Chinese Researchers.

TactoFind addresses object retrieval without vision, prior knowledge of object shape or a fixed object—for example, taking something from a drawer while objects can move freely.

TactoFind tactile-only object retrieval, from the original article.

Source: TactoFind paper.

Visual Dexterity: reorienting objects within the hand. Editorial clarification: this related project shares collaborators with the tactile research discussed here, but its reorientation controller observes objects with an external depth camera; it is not a GelSight tactile-sensing controller.

Visual Dexterity depth-camera-based in-hand reorientation, included as a related collaboration in the original article.

Source: Visual Dexterity paper.

4. Commercializing GelSight

GelSight Mini—GelSight Inc.

Commercialization began early: Adelson co-founded GelSight Inc. in 2011. Its initial business centered on the original 2009 application, surface inspection—think of a handheld magnifier or microscope revealing fine surface condition. Robot tactile-sensor commercialization came more than a decade later. In 2022, it introduced GelSight Mini, priced at US$499 as reported in this March 2024 article. Its design follows the 2014 fingertip GelSight approach. That is a historical price, not a current purchasing quote.

GelSight Mini. #RobotTouch #GelSight (watch in the original WeChat article).

Source: GelSight Inc. website.

A robot fitted with GelSight Mini can distinguish water from oil in transparent bottles in the demonstrated experiment, through the measured interaction rather than directly seeing the liquid’s chemistry.

GelSight Mini distinguishes water and oil. #GelSight #RobotTouch (watch in the original WeChat article).

Source: GelSight Inc. website.

DIGIT—Meta and GelSight Inc.

GelSight Inc. and Facebook AI Research collaborated to commercialize DIGIT, an easy-to-build, reliable, low-cost, compact, high-resolution tactile sensor for in-hand manipulation. The source reports a US$300 hardware price and open-source design. [Editor’s note: It gives 2022 as the launch year; the research sensor and paper date to 2020, distinct from subsequent commercialization. The price is the original article’s historical figure.]

DIGIT compact tactile sensor
DIGIT compact tactile sensor.

Source: DIGIT website.

A hand equipped with DIGIT can control grasp force more effectively when picking up an egg, helping prevent breakage, as illustrated below.

Egg-grasping demonstration with and without tactile sensing
Egg-grasping demonstration with and without tactile sensing.

Source: DIGIT website.

5. Other vision-based tactile sensors

Since GelSight’s introduction in 2009 and its compact robotics adaptation in 2014, research groups worldwide have developed camera-based tactile sensors. Here are several representative examples.

OmniTact—UC Berkeley, 2020

To overcome flat contact surfaces and limited sensing areas, Sergey Levine’s UC Berkeley team developed OmniTact, a multidirectional high-resolution sensor. Multiple cameras provide coverage in several directions. Below, the left side shows the 2017 GelSight fingertip design and the right side OmniTact.

GelSight and OmniTact optical layouts and hardware
GelSight and OmniTact optical layouts and hardware.

Source: OmniTact paper.

9DTact—Tsinghua IIIS, 2023

Huazhe Xu’s team at Tsinghua’s Institute for Interdisciplinary Information Sciences developed 9DTact. It builds on DTact’s optical principles, optimizing accurate and efficient 3D shape reconstruction. Both hardware and software are open-source.

9DTact sensor design, compact size and design priorities
9DTact sensor design, compact size and design priorities.

Source: 9DTact paper.

Insight—Max Planck Institute, 2022

Researchers at Germany’s Max Planck Institute developed Insight, a thumb-sized 3D optical tactile sensor that continuously estimates directional force distributions across its conical surface. It uses one internal camera and an elastomer layer over a supporting frame.

Insight sensor construction, force training and estimation pipeline
Insight sensor construction, force training and estimation pipeline.

Source: Insight paper.

ViTacTip—University of Bristol, 2023

Nathan Lepora’s team at Bristol developed ViTacTip. Its embedded biomimetic pins amplify tactile movements, and the resulting images naturally combine visual and tactile information in one device. For comparisons, the team also developed ViTac without pins and TacTip without transparent skin. Lepora’s biomimetic tactile work has received many awards, but my mind is running out of steam right now; I will leave Bristol’s tactile-sensing ideas for another article!

ViTacTip combined visual and tactile sensing demonstration, from the original article.

Source: Haoran Li’s website.

6. Opportunities and challenges for VBTS

Touch provides distinctive information about the physical world and supports activity even without vision, as blind people demonstrate. GelSight-like sensing offers rich force-related information, texture and hardness estimates, and high spatial resolution, creating substantial application potential. Because it records images using computer-vision principles, tactile images and the robot’s external visual observations can train within a shared model, enabling genuinely integrated visual–tactile manipulation control.

Widespread use still faces challenges. A common hardware platform should be inexpensive, practical and offer simple interfaces. Tactile datasets need scale and consistency comparable to what ImageNet brought to vision; at this article’s publication, no similarly established general tactile-image benchmark existed in my assessment. After suitable datasets are available, tactile foundation models are also needed to improve general recognition of physical properties—an idea informed by Shuibai’s Zhihu discussion of GelSight. Gel durability also matters, because contact wears its delicate surface. Image processing requires computation. Finally, camera-based sensing coverage is limited; recent designs such as GelSight Svelte explicitly expand its area and reach.

GelSight’s essence is image processing, and its accelerating development is closely tied to deep learning. Much VBTS progress remains research-oriented, making commercialization urgent. Large models are accelerating robotics. Returning to the opening questions: is touch necessary, and if so, which technology?

My view is that general manipulation needs touch, but the technology depends on the setting. Near-term “general” manipulation will likely mean varied tasks within a particular environment, so sensing should fit its tasks and requirements. Over the longer term, I think tactile technology must also become unified. Among existing options, camera-based touch seems most promising to me, although an undiscovered approach could ultimately emerge. Beyond general manipulation, expanding existing robots’ applications and capabilities with touch is also exciting.

References

Retrographic sensing for the measurement of surface texture and shape

Johnson and Adelson, 2009

https://people.csail.mit.edu/kimo/publications/retrographic/

Paper: https://people.csail.mit.edu/kimo/publications/retrographic/sensor.pdf

Video: https://www.youtube.com/watch?v=aKoKVA4Vcu0

Microgeometry capture using an elastomeric sensor

Johnson et al. and Edward Adelson, 2011

https://people.csail.mit.edu/kimo/publications/microgeometry/

Paper: https://people.csail.mit.edu/kimo/publications/microgeometry/microgeometry.pdf

Video: https://www.youtube.com/watch?v=ste7l2OvVHs

Sensing and Recognizing Surface Textures Using a GelSight Sensor

Rui Li and Edward Adelson, 2013

Paper: https://dspace.mit.edu/handle/1721.1/86137

Rui Li, Platt, Wenzhen Yuan et al. and Edward Adelson, 2014

Paper https://dspace.mit.edu/bitstream/handle/1721.1/88136/GelSight_IROS%202014_final.pdf?sequence=1&isAllowed=y

Measurement of Shear and Slip with a GelSight Tactile Sensor

Wenzhen Yuan, Rui Li, Srinivasan and Edward Adelson, 2015

Paper: https://people.csail.mit.edu/yuan_wz/GelSight1/ICRA15_2740_FI.pdf

Video: https://www.youtube.com/watch?v=4DSilbo2B24

GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force (Sensors, 2017)

Wenzhen Yuan, Siyuan Dong and Edward Adelson, 2017

Paper:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5751610/pdf/sensors-17-02762.pdf

Improved GelSight Tactile Sensor for measuring geometry and slip (IROS 2017)

Siyuan Dong, Wenzhen Yuan and Edward Adelson, 2017

Paper: https://arxiv.org/pdf/1708.00922.pdf

Shape-independent Hardness Estimation

Wenzhen Yuan, Chenzhuo Zhu et al. and Edward Adelson, 2017

Video: https://people.csail.mit.edu/yuan_wz/hardness-estimation.htm

Paper: https://arxiv.org/pdf/1704.03955.pdf

Estimating Object Hardness with a GelSight Touch Sensor

Wenzhen Yuan, Mandayam A. Srinivasan and Edward H. Adelson, 2016. [The source gives 2017 and abbreviates the author list; the linked paper is IROS 2016.]

Video: https://people.csail.mit.edu/yuan_wz/hardness-estimation.htm

Paper: https://people.csail.mit.edu/yuan_wz/GelSight1/IROS16_1542_FI.pdf

Connecting Look and Feel

Wenzhen Yuan, Shaoxiong Wang, Siyuan Dong and Edward Adelson, 2017

Paper: https://dspace.mit.edu/handle/1721.1/137957

Video: https://people.csail.mit.edu/yuan_wz/fabric-perception.htm

Active Clothing Material Perception

Wenzhen Yuan, Yuchen Mo, Shaoxiong Wang and Edward Adelson, 2019

Paper: https://dspace.mit.edu/handle/1721.1/126688

Video: https://people.csail.mit.edu/yuan_wz/clothing-perception.htm

GelSlim

Elliott Donlon, Siyuan Dong, Melody Liu, Jianhua Li, Edward Adelson and Alberto Rodriguez, 2018

Paper: https://arxiv.org/pdf/1803.00628.pdf

Cable Manipulation with a Tactile-Reactive Gripper (RSS, 2020 Best Paper Award Finalist )

Yu She*, Shaoxiong Wang*, Siyuan Dong*, Neha Sunil, Alberto Rodriguez and Edward Adelson, 2020

Paper: https://arxiv.org/pdf/1910.02860.pdf

Video: https://www.youtube.com/watch?v=-xKeWdrmuBc

SwingBot: Learning Physical Features from In-Hand Tactile Exploration for Dynamic Swing-up Manipulation

Chen Wang*, Shaoxiong Wang*, et al. and Edward Adelson, 2020

Project Page: http://gelsight.csail.mit.edu/swingbot/

GelSight Wedge: Measuring High-Resolution 3D Contact Geometry with a Compact Robot Finger

Shaoxiong Wang*, Yu She*, et al. and Edward Adelson, 2021

Project Page: http://gelsight.csail.mit.edu/wedge/

GelSlim 3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Finger

Ian H. Taylor, Siyuan Dong , and Alberto Rodriguez 2022

Project Page: https://ianhtaylor.net/gelslim-30

GelSight Svelte: A Human Finger-shaped Single-camera Tactile Robot Finger with Large Sensing Coverage and Proprioceptive Sensing

Jialiang Zhao and Edward H. Adelson, IROS 2023 Best Paper Award Finalist.

Project Page: https://gelsight-svelte.alanz.info/

GelSight Svelte Hand: A Three-finger, Two-DoF, Tactile-rich, Low-cost Robot Hand for Dexterous Manipulation

Jialiang Zhao and Edward H. Adelson IROS - RoboTac 2023

Project Page: https://gelsight-svelte.alanz.info/

GelSight360: An Omnidirectional Camera-Based Tactile Sensor for Dexterous Robotic Manipulation

Megha H. Tippur and Edward H. Adelson, 2023

Paper: https://arxiv.org/pdf/2304.04268.pdf

TactoFind: A Tactile Only System for Object Retrieval

S Pai, Tao Chen et al. and E Adelson, A Gupta, P Agrawal, 2023

Project Page: https://taochenshh.github.io/projects/tactofind

Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes

Tao Chen et al. and Edward Adelson, Pulkit Agrawal, Science Robotics 2023

Project Page: https://taochenshh.github.io/projects/visual-dexterity

OmniTact: A Multi-Directional High-Resolution Touch Sensor

Akhil et al. and Chelsea Finn, Sergey Levine, 2020 ICRA

Project Page: https://bair.berkeley.edu/blog/2020/05/14/omnitact/

DIGIT

https://digit.ml/

https://pages.gelsight.com/digit-tactile-sensor

GelSight Mini—GelSight Inc., 2022

https://www.gelsight.com/products/

9DTact: A Compact Vision-Based Tactile Sensor for Accurate 3D Shape Reconstruction and Generalizable 6D Force Estimation

Changyi Lin et al., Huazhe Xu 2023

https://linchangyi1.github.io/9DTact

A soft thumb-sized vision-based sensor with accurate all-round force perception

Huanbo Sun et al. 2022

https://www.nature.com/articles/s42256-021-00439-3

GelSight Baby Fin Ray: A Compact, Compliant, Flexible Finger with High-Resolution Tactile Sensing

Sandra Q. Liu, Yuxiang Ma, Edward H. Adelson 2023

https://arxiv.org/abs/2303.14883

GelSight EndoFlex: A Soft Endoskeleton Hand with Continuous High-Resolution Tactile Sensing

Sandra Q. Liu, Leonardo Zamora Yañez, Edward H. Adelson, 2023

https://arxiv.org/abs/2303.17935

Shuibai on Zhihu: https://zhuanlan.zhihu.com/p/620573836

Smart Note · Sensing & Dexterous Manipulation