Electrotactile Arrays for Texture and Pressure Feedback During Robotic Teleoperation

Electrotactile array for viewing images on your forehead.

Electrotactile arrays are a lesser-known form of human-machine interface (HMI) that apply electric current to skin-contacting surface electrodes to excite cutaneous nerves and give the illusion of texture, pressure, or pinpricks (depending on current strength and electrode resolution) all without mechanical vibration.  This technique has been around for many years for:  non-visual fighter pilot status displays, tongue interfaces, surgery guides, and for forehead-mounted camera displays for the blind.  Enough background... The exciting news is a recent product developed by Senseg and Toshiba Information Systems called "E-Sense" that successfully embeds an electrotactile display into a touchpad, LCD, or other curved surface (eg. all over a cellphone), thereby providing programmable high-resolution texture feedback to a user -- see the video embedded below.  I would wager that this feedback could greatly enhance haptic shared awareness in teleoperation / telemanipulation systems.

Update August 15th, 2010:  Mr. Kurt Kaczmarek, an expert in this area and co-developer of the BrainPort with Dr. Paul Bach-y-Rita, provides a correction:  The Senseg device is actually causing deflections of the skin through electrostatic forces, making it an electrostatic variant of vibrotactile (sometimes referred to as electrovibration) rather than being electrotactile -- see the comments for more extensive clarification.

 

 

The idea of using electrotactile feedback in robotics isn't entirely new.  Two papers (one and two) in the year 2000 explored the technology for user feedback during robotic manipulation.  However, I don't believe they closed the book on the subject; recent advances in electronic integration (eg. in the video), miniaturization, and form-factor (eg. textiles) could offer vastly superior implementations and capabilities, especially if coupled to a texture classification system on the robot.  Just remember:  If you happen to act on the idea I'm sharing, gratuitous collaboration and co-authorship is always welcome. ;-)

I have no intent to provide a lengthy diatribe about the "coolness" of electrotactile displays, nor their vibrotactile cousins (eg. cellphone "vibration" motors). So instead, here's a quote from HowStuffWorks:

What we're talking about here is electrotactile stimulation for sensory augmentation or substitution, an area of study that involves using encoded electric current to represent sensory information -- information that a person cannot receive through the traditional channel -- and applying that current to the skin, which sends the information to the brain. The brain then learns to interpret that sensory information as if it were being sent through the traditional channel for such data. In the 1960s and '70s, this process was the subject of ground-breaking research in sensory substitution at the Smith-Kettlewell Institute led by Paul Bach-y-Rita, MD, Professor of Orthopedics and Rehabilitation and Biomedical Engineering at the University of Wisconsin, Madison.

Here are a couple other good references and a number of application photos, each of which links to an associated publication:

Electrotactile Display for the Tongue  Electrotactile Display for the Tongue  Electrotactile Display for the Tongue

Surgical Electrotactile Tongue Interface Surgical Electrotactile Tongue Interface

Electrotactile Forehead Display for vision  Electrotactile Forehead Display for vision

 

 

It is only fair to point out that two of my acquaintances (Seungyon Claire Lee and Dr. Thad Starner) championed wearable, textile electrotactile displays: "Stop Burdening Your Eyes: a Wearable Electro-Tactile Display" and "Mobile Gesture Interaction Using Wearable Tactile Displays."  Some time ago I was an early user-study participant for their system and infrequently consulted them on the electronics; more pointedly, our conversations drove my curiosity to explore this neat technology.  Congrats on your recently-conferred PhD, Seungyon!

 

Comments

The Senseg device is electrostatic, not electrotactile.  See for example:

Kaczmarek, K.A., Nammi, K.K., Agarwal, A.K., Tyler, M.E., Haase, S.J., Beebe, D.J. (2006). Polarity effect in electro-vibration for tactile display.IEEE J. Transact. BME, 53:10, pp. 2047-54.

http://tcnl.med.wisc.edu/pubs/2006-Kaczmarek-Electrovibration-Polarity.pdf

Information on development of BrainPort devices available at:

http://tcnl.med.wisc.edu/contact_wicab.php

I had an email exchange with Kurt on this topic.  Here is my inquiry:

Thanks for the clarification.  I'm happy to see an expert weighing in, and I apologize for my error.  I was aware that some systems used electrostatics, but had assumed that these were just another form of electrotactile in which displacement currents were exciting the nerves.  According to your paper, it is actually the electrostatic forces (and hence vibrations / vibrotactile) that excite the nerves.  I'm just curious... are the physical effects (electrotactile vs. vibrotactile) similar or do they feel markedly different to a user?

I've written about electrostatic systems (specifically electroadhesives) in the past (http://www.hizook.com/blog/2009/08/06/electroadhesive-robot-climbers), and am quite familiar with the extreme forces that can be generated and sometimes obscene excitation voltages required.  Using the electrostatic forces to perform high resolution, low-amplitude vibrations is quite clever!  It seems the Brainport is using copper electrodes and kapton dielectrics (basically, flex-PCBs); do you know if Senseg's system is using ITO (Indium Tin Oxide) electrodes?  [I was under the impression that ITO had a relatively low breakdown voltage, but they seemed to be using clear electrodes...]

 

And his response (re-posted with permission):

BrainPort devices are electrotactile (not electrostatic) and pass current into the tongue to stimulate afferent nerves. The kapton is just the substrate for the printed circuit on which the electrodes (titanium, I think in some versions; otherwise gold plate) are inserted. I know little about Senseg's system. I contacted the company for details but they have not responded. I am surprised this technology has moved to a commercial product given the problems I had with even small amounts of moisture. Perhaps they solved these problems and consider the solution a trade secret. (I would.) They do have a US patent application filed and published but I dont' think it really reveals anything new.

Electrotactile generally feels like tingling and electrostatic feels like texture. Electrostatic (or as it has also been termed, electrovibration) requires the finger to move over the electrode, whether insulated or not. If you want to know more my paper references most of the relevant literature (there isn't much); four decades later the definitive work is still Robert Strong's 1969 MIT PhD dissertation. It is highly recommended and relatively easy reading. My work in this area is merely an addendum to this tome. GRimnes' electrovibration paper is excellent and Mallinkrodt's short paper laid the stage well for what followed.

I am not aware of any really good vibrotactile transducers. Most efforts involve re-purposing some extant transducer (solenoid, loudspeaker, vibrator) but the end result is usually less than satisfactory because the transducer does not match the mechanical impedance of the skin and hence has very low efficiency. A first-principles design is recommended, of which I do not have the requisite expertise. I have not seen this approach taken since a paper by Holmlund, 1970, "An electromagnetic tactile stimulator", J. Biomed. Sys. 1:3, pp 25-40. 

 

There you have it.  The Brainport device is electrotactile, while the Senseg device is an electrostatic variant of vibrotactile.  Thanks again for the clarifications, Kurt.

—Travis Deyle

I just learned of a new company named TeslaTouch that is also developing an electrostatic vibrotactile (electrovibration) display.  Based on the author affiliations in their technical paper, the company seems to have originated from Disney Research and Carnegie Mellon University.  Their technology overview page has a wealth of information:

TeslaTouch vibrotactile displays

 

TeslaTouch can be added to wide variety of touch platforms, from small handheld devices to large multitouch collaborative surfaces. Our current prototype electronics package is 35x4x40 mm in size, and requires as little as 8 Volts to generate tactile sensations. Power draw is small, allowing immediate integration into battery-powered mobile devices. Importantly, no electric charge passes through the user. Instead an electrostatic force physically attracts the finger (almost like a magnet) to the interactive surface. Additionally, the input signal is uniformly propagated across the touch surface; therefore, the resulting tactile sensation is spatially uniform.

The statement about "no electric charge passing through the user" is a bit disingenuous -- there may be no DC current, but there are certainly AC displacement currents.  Either way, the design looks amenable to miniaturization and probably even reproducible by a competent hardware engineer.   

I want one for research (tinkering) purposes.  It could be fun to augment haptic interfaces with this type of display, eg. for robotic surgery.  I wonder how much it would cost to procure one and when they'll be available...?

TeslaTouch vibrotactile displays  TeslaTouch vibrotactile displays

 

Their overview of electrostatic vibrotactile technology (electrovibration) is also interesting:

What is Electrovibration?

The effect of electrovibration was discovered in 1954 by accident. Mallinckrodt et al. reported that dragging a dry finger over a conductive surface covered with a thin insulating layer and excited with a 110 V signal, created a characteristic “rubbery” feeling. They explained this effect by suggesting that the insulating layer of dry outer skin formed the dielectric layer of a capacitor, in which conductive surfaces and fluids in the finger’s tissue are the two opposing plates. When alternating voltage is applied to the conductive surface, an intermittent attraction force develops between the finger and conductive surface. While this force is too weak to be perceived when the finger is static, it modulates friction between the surface and skin of the moving hand, creating the rubbery sensation. This effect was named “electrovibration”.

It is important to highlight the differences between electrocutaneous, electrostatic, and electrovibration tactile actuation. Electrocutaneous displays stimulate tactile receptors in human fingers with electric charge passing through the skin. In contrast, there is no passing charge in electrovibration: the charge in the finger is induced by a charge moving on a conductive surface. Furthermore, unlike electrocutaneous tactile feedback, where current is directly stimulating the nerve endings, stimulation with electrovibration is mechanical, created by a periodic electrostatic force deforming the skin of the sliding finger.

In the electrostatic approach, a user is manipulating an intermediate object, such as a piece of aluminum foil, over an electrode pattern. A periodic signal applied to this pattern creates weak electrostatic attraction between an object and an electrode, which is perceived as vibration when the object is moved by the user’s finger. The tactile sensation, therefore, is created indirectly: the vibration induced by electrostatic force on an object is transferred to the touching human finger. In case of electrovibration, no intermediate elements are required; the tactile sensation is created by directly actuating the fingers.

TeslaTouch vibrotactile displays

 

Compared to Conventional Vibrotactile Technologies

In general, adding tactile feedback to touch interfaces has been challenging. One direction, for small handheld devices, has been to mechanically vibrate the entire touch surface with piezoelectric actuators, voice coils and other actuators. With low frequency vibrations, a simple “click” sensation can be simulated. When ultrasonic frequencies are used, a sensation of variable friction between the finger and surface can be created.

A major challenge in using mechanical actuation with mobile touch surfaces is the difficulty of creating actuators that fit into mobile devices and produce sufficient force to displace the touch surface. Creating tactile interfaces for large touch screens such as interactive kiosks and desktop computers allows for larger actuators. Larger actuated surfaces, however, begin to behave as a flexible membrane instead of a rigid plate. Complex mechanical deformations occur when larger plates are actuated, making it difficult to predictably control tactile sensation or even provide enough power for actuation.

An alternative approach to actuation of the touch surface is to decouple the tactile and visual displays. In the case of mobile devices, tactile feedback can be provided by vibrating the backside of the device, stimulating the holding hand. Alternatively, it is possible to embed localized tactile actuators into the body of a mobile device or into tools used in conjunction with touch interfaces. This approach, however, breaks the metaphor of direct interaction, requires external devices and still does not solve the problem of developing tactile feedback for large surfaces.

 

—Travis Deyle

There was recently some cool work out of Duke University about physically changing the texture of plastics on demand using applied voltages.    The work made it to Slashdot:

Imagine a pair of rubber gloves whose surface texture could be altered on demand to provide more grip for climbing. Or maybe gloves with "fingerprints" that can be changed in the blink of an eye. They are just a couple of the many potential applications envisioned by researchers at Duke University for a process they have developed that allows the texture of plastics to be changed at will. By applying specific voltages, the researchers have been able to dynamically switch polymer surfaces among various patterns ranging from dots, segments, lines to circles

 

This could be an interesting (competing) way to achieve the same sort of tactile feedback.

—Travis Deyle

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <p>
  • HTML tags will be transformed to conform to HTML standards.

More information about formatting options