Man's Robotic Arm Works Faster With High-Tech Sense of Touch

MONDAY, May 24, 2021 (HealthDay News) -- The robot hand extends toward a small cube, guided by signals from electrodes implanted in the brain of partially paralyzed patient Nathan Copeland.

In surprisingly smooth fashion, Copeland's mind directs the robot hand to pick up the cube and move it to another part of the table.

The process works so well -- at speeds approaching those of average folks -- because for the first time, a patient guiding a robot arm can feel the object they are picking up.

Copeland, 34, is the first human to have electrodes implanted in the sensory cortex of his brain. The robot hand delivers tactile feedback to his brain, informing him whether he's got a good grip on whatever object he's trying to manipulate.

"I could see the hand was touching the object, but I also had that extra reassurance and confidence that I definitely had made contact and I was applying a certain amount of pressure," said Copeland, who lives in Dunbar, Pa. "I knew if I went to lift the object off the table, it wouldn't fall out of my grasp."

Provided this sensory feedback, Copeland dramatically slashed the amount of time it took him to use the robot hand to pick up and move objects in lab experiments, said Jennifer Collinger, an associate professor with the University of Pittsburgh's Department of Physical Medicine and Rehabilitation.

Guiding the arm by sight alone, Copeland completed these sort of tasks in about 20 seconds, on average, Collinger said.

"When he had sensory feedback, that was cut in half, to 10 seconds," Collinger said. "It wasn't a small improvement. It was quite dramatic. He was actually able to do this many times under 5 seconds, which is considered normal time for an able-bodied person."

Take a look at the improvements in this video of Copeland using the robotic arm with and without the new technology:

For years, researchers working on brain-computer interfaces have focused on planting electrodes in the motor cortex, the part of the brain that guides movement. They proved that signals from the motor cortex could be interpreted by a computer and used to guide objects like robot arms or cursors on a computer screen.

Now they're focusing on another important part of movement -- the information that the brain receives back from limbs in motion.

Any time an average person reaches for a mug of coffee, their brains are flooded with feedback data that allow them to pick it up properly. They feel when they make contact with the mug so they don't knock it over; they adjust their grip based on the weight of the mug.

"The sensory piece of what we do as humans to make that happen is really important," said David Putrino, director of rehabilitation innovation at the Icahn School of Medicine at Mount Sinai. "It's difficult to perform a task like that without having constant sensory feedback about where your hand is in space, if your hand is grasping the object too soft, too hard."

Six years ago, bioengineers implanted an array of electrodes in both the motor and sensory cortexes of Copeland's brain. A 2004 car accident at age 18 left Copeland with limited use of his arms.

The electrodes lead to two jacks protruding from the top of Copeland's skull. Computer cables screw into the jacks, which are otherwise covered with caps, and relay information back and forth from Copeland to a robot arm sitting at his side.

The electrodes placed in the motor cortex track about 200 channels of information heading out from the brain, Collinger said.

"We look at the way that neurons are firing as a population, and turn that into control signals for a robotic arm, so the participant can reach and grasp and orient the hand in the work space to pick up objects and move them around," Collinger explained.

Meanwhile, the sensory cortex electrodes are being fed information gathered by the robot hand regarding the object being handled.

"We can then turn that information into stimulation patterns that are played through electrodes that are implanted into the somatosensory area of the brain, the area that typically receives tactile input from your hand," Collinger said. "Here is really the first time that we put those two components together to see how sensory feedback might improve motor control in the context of a brain-computer interface."

"The sensations I feel really depend on which electrodes they stimulate," Copeland said. "They can range between pressure or tingles, warmth, sometimes a tapping, and then I usually feel them at the pads at the base of my fingers."

While groundbreaking, still a long way to go

It's not easy picking something up by sight alone, said Copeland, who spent years practicing that way with the robot arm before his sensory electrodes were added to the process.

"Before, when I didn't have stimulation, I would be able to see the robotic hand touching an object but sometimes I would go to lift it off the table and it would fall out or be in the hand awkwardly, so I would spent extra time making sure I actually had hold of something before I started to move it," Copeland said.

Experts said this research, reported May 20 in the journal Science, is an important advance in prosthetics technology.

"It has long been known that motor prosthetics need to take into account the external world in order to offer improved performance," said J. Luis Lujan, an associate professor of neurologic surgery at Mayo Clinic. "The work of this research team further demonstrates the importance of tactile and visual feedback in the control of movement, whether of an individual's own limbs or robotic devices."

However, the experts said this technology will not be available in the mainstream anytime soon.

"There's a lot of industry interest in this area that I hope will accelerate the process, but there's definitely still a lot of work to do," Collinger said.

For example, the technology needs to be less invasive and easier to connect to the machines it will guide, Collinger and Putrino said.

"This is a first step in a technology that's incredibly invasive. It requires open brain surgery. You're implanting electrodes that have a limited useful life span in the brain tissue," Putrino said. "It's a long way to go before this is being used mainstream in the home."

For his part, Copeland didn't expect much personal gain from his participation.

"There is a portable tablet I have here at the house now that can be used to do basic stuff like draw or play games, because I can control a cursor on the screen" using his interface, Copeland said. "When I joined the study, I knew it was never going to give me back function, and it was never going to be a forever thing for me.

"It's literally the beginning of research. I just hope that through me spending my time trying to push the science forward, someone down the road that has an injury like mine and some sort of condition that limits their function, that this technology can help them in the future," he said.

More information

The University of Pittsburgh has more about brain-computer interfaces.

SOURCES: Nathan Copeland, Dunbar, Pa.; Jennifer Collinger, PhD, associate professor, University of Pittsburgh, Department of Physical Medicine and Rehabilitation; David Putrino, PhD, director, rehabilitation innovation, Icahn School of Medicine at Mount Sinai, New York City; J. Luis Lujan, PhD, associate professor, neurologic surgery, Mayo Clinic, Rochester, Minn.; Science, May 20, 2021