Skip to main content

Image-recognition A.I. has a big weakness. This could be the solution

You’re probably familiar with deepfakes, the digitally altered “synthetic media” that’s capable of fooling people into seeing or hearing things that never actually happened. Adversarial examples are like deepfakes for image-recognition A.I. systems — and while they don’t look even slightly strange to us, they’re capable of befuddling the heck out of machines.

Several years ago, researchers at the Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Laboratory (CSAIL) found that they could fool even sophisticated image recognition algorithms into confusing objects simply by slightly altering their surface texture. These weren’t minor mix-ups, either.

Image Recognition Turtle Recognized as a Rifle
Image used with permission by copyright holder

In the researchers’ demonstration, they showed that it was possible to get a cutting-edge neural network to look at a 3D-printed turtle and see a rifle instead. Or to gaze upon a baseball and come away with the conclusion that it is an espresso. Were such visual agnosia to manifest in a human, it would be the kind of neurological case study that would find its way into a book like Oliver Sacks’ classic The Man Who Mistook His Wife for a Hat.

Adversarial examples represent a fascinating vulnerability when it comes to how visual A.I. systems view the world. But they also, as you might expect from a flaw that confuses a novelty toy turtle with a rifle, represent a potentially alarming one. It’s one that researchers have been desperately figuring out how to patch.

Now, another group of researchers from MIT have come come up with a new system that could help to dodge “adversarial” inputs. In the process, they have imagined a frankly terrifying use case for adversarial examples, one that could, if implemented by hackers, be used to deadly effect.

The scenario is this: Autonomous cars are getting better and better at perceiving the world around them. But what if, suddenly, the visual input-based onboard cameras in a car were either purposely or accidentally rendered unable to identify what was in front of them? Miscategorizing an object on the road — such as failing to correctly identify and place a pedestrian — could potentially end very, very badly indeed.

Fending off adversarial attacks

“Our group has been working at the interface of deep learning, robotics, and control theory for several years — including work on using deep RL [reinforcement learning] to train robots to navigate in a socially aware manner around pedestrians,” Michael Everett, a postdoctoral researcher in the ‎MIT Department of Aeronautics and Astronautics, told Digital Trends. “As we were thinking about how to bring those ideas onto bigger and faster vehicles, the safety and robustness questions became the biggest challenge. We saw a great opportunity to study this problem in deep learning from the perspective of robust control and robust optimization.”

Socially Aware Motion Planning with Deep Reinforcement Learning

Reinforcement learning is a trial-and-error-based approach to machine learning that, famously, has been used by researchers to get computers to learn to play video games without being explicitly taught how. The team’s new reinforcement learning and deep neural network-based algorithm is called CARRL, short for Certified Adversarial Robustness for Deep Reinforcement Learning. In essence, it’s a neural network with an added dose of skepticism when it comes to what it’s seeing.

In one demonstration of their work, which was supported by the Ford Motor Company, the researchers built a reinforcement learning algorithm able to play the classic Atari game Pong. But, unlike previous RL game players, in their version, they applied an adversarial attack that threw off the A.I. agent’s assessment of the game’s ball position, making it think that it was a few pixels lower than it actually was. Normally, this would put the A.I. player at a major disadvantage, causing it to lose repeatedly to the computer opponent. In this case, however, the RL agent thinks about all the places the ball could be, and then places the paddle someplace where it won’t miss regardless of the shift in position.

“This new category of robust deep learning algorithms will be essential to bring promising A.I. techniques into the real world.”

Of course, games are vastly more simplified than the real world, as Everett readily admits.

“The real world has much more uncertainty than video games, from imperfect sensors or adversarial attacks, which can be enough to trick deep learning systems to make dangerous decisions — [such as] spray-painting a dot on the road [which may cause a self-driving car] to swerve into another lane,” he explained. “Our work presents a deep RL algorithm that is certifiably robust to imperfect measurements. The key innovation is that, rather than blindly trusting its measurements, as is done today, our algorithm thinks through all possible measurements that could have been made, and makes a decision that considers the worst-case outcome.”

In another demonstration, they showed that the algorithm can, in a simulated driving context, avoid collisions even when its sensors are being attacked by an adversary that wants the agent to collide. “This new category of robust deep learning algorithms will be essential to bring promising A.I. techniques into the real world,” Everett said.

More work still to be done

It’s still early days for this work, and there’s more that needs to be done. There’s also the potential issue that this could, in some scenarios, cause the A.I. agent to behave too conservatively, thereby making it less efficient. Nonetheless, it’s a valuable piece of research that could have profound impacts going forward.

Image used with permission by copyright holder

“[There are other research projects] that focus on protecting against [certain types] of adversarial example, where the neural network’s job is to classify an image and it’s either right [or] wrong, and the story ends there,” Everett said, when asked about the classic turtle-versus-rifle problem. “Our work builds on some of those ideas, but is focused on reinforcement learning, where the agent has to take actions and gets some reward if it does well. So we are looking at a longer-term question of ‘If I say this is a turtle, what are the future implications of that decision?’ and that’s where our algorithm can really help. Our algorithm would think about the worst-case future implications of choosing either a turtle or a rifle, which could be an important step toward solving important security issues when A.I. agents’ decisions have a long-term effect.”

A paper describing the research is available to read on the electronic preprint repository arXiv.

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
Language supermodel: How GPT-3 is quietly ushering in the A.I. revolution
Profile of head on computer chip artificial intelligence.

OpenAI’s GPT-2 text-generating algorithm was once considered too dangerous to release. Then it got released -- and the world kept on turning.

In retrospect, the comparatively small GPT-2 language model (a puny 1.5 billion parameters) looks paltry next to its sequel, GPT-3, which boasts a massive 175 billion parameters, was trained on 45 TB of text data, and cost a reported $12 million (at least) to build.

Read more
Women with Byte: Vivienne Ming’s plan to solve ‘messy human problems’ with A.I.
women with byte vivienne ming adjusted

Building A.I. is one thing. Actually putting it to use and leveraging it for the betterment of humanity is entirely another. Vivienne Ming does both.

As a theoretical neuroscientist and A.I. expert, Ming is the founder of Socos Lab, an incubator that works to find solutions to what she calls “messy human problems” -- issues in fields like education, mental health, and other areas where problems don't always have clear-cut solutions.

Read more
Why teaching robots to play hide-and-seek could be the key to next-gen A.I.
AI2-Thor multi-agent

Artificial general intelligence, the idea of an intelligent A.I. agent that’s able to understand and learn any intellectual task that humans can do, has long been a component of science fiction. As A.I. gets smarter and smarter -- especially with breakthroughs in machine learning tools that are able to rewrite their code to learn from new experiences -- it’s increasingly widely a part of real artificial intelligence conversations as well.

But how do we measure AGI when it does arrive? Over the years, researchers have laid out a number of possibilities. The most famous remains the Turing Test, in which a human judge interacts, sight unseen, with both humans and a machine, and must try and guess which is which. Two others, Ben Goertzel’s Robot College Student Test and Nils J. Nilsson’s Employment Test, seek to practically test an A.I.’s abilities by seeing whether it could earn a college degree or carry out workplace jobs. Another, which I should personally love to discount, posits that intelligence may be measured by the successful ability to assemble Ikea-style flatpack furniture without problems.

Read more