Skip to main content

Facebook’s new image-recognition A.I. is trained on 1 billion Instagram photos

If Facebook has an unofficial slogan, an equivalent to Google’s “Don’t Be Evil” or Apple’s “Think Different,” it is “Move Fast and Break Things.” It means, at least in theory, that one should iterate to try news things and not be afraid of the possibility of failure. In 2021, however, with social media currently being blamed for a plethora of societal ills, the phrase should, perhaps, be modified to: “Move Fast and Fix Things.”

One of the many areas social media, not just Facebook, has been pilloried for is its spreading of certain images online. It’s a challenging problem by any stretch of the imagination: Some 4,000 photo uploads are made to Facebook every single second. That equates to 14.58 million images per hour, or 350 million photos each day. Handling this job manually would require every single Facebook employee to work 12-hour shifts, approving or vetoing an uploaded image every nine seconds.

facebook hacked
Digital Trends / Digital Trends

That’s not likely to happen any time soon. This is why the job of classifying images is handed over to artificial intelligence systems. A new piece of Facebook research, published today, describes a new, large-scale computer vision model called SEER (that’s “SElf-supERvised” in the hopelessly mangled backronym tradition that tech folks love to embrace). Trained on over 1 billion public images on Instagram, it can outperform the most cutting-edge self-monitoring image-recognition system, even when the images are of low quality and thereby difficult to read.

It’s a development that could, its creators claim, “[pave] the way for more flexible, precise, and adaptable computer vision models.” It may be used to better keep “harmful images or memes away from our platform.” It could be equally useful for automatically generating alt-text-describing images for visually impaired people, superior automatic categorization of items to be sold on Marketplace or Facebook Shops, and a multitude of other applications that require improved computer vision.

Welcome to the self-supervised revolution

“Using self-supervision, we can train on any random image,” Priya Goyal, a software engineer at Facebook AI Research (FAIR), where the company is carrying out plenty of innovative image-recognition research, told Digital Trends. “[That] means that, as the harmful content evolves, we can quickly train a new model on the evolving data and, as a result, respond faster to the situations.”

The self-supervision Goyal refers to is a brand of machine learning that requires less in the way of human input. Semisupervised learning is an approach to machine learning that sits somewhere between supervised and unsupervised learning. In supervised learning, training data is fully labeled. In unsupervised learning, there is no labeled training data. In semisupervised learning … well, you get the idea. It is, to machine learning, what keeping half an eye on your kid while they charge autonomously around a park is to parenting. Self-supervised learning has been used to transformative effects in the world of natural language processing for everything from machine translation to question answering. Now, it’s being applied to image recognition, too.

brain network on veins illustration
Chris DeGraw/Digital Trends, Getty Images

“Unsupervised learning is a very broad term that suggests that the learning uses no supervision at all,” Goyal said. “Self-supervised learning is a subset — or more specific case — of unsupervised learning, as self-supervision derives the supervisory signals automatically from the training data.”

What self-supervised learning means for Facebook is that its engineers can train models on random images, and do so quickly while achieving good performance on many tasks.

“Being able to train on any random internet image allows us to capture the visual diversity of the world,” said Goyal. “Supervised learning, on the other hand, requires data annotations, which limits the visual understanding of the world as the model is trained to learn only very limited visual annotated concepts. Also, creating annotated datasets limits the data amount that our systems can be trained on, hence supervised systems are likely to be more biased.”

What this means is A.I. systems that can better learn from whatever information they’re given, without having to rely on curated and labeled datasets that teach them how to recognize specific objects in a photo. In a world that moves as fast as the online one, that’s essential. It should mean smarter image recognition that acts more quickly.

Other possible applications

“We can use the self-supervised models to solve problems in domains which have very limited data or no metadata, like medical imaging,” Goyal said. “Being able to train high-quality, self-supervised models from just random, unlabeled, and uncurated images, we can train models on any internet image, and this allows us to capture diversity of visual content, and mitigate the biases otherwise introduced by data curation. Since we require no labels or data curation for training a self-supervised model, we can quickly create and deploy new models to solve problems.”

As with all of FAIR’s work, right now this is firmly in the research stages, rather than being technology that will roll out on your Facebook feed in the next couple of weeks. That means this won’t be immediately deployed to solve the problem of harmful images spreading online. At the same time, it means that conversations about the use of A.I. to further identify fine details in uploaded images are premature.

Like it or not, though, image-classifying A.I. tools are getting smarter. The big question is whether they’re used to break things further or start fixing them back up again.

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
How to change your language in Google Chrome on desktop
Chrome OS

Google Chrome supports a wide range of languages. While it'll default to English in most cases, there's nothing stopping you from changing its settings and displaying pages in Spanish, French, or dozens of other languages.

Changing your default language in Chrome takes only a few seconds, and the technique used is the same across Windows and Mac. Aside from changing your language, note that Chrome now gives you the option to automatically translate pages written in another language – making it easy to read content from around the globe.

Read more
23 of the best Netflix hacks, tips, and tricks
The Netflix home screen.

Netflix is one of the most popular streaming platforms for all things movies and TV shows. Home to an immense library of titles, the Netflix archive is constantly changing and evolving, and so are the many ways you can use your Netflix account. 

For instance, did you know you can access region-locked Netflix shows and flicks by using a VPN? Or that you can disable that pesky Autoplay feature? There are tons of Netflix hacks, tips, and tricks out there, so we’ve gone ahead and rounded up all of our favorites! 
Expand your streaming with a VPN

Read more
How to make a GIF from a YouTube video
woman sitting and using laptop

Sometimes, whether you're chatting with friends or posting on social media, words just aren't enough -- you need a GIF to fully convey your feelings. If there's a moment from a YouTube video that you want to snip into a GIF, the good news is that you don't need complex software to so it. There are now a bunch of ways to make a GIF from a YouTube video right in your browser.

If you want to use desktop software like Photoshop to make a GIF, then you'll need to download the YouTube video first before you can start making a GIF. However, if you don't want to go through that bother then there are several ways you can make a GIF right in your browser, without the need to download anything. That's ideal if you're working with a low-specced laptop or on a phone, as all the processing to make the GIF is done in the cloud rather than on your machine. With these options you can make quick and fun GIFs from YouTube videos in just a few minutes.
Use GIFs.com for great customization
Step 1: Find the YouTube video that you want to turn into a GIF (perhaps a NASA archive?) and copy its URL.

Read more