Chances are that if you’ve used Twitter in recent months you’ll have come across at least one tweet containing an image that urges you to “open for a surprise“.
More often than not this is a trick to entice you to click on the tweet, though we’ll admit some of them are quite clever.
Like them or not, soon those tweets will be a thing of the past as Twitter has a new way to crop photos so that you see the most interesting part of the image before you open it.
This is accomplished with a neural network and something called salient regions.
What is a salient region? Good question.
Machine learning researchers at Twitter, Zehan Wang and Lucas Theis, explain in a blog post.
“A region having high saliency means that a person is likely to look at it when freely viewing the image. Academics have studied and measured saliency by using eye trackers, which record the pixels people fixated with their eyes. In general, people tend to pay more attention to faces, text, animals, but also other objects and regions of high contrast.”
Simply put then it’s a region in a photo that your eyes are mostly to focus on and it can be used to train neural networks to predict how best to crop an image.
Pretty cool right? The trouble is that the neural network used to predict this saliency is slow.
To counter this Wang and Theis trained a smaller neural network to do the job that the larger network does just in a rougher fashion. So rather than having to map and analyse every single pixel of an image they cut out the features that they didn’t need.
The result was a neural network that was able to crop images ten times faster while still showing you what you might want to see in the cropped image.
The researchers say this feature is currently being rolled out to the Desktop website as well as Android and iOS Twitter apps.