Pinterest Lens: What the Latest Updates Mean for Visual Search
I’ll level with you: I find visual search fascinating.
In an industry where baseless buzz is never in short supply, I believe visual search actually offers something substantial. You can even hear me talking about it on one of Canada’s biggest radio networks, believe it or not.
Where other “trends” like voice search are often about the same consumer intent states, just expressed in another way, visual search extends the user’s communicative range.
It’s for those “I know what I want, but I don’t know how to say it” or “I don’t know what I want, but I’ll know it when I see it” moments that we’ve all felt while shopping.
Pinterest is among the leaders in a crowded visual search technology field. Google, Amazon, Microsoft, Facebook, and many others are getting in on the action. You can read about them in my humbly-titled “Ultimate Guide to Visual Search”.
For now, we’ll stick with Pinterest. After their IPO last year, the visual search engine has continued to grow its user base and improve its technology at a steady pace. It announced some iterative changes to its core computer vision technology, Pinterest Lens, on Tuesday.
So, the aim of this article is to start with a broad look at what visual search means, then zone in on Pinterest and its new Lens updates, before ending with the unanswered questions that will ultimately dictate the direction of this intriguing field.
VISUAL SEARCH: FOR WHEN YOU DON’T HAVE THE KEYWORDS
Put succinctly, online visual search scans and analyzes images, interprets their content and the user’s intent, then displays relevant results in response to the image.
Rather than a text query like “blue jeans”, the user can take a picture of some jeans the the computer vision technology will take it from there.
More broadly, we all conduct visual searches every day.
Looking for someone at the airport or trying to find your keys would count as visual searches. Or, that time I couldn’t find my glasses in a hotel room and had to call reception for some visual search assistance. Without glasses, it’s hard to find one’s glasses, I found.
A picture of a pair of jeans gives away a lot more than just their color, however.
A visual search engine can pick up signals like the cut of the jeans, where and when the picture was taken, and the user’s past search behavior.
That opens the possibility of interpreting two areas that have thus far been beyond the reach of pure, linguistic analysis: the ethereal nature of “style” and a speaker’s intent.
In the former case, it is difficult to pinpoint what exactly distinguishes an outfit, for example. In the latter, people don’t always know what they want and, if they do, they may struggle to express it.
So, these are latent elements and they typically remain beneath the surface.
Visual search can close that gap between signifier and signified. Where I may do a poor job of describing the item I want, a picture makes a pretty clear case.
But how does a visual search engine understand images?
I put together the image below a little while ago, based on some posts on the fantastic Pinterest engineering blog.
Their visual search product, Pinterest Lens, works through two complex stages: query understanding and final output.
As we can see at the top, Lens takes the picture of my sneaker and identifies its most salient features. Pinterest has access to billions of images (or ‘Pins’), along with the boards its users create. That brings with it a lot of metadata that can help Pinterest understand not just what is in an image, but also how the images relate to each other.
The next stage is to select the right results in response to the stimulus. As the diagram shows, Pinterest conducts a visual search for similar styles, an object search for similar items, and an image search, then blends these three sources and displays the results. The ratio of results in the “blend” stage is dictated by the historical data on past searches for such a product.
For example, what do people typically respond well to when they search for a sneaker? Are they trying to find that exact product, or do they want outfit ideas that go well with the product?
I spoke to Pinterest about Lens last year and they used the example of searching for a pineapple. If I’m in the store and I search with a picture of a pineapple, I may want to know if I can get the same product for a cheaper price nearby. If I conduct the same search when I’m at home, I probably want recipe ideas that contain pineapple. It’s not a perfect science yet, but the potential use cases are plentiful.
Google Lens offers a similar experience (and name), but its data sources are very different to the ones at Pinterest’s disposal. Amazon visual search, for its part, excels with product searches, while Instagram’s shoppable images avail of sophisticated visual search technology too.
For technology companies and retailers alike, this is about more than bridging that linguistic gap between the signifying terms and the intended, signified items.
It is about closing the chasms between online and offline shopping, inspiration and purchase, consumers and companies.
At a time when brands need to convince consumers to share data through a value exchange, visual search is a compelling option. Tell us about what you like and we’ll find items for you
This doesn’t need to replace text-based search, but rather it can extend the search journey and allow the user to arrive at better answers.
It becomes possible to search using a general style, an item, or simply a pattern.
There is a growing sense that shoppers are warming to this option:
62% of millennials want visual search over any other new retail technology.
36% of consumers have conducted a visual search.
59% of shoppers think visual information is more important than textual information.
There are over 600 million visual searches on Pinterest every month.
That all makes instinctive sense; shopping is a highly visual pursuit, after all.
Businesses and advertisers go to where their audience hangs out, so it is not surprising that so many have made this a priority:
21% of advertisers believe visual search is the most important trend for their business in 2019.
35% of marketers plan to optimise for visual search through 2020.
45% of major retailers in the UK now offer a visual search option.
And yet, there is a sense that this technology has not reached its inflection point.
If it really is so revolutionary and it delivers a much better experience than any other way of shopping, why isn’t everyone using it? These statistics suggest that usage is growing, but we all know that such numbers are indicative, at best.
Pinterest has done more than any other company to bring visual search to the public consciousness, while Google and Amazon have watched on with interest.
“If you see it — online or off — you can shop it.”
Pinterest’s earnings have exceeded forecasts so far this year and, at a time when so many tech companies face increased scrutiny, it is quietly going about its business.
Monthly active user numbers continue to grow and so too does average revenue per user.
Pinterest updates its Lens product on a frequent basis.
That has helped to keep its visual search technology one step ahead of the competition, without ever really posing a serious threat to its rival social networks. Google’s image search looks increasingly like Pinterest’s results, which I suppose should be taken as a compliment.
Earlier this week, Pinterest announced another couple of Lens updates:
Shoppable Product Pins with retailer details and checkout links will appear during visual searches, in the home and fashion categories.
Users can save photos through Lens, turn them into Pins, and add them to boards. Up to now, the photos were used as a query, but couldn’t be used on Pinterest afterwards.
This build on other recent announcements, such as Complete the Look (I wrote about it here), which can take an image of an item of clothing or furniture, then suggest complementary purchases.
Such announcements bring Pinterest closer to delivering on the promise of true understanding of context.
If I have just moved into an apartment and I show Pinterest a picture of my new sofa in a sparsely-decorated living room, I don’t just want the search engine to identify the item for me. It is unlikely that I want to buy another, similar sofa so soon. Most visual search engines, including Amazon and Google, tend to stop at this point. Pinterest can recognise what’s in the image, but can also start to interpret what the user means.
In the example of my brand-new sofa, I may want ideas for how to decorate the rest of the room, which then opens the opportunity for brands to communicate with me at a moment of high purchase intent.
A furniture retailer would love the chance to develop a virtual showroom to cater for these moments, and Pinterest could provide the ideal platform.
Although such scenarios do not run quite so smoothly just yet, Pinterest’s latest announcements move us a small step closer. The idea of shoppable social media is most often related to Instagram, for good reason, but Pinterest can now offer a more effective visual search experience.
Pinterest is beginning to monetize this technology through advertising, which sounds pretty boring and unimaginative. However, the connections that Pinterest can facilitate allow for more interesting campaigns than the same old keyword bidding on Google Ads.
Advertising revenues are expected to pass the $1 billion mark in 2020 and brands can now target over 5,000 categories through visual search advertising on Pinterest Lens.
That said, fashion, furniture, and beauty are the most popular categories and companies like Wayfair have seen great results already.
Pinterest has also invested in features to make their technology more inclusive, like the skin tone selector that helps filter results.
Advertisers would certainly welcome a genuine alternative to Google and Facebook. Pinterest’s ad revenues are paltry in comparison to those two digital giants, but it is making steady improvements and has been judicious about how to grow.
Pinterest has developed its own TV and out-of-home campaigns to try and broaden its appeal. It does still have a reputation as a specialist social network for cutesy wedding planning and recipe ideas, rather than an alternative to Instagram for fashion shopping.
Its sales pitch, as a “place for inspiration”, is a compelling one and appeals through its distinction from the likes of Google.
Nonetheless, there are a number of unanswered questions that surround Pinterest’s potential:
Can Pinterest attract and retain a large, mainstream audience?
Can it appeal to advertisers for categories outside of home and fashion?
How can Pinterest provide the measurement that digital marketers demand? The “inspiration” angle is interesting, but not as measurable as a plan based on consumer actions.
Will Pinterest remain the market leader for visual search, or will Google, Facebook, and Amazon redouble their efforts when the market grows?
Can Pinterest integrate visual search with other technologies, like augmented reality, to develop a distinctive offering that others cannot copy so easily?
Will users get into the habit of using images to search?
I’ll certainly be keeping a close eye on developments, updating my “Ultimate Guide to Visual Search” along the way.
“The future of search will be about pictures rather than keywords.”
— Ben Silbermann, Pinterest