Shutterstock’s composition photo search is powered by AI

0

Shutterstock

Fresh off its AI-powered tool for countering watermark removal from photos, Shutterstock is using machine learning for something else. In this case, it’s launching a composition-aware search tool.

“This tool allows users to specify one or more keywords, or to search for copy space, and arrange them spatially on a canvas to reflect the specific layout of the image they are seeking,” a press release reads. “The patent pending tool uses a combination of machine vision, natural language processing and state of the art information retrieval techniques to find strong matches against complex spatially aware search criteria.”

So, dragging “pen” to the lower left corner of the search box, and “desk” to the upper right corner will come back with photos where the pen is in the lower left of the frame, and a desk is in the upper right. At least that’s how it’s supposed to work in theory. Plenty of the results had the pen all over the photo, and a desk was always in the background. Adding “mug” to the search and moving it around the space performed as it should’ve though.

Proper nouns don’t work so hot. Searching for “Beyonce” resulted in pictures of (mostly) Caucasian women, and even one of a lady wearing a whipped cream bikini. So, yeah, it still has a ways to go. But, with normal stuff it works pretty well. Google Photos, which also uses AI and computer vision to sort and search photos, runs into similar hiccups, so this isn’t unheard of.

As is the case with any type of machine learning, Shutterstock’s tool will only get better with time and use. The company can’t do anything about the former, but since the feature is free for anyone to mess around with, the latter shouldn’t be an obstacle.

Over the past nine years, Timothy’s covered everything from drag shows to heavy metal, and he even debunked a local ghost story before joining Engadget in 2013. He’s an A/V enthusiast who adores physical media, much to the chagrin of his available shelf space. Movies by David Fincher and music from Amon Tobin, Deftones, Run the Jewels and Trent Reznor are his favorites. He has a complicated relationship with photography too and shares an exact birth date with Katy Perry.

Share.