Thanks to UC Davis and UC Berkeley scientists, visual search isn’t limited to Android users any longer. With the help of crowdsourcing, computer vision and ‘a bit of magic,’ iPhone application oMoby is the newest addition to visual search.
oMoby is used to recognize and label photos through crowdsourcing and computer vision. Businesses like IQ Engines, the creators of oMoby, tap into public knowledge to draw information so the company doesn’t have to do all the work, similar to how Wikipedia works.
Computer vision, a technology also used in surveillance and topographical modeling, serves as the eyes of the application.
A photo is first processed by object-recognizing computer vision modules, which can process an image in seconds.
“If those [modules] are not successful, the photo then goes to a human crowdsourcing network,” said IQ Engine co-founder Gerry Pesavento in an e-mail interview. “In this way, 100 percent of photos submitted get tagged.”
Currently, oMoby is best at identifying books, CDs and DVDs, and can do so without scanning barcodes. Once a picture is taken, the application is used as a shopping comparison tool to pull up product information, reviews and prices.
While out at dinner, someone could estimate if it’s worth buying a bottle of wine at the restaurant’s price by snapping a picture, identifying the bottle and comparing prices with oMoby.
Developed for the iPhone, oMoby provides visual intelligence as a service since anyone can use IQ Engines’ image recognition engine, whereas Google provides them as a mobile application, Pesavento said, distinguishing the two.
David Warland [cq], a research professor from UC Davis, co-founded IQ Engines and helped set the framework for their computer vision system.
Software infrastructure developer Huy Nguyen and software engineer Spencer Mathews, both graduate students at UC Davis, worked on the application with a team of computer neuroscientists and researchers from UC Berkeley. The researchers apply their expertise in biological and computer vision to develop applications like oMoby.
IQ Engines has also developed software to monetize pictures by matching labels to relevant retailers and advertisers.
By using their software, you connect a picture of yourself drinking from a Coca-Cola bottle to an advertising marketplace. Once it reaches the marketplace, Coca-Cola could pay to turn their logo in the photo into a clickable link or advertisement.
“There are a lot of advantages and disadvantages to this type of technology,” said Jesse Drew, professor of technocultural studies. “Because it is electronic, it leaves it open for hacking and error. Reliance on this technology can weaken peoples’ ability to think for themselves.”
Objects with distinct patterns and logos are relatively easy for the application to identify, but applications like oMoby and Google Goggles have trouble identifying objects with abstract shapes or plain textures, like purses.
“The camera is the eye and we’re building the brain,” Pesavento said in an interview with the San Francisco Chronicle.
Initially, oMoby had difficulty recognizing products, identifying a picture frame from Target as a “wood thing,” wrote an oMoby reviewer on everythingiCafe.com.
Within an hour after first identifying the ‘wood thing’ however, the item was updated and identified as a picture frame.
GABRIELLE GROW can be reached at campus@theaggie.org.
