Using Text Search Ideas to Speed Up Image Search
Microsoft Research news: Text-Search Tricks Speak Volumes in Image Search, May 2007
Finding similar images on Web used to require prohibitively high computational cost. Now however, researchers use text-search ideas to make content-based image search commercially feasible. For each image, a set of features are detected, and each feature is represented by a vector describing its characteristics such as orientation and intensity. In this way, each image resembles a document in text-search, and each vector resembles a token. Vocabulary is generated from millions of tokens gathered, and an inverted index could be built. Thus, the speed of finding a similar image on the Web falls to around 0.1 seconds.
A project call “Photo2search in Beijing” has turns this idea into reality. Geo-tagged street view images with longitude and latitude are crawled from photo-sharing websites like Flickr and put into an inverted index. Imagine you get lost in Beijing. Just pull out your camera phone, shoot a photo of your surroundings, send it to the system, and you get a digital map with your position marked on it by matching your photo to the most similar geo-tagged street view images.
Relevant lecture:
23. VECTOR MODELS (11/17)
27. MULTIMEDIA IR (12/1)
Longhao Wang Said,
September 1, 2008 @ 7:34 pm
This “Photo2search in Beijing” is built by a man sitting next to me in Microsoft Research’s Beijing office. This not published because search accuracy is not yet satisfactory. Questions like “what feature to extract from an image” and “how to build a vocabulary” greatly affect overall accuracy, and these questions are still open to further computer vision research.
Matt Gedigian Said,
September 2, 2008 @ 8:47 am
There is interesting work done at Berkeley on object recognition, by Professor Jitendra Malik.
http://www.eecs.berkeley.edu/Research/Projects/CS/vision/shape/
http://www.cs.berkeley.edu/~malik/cs294.html
Longhao Wang Said,
September 7, 2008 @ 10:49 pm
Thanks, Matt! The courses are really interesting.
Object recognition really is important here, I just don’t have enough space (100 words as Bob said on class) to delve into the details of this work.
Indeed, they use object recognition algorithms to extract features from images. Since the project focuses on matching street view images, special attention is paid to salient street view features, like slogans, banners and profile of the building. Here’s where “object recognition” come into play.