INFO 202 Fall 08 Blog

Sunny’s blog posting reminded me of the effort for ‘Image-based’ search by using mobile phone – Vodafone and Nokia. Internet service companies are not the only ones who try to find out an effective and accurate way for ‘Image’ search. In order to provide better service to their customers and to invent their killer application, mobile telecommunication incumbents are also striving for this area.

• Vodafone‘ image-based mobile search ‘Otello’
  : At CeBIT 2008, Vodafone announced that they’ll be trialling their Otello image-based search technology in Germany. In other words, handset owners just snap a picture of anything — a landmark, DVD case, unidentified flying object, etc. — and Otello then “returns information relevant to the picture to the mobile phone.

• Nokia was also planning a semantic visual search engine, which makes plenty of sense given their push in high-quality cameras for their mobile phones.
  : The visual search engine uses three process levels to extract semantic information from an image.1) When analyzing the images, at first they are converted into a plurality of candidate low-level
  features (like shape, color and texture strength) and these features are extracted locally around salient points of the image.

  2) Then a supervised learning approach is used to select prominent low-level features from the plurality of candidate low-level features. The prominent low-level features are associated with predefined object categories, that describe generic objects (e.g., cars, planes, etc.); parts of a person’s body (e.g., faces), geographical landmarks (e.g., mountains, trees. etc.), or other items.

• 3) When a new item is to be categorized, the target item is converted into a plurality of multi-scale local features and then each local feature is matched with the prominent low-level features using a probabilistic model. So, if the target item has a face, then this feature will be matched accordingly to the other items having a face and the item will be categorized.

Besides these two players, even though I couldn’t find a success story so far, numerous mobile-related companies (including KDDI, iPhone) are working on this huge project continuously. However, this technology could one day be used to search for more information on famous places, DVDs, toys and so on by simply taking a picture with your handset while walking on the street.