“A New Era for Image Annotation”..?

Searching for images on the Web has traditionally been more complicated than text search – for instance, a Google image search for “tiger” not only yields images of tigers, but also returns images of Tiger Woods, tiger sharks and many others that are ‘related’ to the text in the query string. This is because contemporary search engines look for images using any ‘text’ linked to images rather than the ‘content’ of the picture itself.  In an effort to improve the recall of image searches, folks from UC San Diego are working on a search engine that works differently – one that analyzes the image itself. “You might finally find all those unlabeled pictures of your kids playing soccer that are on your computer somewhere,” says Nuno Vasconcelos, a professor of electrical engineering at the UCSD Jacobs School of Engineering. They claim that their Supervised Multiclass Labeling System “may be folded into next-generation image search engines for the Internet; and in the shorter term, could be used to annotate and search commercial and private image collections.”

What is Supervised Multiclass Labeling System anyway?

Supervised refers to the fact that the users train the image labeling system to identify classes of objects, such as “tigers,” “mountains” and “blossoms,” by exposing the system to many different pictures of tigers, mountains and blossoms. The supervised approach allows the system to differentiate between similar visual concepts – such as polar bears and grizzly bears. In contrast, “unsupervised” approaches to the same technical challenges do not permit such fine-grained distinctions. “Multiclass” means that the training process can be repeated for many visual concepts. The same system can be trained to identify lions, tigers, trees, cars, rivers, mountains, sky or any concrete object. This is in contrast to systems that can answer just one question at a time, such as “Is there a horse in this picture?” (Abstract concepts like “happiness” are currently beyond the reach of the new system, however.) “Labeling” refers to the process of linking specific features within images directly to words that describe these features.

While the idea of searching images by their ‘content’ is indeed promising, there are some questions that still need to be answered. To what extent does the system’s efficiency depend on the sample of images used for training?  What is the impact of variations in the quality of photos on the algorithm’s performance? What big a role will these play in affecting the user’s supposedly improved search experience? Finally, do we foresee an extension of the algorithm to determine abstract concepts in the images too? Indeed, these are interesting areas to explore; nevertheless, the SML seems to be a significant step towards better image retrieval mechanisms.

Read more about the SML at http://www.jacobsschool.ucsd.edu/news/news_releases/release.sfe?id=650

Comments (1)