In continuation with Nick’s very valuable info on ‘NY Times tags API’

Jacob harris highlights the importance of metadata in News industry. And they have been using it since 1851 phew!!  

On a different note the following excerpt (from this article) touches upon the ‘automation vs manual’ tradeoff discussed in today’s class. 

“Still my snarky aside has truth to it: people are ultimately controlling the process. In the beginning, rules for the automatic extraction and tagging are set by an Information Architect. In the end, final approval and correction of suggested metadata is done by various Web producers before publication. Web producers also do the important job of accurately summarizing the story. So, while we have machines to help out the process, it’s still ultimately a human endeavor, largely because automated summarization and classification has its problems.”

Comments off

Weinberger Need Statisticians

I’ve always wondered how Weinberger could get meaningful information out of his “huge pile”, and in his interview with Doctorow[1], Weinberger mentioned a way to make use of it: statistical analysis. This is what he said:

“Tags are chaos, and as you get more and more of them, it will get more and more chaotic.  It turns out that when you have a lot of them, the statistical analysis becomes really pretty precise.”

This reminds me of a paper I’ve previously read, “Toward Extracting Flickr Tag Semantics”, written by Yahoo! Research Berkeley and published on WWW2007[2]. The method described in the paper could identify “place tag” and “event tag” from the tags store in Flickr. For instance, the authors could “detect that the tag Bay Bridge describes a place, and that the tag WWW2007 is an event.” (WWW2007 is a conference held in Canada in 2007.)

How did they do that? The main idea is, “place tag” like Bay Bridge has significant spatial patterns, tending to concentrate within a certain geographic range, and “event tag” like a conference has significant temporal patterns, tending to appear around a certain time period. So by using preexisting spatial and temporal statistical methods, computer scientists are able to discover the “semantics” of Fickr tags.

In all, statistical analysis can help Weinberger make use of the huge amount of information, and it may also serve as a “filter” to deal with information overload problems.



[1] Metacrap and Flickr Tags: An Interview with Cory Doctorow,

[2] Towards Extracting Flickr Tag Semantics,

Comments off

Widespread video surveillance brings information management challenges

Video surveillance is increasingly common in America and other parts of the world. While cameras may be monitored in real time, they generally record video which can be reviewed later. This article covers two issues relating to video surveillance: privacy and retrieval.

The fact that cameras can monitor and record the actions of law-abiding people, in public, raises privacy concerns. To protect privacy IBM wants to blur the faces of ‘innocent bystanders’ [1].  The challenge is incorporating this into a product whose primary purpose is recording details of people suspected of wrongdoing. The two possibilies discussed in the article are 1) to automatically identify suspicious behavior and retain intact video only in those cases or 2) to store both the degraded and intact video and have separate access controls during retrieval.

The second part of the article discusses the information retrieval capabilities of current systems. These systems can automatically add metadata to the recorded video. During indexing, the system adds tags describing the colors and size of objects in the scene. Few details are provided about how queries are handled. The one example given is investigators, looking for a suspect wearing red, searching for the word “Red” to retrieve all video containing that color. 



[1] Google recently began blurring faces in Street View: 

Comments (3)

One Picture, 1,000 Tags

One Picture, 1000 tags (The New York Times, March 28, 2007)

If you try to find paintings on the museum’s web site, you will probably fail unless you know the title or artist. In order to increase accessibility, a dozen museums such as the Metropolitan Museum of Art are redesigning their online site by encouraging the public to annotate their collections with descriptive tags.

However, this tagging application could cause a huge semantic gap between the public and curators. For example, the Metropolitan Museum of Art ran a test in which volunteers supplied keywords for 30 images of paintings and sculptures. The tags were compared with the museum’s curatorial catalog, and more than 80 percent of the terms were not in the museum’s documentation. Nevertheless, ironically, since the art professionals can find it difficult to describe the visual elements of a picture and there is no taxonomic system, museums want the public to participate for a lot of tags of each image. Tags – from obvious to personal – can also be used to proclaim a personal connection with a work of art. These ‘collective intelligence’ projects might bring the collection alive. 

[Relevant lectures]


Comments (2)

Using Text Search Ideas to Speed Up Image Search

Microsoft Research news: Text-Search Tricks Speak Volumes in Image Search, May 2007

Finding similar images on Web used to require prohibitively high computational cost. Now however, researchers use text-search ideas to make content-based image search commercially feasible. For each image, a set of features are detected, and each feature is represented by a vector describing its characteristics such as orientation and intensity. In this way, each image resembles a document in text-search, and each vector resembles a token. Vocabulary is generated from millions of tokens gathered, and an inverted index could be built. Thus, the speed of finding a similar image on the Web falls to around 0.1 seconds.

A project call “Photo2search in Beijing” has turns this idea into reality. Geo-tagged street view images with longitude and latitude are crawled from photo-sharing websites like Flickr and put into an inverted index. Imagine you get lost in Beijing. Just pull out your camera phone, shoot a photo of your surroundings, send it to the system, and you get a digital map with your position marked on it by matching your photo to the most similar geo-tagged street view images.


Relevant lecture:  

23. VECTOR MODELS (11/17)

27. MULTIMEDIA IR (12/1)

Comments (3)

Pensieve, Delicious and “trails”

About a month ago, IBM published a press release about a project for personal memory organization called “Pensieve“.  There are a lot of similarities to MyLifeBits — its focus is on recording disparate types of information (business cards, photographs, timestamps, etc.) and then associating them together in the data store to ease retrieval.

That associative quality reminds me a lot of Vannevar Bush’s “trails”.  The reader wants to connect several documents together (or have it done automatically) so that they can be easily retrieved together later.  I can’t wait for this sort of technology to be commonplace (though I wonder if it will need to be done with a monolithic application like MyLifeBits or PENSIEVE rather than a series of integrated applications like Flickr, Delicious, GMail, etc.).

And just finding this link for this blog posts gives an example of why I’d like this “associative” information organizer.  Using delicious (a bookmark organizer that I’d heartily recommend to all of you), I wanted to connect the MyLifeBits link and the Pensieve link since there was such an explicit comparison there.  But delicious doesn’t provide functionality for explicit connections (Vannevar Bush’s trails are still lacking, even for something as simple as links in a single service).  Instead I’m forced to awkwardly create a unique tag (“cPensieve”) for the connection between them (that won’t recall lots of Harry Potter links as well).  So to see all the projects I’d like to compare to MyLifeBits (there’s another called Daytum that’s also worth looking at), you can go to this link:

(This should fit into the next lecture, or whenever we talk about Vannevar Bush and MyLifeBits.)

Comments off