INFO 202 Fall 08 Blog

How much of the World Wide Web is actually indexed… 27.65 billion pages? Maybe about 0.2% of the total content? The “Deep Web” (web documents not immediately accessible by direct hyperlink from public pages) may contain something like 91,000 terabytes of data… as compared to an estimated 167 terabytes of Surface Web data.

A new service, called Infovell, hopes to help users find more of this “Deep Web” data… yet unlike Google and other Surface Web engines, it won’t be ad-supported. Instead, the service will be subscription based. Read more at ReadWriteWeb blog. Here is an excerpt from the article:

The engine scours through open-access repositories of information like PubMed Central and the U.S. Patent and Trademark Office Claims, but it also allows access to scholarly journals such as those from Oxford University Press, SAGE, Taylor & Francis, Annual Reviews, Mary Ann Liebert Publications, and more. The culmination of these billions of pages currently unindexed by other engines, gives you access to content in the areas of Life Sciences, Medicines, Patents, Industry News, and other reference content from expert sources.

http://www.techcrunch.com/2008/07/29/how-to-lose-your-cuil-20-seconds-after-launch/

Stealth search start-up Cuil (pronounced “cool”) launched its product on July 29th of this year, and was promptly subject to an angry backlash from its users. The start-up – founded by three former senior Google employees – claims to have an index size of 120 billion web pages (larger than that of Google’s, they say). On the day of its launch however, Cuil’s search results were not as rewarding. For example, a search for “Dog” resulted in 280 million hits on Cuil and 498 million on Google. Of course, quantity isn’t everything, but even in relevance, Google’s results were better.

The comparison to Google is but natural, since Google defines what search means to most of us today – search results that are relevant, but also photos, news articles, video files, etc. that complete the picture. It remains to be seen whether Cuil can offer a superior ‘universal’ search package to one that we’re already used to.

Relevant lectures: 21/22