"The Web is BIG". A small piece of MIT's homepage from August 15, 1994.
Matthew Gray is the creator of "Wanderer, the first autonomous agent on the web, used to track web growth since 1993". The data is available online and shows that in June 1993 there were 130 web sites (only two on the .com domain), but most Internet activity was on FTP.
Based on this crawler, Matthew created Wandex, the first web search engine. "It was not very good, but again, the web was small. Within a year, numerous better web search engines appeared. But, I am proud to have had the opportunity to be first," says Matthew.
Now he works at Google. "In the years since, I started a web analytics company, earned two degrees, joined a hardware startup, and started a wireless location company. Now, I have come full circle. In February, I started work at Google in Boston. The first month has been great and it's a remarkable company."
The last he did was to map the world, by looking at the frequency of locations mentioned in the books from Google's index. "I wanted to show the Earth viewed from books, where individual mentions of locations in books combine to yield another interpretation of the globe. The intensity of each pixel is proportional to the number of times the location at a given set of coordinates is mentioned across all of the books in Google Books Search."
Oh yeah?
ReplyDeleteAnd WebCrawler's creator works at Technorati - aha!
"WebCrawler was the first full text search engine*. It went live on April 20, 1994 and was created by Brian Pinkerton** at the University of Washington."
ReplyDeleteFrom en.wikipedia.org/wiki/WebCrawler
* About search engines
** Brian Pinkerton's site
"WebCrawler was the first full text search engine"
ReplyDeleteWandex was the first web search engine, in Spring, 1993. It was, however, not a full text search engine.
There were previously existing non-web full text search engines at the time, but it may be correct that WebCrawler was the first full-text web search engine.
(I am the author of the Web Wanderer and the Wandex search engine.)
Matthew did a lot with the Web back in 1993 - and I recall his work on "sizing" the web. As a result I was fortunate to have him as part of the team that developed MIT's fishWrap personalized news system. Great stuff - definitely a web pioneer.
ReplyDeleteActually Aliweb was the first Search Engine for the Web.
ReplyDeleteCan anyone say a timeline for this.And is 'Wolfram Alpha' is the latest one?
ReplyDelete