Early Search Fun

I’m at TechStars for a Day watching my partner Ryan McIntyre give a presentation on Excite where he was a founder.  Each time he gives this presentation, it reminds me of the very first web spider – at least that I’m aware of.  Can you name it?

It’s the World Wide Web Wanderer (also known as W4 – which created the Wandex index.)  The creator was Matthew Gray which was one of the co-founders of NetGenesis – my first angel investment.  Matthew was one of the gang (including my close friend Raj Bhargava – now the CEO of StillSecure) who showed me the web (and Freshman Fishwrap) for the first time at an Athena Cluster in the Student Center at MIT in 1994. 

Ah – nostalgia.  Matthew has some great old stats up on his old MIT page, including a summary of web growth.  According to Wandex, on 6/93 there were 130 web sites.  130.  A year later on 6/94 there were 2,738.  By 6/95 the web was up to 23,500 and by 6/96 it was 230,000.  Oh the way we were.

It’s amazing what has occurred in the past 15 years.

  • tom

    yadda yadda

  • tom

    yadda yadda

  • Jerry Colonna

    Lycos. The first web spider. Named by Michael ” Fuzzy” Mauldin it's inventor for the Lyocsa genus of Wolf Spiders. I was there at the beginning.

    • I thought Lycos didn't appear until early 1994. W4's first release was mid 1993. I believe the first W4 crawl found less than 100 web sites. What was Lycos' first crawl? Of course, I'm now digging into my personal wayback machine .

  • Jerry Colonna

    Lycos. The first web spider. Named by Michael " Fuzzy" Mauldin it's inventor for the Lyocsa genus of Wolf Spiders. I was there at the beginning.

  • I thought Lycos didn't appear until early 1994. W4's first release was mid 1993. I believe the first W4 crawl found less than 100 web sites. What was Lycos' first crawl? Of course, I'm now digging into my personal wayback machine <g>.

  • Brad/Jerry, I have to differ. The World Wide Web Worm was invented in 1993 by Oliver McBryan at the University of Colorado. Yes, the search engine crawler was invented in Colorado, not that second rate school in MA.

    http://en.wikipedia.org/wiki/World-Wide_Web_Worm

    http://www.cs.colorado.edu/people/oliver_mcbryan….

  • Well – Wikipedia (the source of all truth in the universe) claims a 9/93 date for Ben / the World Wide Web Worm. W4 was created in the spring of 1993 and the first crawl was 6/93. http://www.mit.edu/people/mkgray/growth/.

  • Damn a scant three months. But, the WWWW (CU version) was sold in 1999 to idealab to form GoTo.com the originator of the paid text search ad. Gotta give Colorado some props for that… 😉

  • here is a cool link from the wayback machine: http://web.archive.org/web/19980626032310/www.got