Shared Nothing Architecture

On the heals of TypePad’s 18 hour outage this week, there’s been (and will be) a lot of continued discussion about how to build scalable and reliable online / web-based applications.  This is not a new problem (I not so fondly remember major and systemic outages in large services such as eBay and Amazon in the late 1990’s) but it’s gotten new attention as some of the emerging applications have scaled up the point as to have an interesting numbers of regular users (e.g. – it sucks if their service goes down for more than 15 minutes).  For example, as far as I can tell, has been down for the last four hours (“ is down for emergency maintenance. we’ll be back as soon possible.”) and on 12/15/05 Bloglines acknowledged that “Bloglines performance has sucked eggs lately.”

Tim Wolters – an extremely capable CTO – has an introduction to how he is approaching this at Collective Intellect.  He’s taking a page from Google’s playbook and developing a web service based on a “shared nothing architecture.”  On Friday, I had two different discussions about scalable architectures (e.g. “we’re going to scale up between 10x and 100x on a meaningful base in 2006 – here’s what we are planning”) and both included elements of what Tim is describing.

I expect we’ll hear a lot more about this in 2006 as a small percentage of the flood of web apps created in 2005 become popular enough to have real scale issues. 

  • The downtime at first typepad and now reminds me of our days in the web hosting market. I used to hate it when the servers were down. The phones would light up like a christmas tree and my mailbox filled with hatemail. The only bright side was that the market was so crazy, even with all of the downtime, we would still wind up with net new customers at the end of the week. I suspect the same will be true in these cases as well. Scaling up to meet high demand is always a challenge.


  • I think its a bit suspicious (enter conspiracy theory background music) that within 3 days Typepad, Bloglines, and all went down for emergency maintenance.


  • There was a bit of a debate that went on last week between Jeremy Wright (of b5 Media) and the folks over at FeedLounge. The issue at hand was whether people should design scalable apps when they’re just starting out, or it scalability can be added in later.

    Scott Sanders from FeedLounge seemed to think that scalability is something you worry about later; early on, all developers should think about is features.

    I strongly disagree. In my post Of Course Scalability Matters, I make the argument that not designing your system to be scalable is essentially not planning on success–it’s better to plan for success (that is, design a scalable app) than to write something that will have to be completely rearchitected and rewritten if anyone ever starts using it!

  • Sounds like Feedlounge still haven’t learnt anything from their experiences in the past year or so.

  • Pingback: tree care inc()

  • Pingback: need cheap car insurance()