Comments on: The Constipation of Scale

By: Brad Feld

Brad Feld — Thu, 06 Sep 2007 12:59:34 +0000

@Jason – there is no easy answer. There are very few young companies that manage the tension between scale and functionality well. As companies get more mature, they tend to figure it out with an occasional major burp. As I said in my closing, “occasional constipation is a fact of life.”

By: Pete Warden

Pete Warden — Thu, 06 Sep 2007 03:28:04 +0000

The only way I’ve been able to manage this is to add features that aren’t dependent on the core architecture, whilst the underlying changes are going on in parallel.

Can you add more content to the site? Change the graphical look? Add more cosmetic doodads? Chances are that there’s some changes that are so cosmetic that from an engineering POV they don’t seem like features, but they sure seem like improvements to the user.

That buys you some time, and helps you avoid ‘going dark’ to your users, or all the other new stakeholders!

By: Adam Messinger

Adam Messinger — Thu, 06 Sep 2007 00:00:52 +0000

I’ve observed that part of the frustrations around this are that the scaling factors change when widespread customer adoption begins. Product development effort, which scales with the number of features, becomes dominated by customer related effort which scales with the number of users.

VCs should see this as a good sign because it means that the thing is actually getting used. The effect of it on development cycle can be ameliorated by dividing the engineering team into two groups, one focussed on customer care and the other on new product development. It is the context switching between the two costs which really eats into productivity.

By: Bob Warfield

Bob Warfield — Wed, 05 Sep 2007 23:16:58 +0000

There are two strategic technical problems that companies face when dealing with scale. The first is that all the chatter about a Multicore Crisis is true, but it is much worse. We don't have 10 years to figure out the right tools and techniques for multicore because most people have already encountered a Multicore Crisis and just didn't know it: http://smoothspan.wordpress.com/2007/08/30/youve-already-had-a-multicore-crisis-and-just-didnt-realize-it/ The second strategic technical problem is the tremendous waste that goes into most programming. At least 70% of the code that is written is wasted because it contributes no proprietary advantage. Instead, it is just reinvention of wheels. There are a lot of reasons for this, but if you can free up even a little of that 70% it allows much more focus on things like scalability and features: http://smoothspan.wordpress.com/2007/09/04/70-of-the-software-you-build-is-wasted-part-1-of-series-of-toolplatform-rants/

By: Jason

Jason — Wed, 05 Sep 2007 15:21:52 +0000

Do you think these companies should have worked on scalability earlier? Or should they just have been more upfront (with themselves perhaps) about the high cost of making their product/service scale after it became popular?

Obviously the situation is different for every startup, but it seems like a popular product that doesn’t scale yet is better than a high-performance product that nobody uses.

If you can have both, that’s great, but I’m not sure it’s even a good idea to spend that much time on scalability and performance until you absolutely have to. Any additional features or design refinements that are developed early in exchange for deferring scalability could make or break the demand for what you’re building.

By: Kimm

Kimm — Wed, 05 Sep 2007 14:47:09 +0000

Implementation and scalability of customer service (the area where I tend to focus) also suffers from this phenomenon and for essentially the same reasons. Sometimes it’s even tougher because of the challenge of scaling the human factor involved in providing service and support.

Although no cure in itself, I find one mitigating factor is to think through the scalability issues early on rather than waiting for the pain to set in.

Of course it’s not justifiable to build out infrastructure too early. And it may not be predictable exactly when the infrastructure is needed. It IS possible to reasonably predict what the markers look like for when it’s time to get started and to have a rough plan already in place for how to proceed.

I figure it’s sort of like predicting the weather (another specialty, albeit ‘former’) – it’s usually timing that’s challenging, not the sequence of events.

Kimm