Measure Everything

I’m at the Microsoft Partner Advisory Council (PAC) meeting (I’m on Microsoft’s Venture Capital PAC).  Simon Witts – the Corporate VP of the Enterprise and Partner Group – is keynoting the morning session.  It unbelievable (and extremely impressive) how much Microsoft measures about their business.  In his session, Simon has been extremely candid about positives and negatives based on what they forecasted for 1H05 (7/04 – 12/04), how they actually did, how this impacts their partner channel, the feedback loop that results, and what they are changing to perform even better.

While he was talking, I got my monthly status report from Rally.  It’s full of metrics – actual against forecast – and a discussion of the delta.  These metrics are across the entire business – it’s not just accounting, financial, or sales data – it’s information across the business.  Last night, I looked at my weekly report from StillSecure – same story – full of metrics along with discussion of what’s going on in the business.

While it’s easy to get lost in “data” rather than synthesizing and understanding “information”, I find way too many early stage companies measuring very little – thinking this is something that “only big companies both with.”  Microsoft shows how it’s completely ingrained in their culture – in my experience with them, they’ve been measuring everything forever. 

The value of building this discipline into the fabric of a startup is huge.  Automate as much as you can early in your life – turn “data” into information – and continually refine what you are measuring to make sure it’s the relevant stuff.  But measure it.

  • Thanks for the post on metrics. Very encouraging and motivating, but without specifics, I’m at a loss as to what to DO. Any place you’d recommend I start (books, sites)?

    I’ve recently been made responsible for monthly and prod sppt releases for the mission critical system of our young (almost 3 years old) company. Currently, I have way to measure the quality of our releases or process as a whole, but I am formulating ideas of what that would look like (number of bugs introduced, number of requests handled, average turnaround time).


  • O Kogbe

    I am currently reading “The Balanced Scorecard” by Kaplan and Norton. I would suggest that you take a look at their book. It does help with pointers on how to derive metrics that are meaningful to your organisation’s strategy.

  • Scott,

    FWIW – those operational measurements you have are good starting points. At the risk of being a little academic for a moment, there is something known as Little’s Law that specifies that average inventory = average throughput * average cycle time. I find when I try to analyze how well a business is working from soup to nuts, it helps to make sure that for each functional area (e.g., sales, marketing, finance, prod. dev) you are homing in on these types of things and then stepping back and sythesizing what it is telling you.

    As two brief examples, if I am tying to figure out the flow from marketing to sales, I will want to try to get an idea of the number of suspects/eyeballs pressed, the number of leads generated, the number of downloads, the number of qualified leads generated, where the lead came from, etc. I may then just have some requirement on how fast one responds to the lead (nailing up the cycle-time). The end result of whatever metrics are chosen are not to service the numbers, but to service what can I do to make things better.

    As a separate example in accounting, revenue often reflects the inflow, accounts receiveable reflects the aging and cycle-time, etc. If one goes further to figure out the distribution of receiveable or revenue, you can figure out whether there is too much risk in one area (e.g., receiveables too high for one client, or dependencies on one client too much for revenue) and try to do something about it.

    Sales also has some typical measurements and forecasts like number of calls, prospect appointments, # opportunities, range of deal sizes, # deals lost, reason why, etc.

    In product development, I try to create maps that map features to the customer, future market, competition, gotta haves, etc.

    I have not really found a single good source on reporting and measurements. Maybe someone else has. I have typically found that one needs to go to books, experts, best practices, etc. on each functional area. Lots of times you can find some starting points for measurements at the functional area level.

    The trick, however, is to figure out how all the flows connect to one another. I think this applies to both small and large companies. Where I might disagree with Brad a little bit on this topic is that I have seen many middle-market and larger companies also missing metrics. That said, the better ones make efforts to constantly improve. The best-in-class ones have measurements that are competitive.

  • Brad ,

    Nice post.

    For software engineering, the SEI CMM has strong foundation for metrics collection and continous improvement. A quarterly process capability report typically samples key orgainzational projects and measures critical parameters using some powerful stastical methods such as pareto analysis, fish-bone analysis etc. Some of the paramaters that are captured are defect density, budget variance, schedule variance etc.

    This level of process capability required organizational commitment and conviction that processes can deliver value.

    PMI offers strong processes and metrics strategy for Project Managers.

    For sales, I have found some intersting client focused sales processes at

  • Nice post. As John Young (former CEO of HP) said, “You can’t manage what you can’t measure.” The way that most successful companies measure how their business is doing is through Key Performance Indicators (or KPIs). Here’s some general advice on how to create a KPI system, if you’re interested:

  • Pingback: instant online car insurance quote()

  • Pingback: cheap online car insurance quotes()