Get Your Metrics Together

As we head into 2017, I have a steady stream of operating plans hitting my inbox. Since many of our investments are companies that are scaling, vs. companies that are just getting started, there are a lot of derivative metrics in these plans.

Q1 is the easiest quarter to make your plan, so most of these companies are getting a free pass for the next three months after fighting the good fight of making or beating plan in Q3 and Q4. For the SaaS and recurring revenue companies, if they missed by more than 5% in 1H16 and didn’t reforecast, they’ve had a particularly grueling uphill climb for the past six months.

While the relief of Q1 was missing last year because of the existential freakout caused by the public markets (anyone remember that?) I know a bunch of people who are hoping Q1 will be nice, calm, and normal. Good luck with that.

Regardless, you can start the year off by being clear on how you calculate your various derivative metrics and make sure that your plan – and the expectations of your board and investors – fit what you are putting out there for the next year. Before you say, “yup – no big deal – we are great at that” go read two posts by Glenn Wisegarver, the CFO at Moz.

If you’ve worked with me, you’ve probably heard me call out CAC as a nonsense metric, since it’s super easy to game. Or maybe you read my post about ICDC (increase conversion, decrease churn). Or, instead of growth rate at a moment in time, you’ve heard me ask for a monthly graph of trailing twelve month growth rate so we can see the actual acceleration or deceleration of growth, which is way more interesting than last months growth number.

There are tens of thousands of words written on the web about SaaS metrics, consumer metrics, recurring revenue metrics, and all kinds of other metrics. Entertainingly (at least to me) there are very few words written about CE / hardware metrics (other than nonsense about how to value CE companies).

As part of getting your metrics together for 2017, I encourage you to go read some of these articles. And think hard about which metrics really matter and where the change in them will impact your business performance in 2017.

Also published on Medium.

  • I think one of the trickiest aspects of devising appropriate metrics is to determine the fundamental transactional unit. There is a real danger that if you don’t think about what the most appropriate such unit is for YOUR business, that you unreflectively try to map over the unit used in articles you read about companies that are in fact subtly (or perhaps not so subtly) different from yours.

    It can often be possible to think about alternative fundamental transactional units, and importantly, some of them are much easier to apply in terms of modeling the unit economics. But IMHO one should go for the best possible unit that most naturally and directly relates to YOUR business, and then figure out how best to do the modeling.

    • Correct. Too many people just toss out a bunch of metrics without actually understanding which ones impact what.

      • Yes.
        And IMHO it all starts with picking the correct fundamental economic transactional unit. In many cases this is pretty straightforward but in others it’s pretty tricky.

        Consider the following example. A company that derives revenue from people watching video streams. What do you measure? The number of customers? The number of films in your catalogue? More customers is likely correlated with more revenue so that ‘could’ make sense. The leads you to measure things like CAC, churn, cohort analysis etc. But what is the unit economics of a customer? Is this really the most fundamental transactional unit? How about films in catalogue? More films in your library is likely correlated with more revenue. So now you measure cost of acquiring titles, maintaining a catalogue etc. But once again, is this the fundamental economic unit? These kind of metrics provide insight but are they the absolutely most fundamental to such a business? How about measuring ‘viewed streams?’ Is this fundamental? To me, the answer is yes. The unit economics are directly related to this metric. If I make more per viewed stream than it costs me to deliver a viewed stream the immediate next question is – what happens when I scale this business? But there’s an issue – measuring cost of a delivered stream is tricky. It’s probably harder than coming up with a CAC. So people balk against these kind of unit economic metrics. IMHO however, you are far better off drilling down to the most fundamental transactional unit then struggling to find ways to provide metrics because imperfect though they may be they are direct proxies for the true nature of your business. And you can’t manage what you can’t measure. Measure the wrong things and you’re not managing the real business. IMHO the key is the fundamental transactional unit.

        • Rich Weisberger

          Techniques like principal component analysis can also take the guess work out of this approach.

          • I’m not sure how that would help. Perhaps after the fact? You figure out what your most fundamental transactional unit is then after devising all metrics and gathering some data you apply PCA to confirm your hypothesis?