As an analyst, you probably spend a lot of your time creating dashboards and other reports to help your teams and Top Brass track metrics. And you probably spend at least some of this time grinding your teeth, because you know more than a few of your startup's metrics aren't helping people solve critical problems or otherwise producing a lot of value.
Odds are you already have thoughts about how your startup can do a better job in deciding what to measure and how they measure it. But if you want to help your company really elevate its game, it's useful to understand the history of businesses' struggle to produce metrics that do more good than harm.
To help explain why measurement is so complicated, I'll give you a whirlwind tour of the history of three types of modern metrics often used by startups—KPIs, OKRs, and the OMTM—that'll help you discover how to find the key to unlock next-level performance for your business.
Defining terms
If you aren’t quite sure what the difference is between a metric and a KPI or a KPI and an OKR, you are not alone. Each of these ways of measuring success comes with a whole lot of baggage, but understanding how they fit together—and why they keep evolving—is a key starting point.
What is a Metric?
A metric is a means to measure performance or progress. Just as apples and oranges are types of fruit, KPIs and OKRs are types of metrics.
Just as apples and oranges are types of fruit, KPIs and OKRs are types of metrics.
Although metrics are usually numbers, they don't have to be. If you've got an indicator on a dashboard that can either be red or green, that indicator is displaying a metric.
At this point you might be wondering, if we've got the word "metric," why do we need all these other terms?
The answer is simple:
Measuring is often hard.
Measuring what matters is harder.
Measuring what matters and what you can do something about—aka, what's actionable—is even harder.
And measuring what matters and you can do something about and doing it without inducing people to act like a hamster on a hamster wheel, game your system, or otherwise engage in deeply dysfunctional behavior that could hamper your startup? That's the hardest of all.
Over time companies have searched for better ways of making the hard work of coming up with high impact metrics a little easier to pull off.
Exhibit A: KPIs.
Money isn't everything: The rise of Key Performance Indicators (KPIs)
Once upon a time, US carmakers were hungry, nimble startups. By the end of World War II, they dominated a globe where they faced virtually no competition (in no small part because every other country's economy had been devastated by two world wars). And at that point, they started raking in serious dough.
As a result, by the 1950s and ‘60s the voices of "car guys" who were passionate about building high quality products that delighted customers had been drowned out by “finance guys” who were myopically focused on the bottom line.
That's why "lean" startups like Toyota ended up kicking their ass.
By the early 1990s, it became brutally clear that putting financial metrics in the driver's seat was a great way for US car companies to crash and burn. So they began to look for a different strategy.
Many companies ended up embracing a concept developed by Robert Kaplan and David: the "Balanced Scorecard." The idea was simple: Corporate Top Brass would use a performance scorecard that balanced financial metrics with everything else that mattered for a company's success.
Money is pretty simple to measure. So if you're going to start focusing your company on factors that are a bit more complicated to measure, how do you do it? The answer: key performance indicators (KPIs), a term that had been around for a while but didn't take off until it was popularized by Balanced Scorecards.
The main idea behind key performance indicators came down to one word: "key." Don't just create a bunch of metrics. Figure out which metrics are most critical to focus on.
One of the reasons KPIs became so popular is that they provided a framework to easily connect the big picture to everyone's work. You could create strategic KPIs for the entire company, then create tactical KPIs for teams and projects. As a result, everyone in the company would be focused on the critical work they needed to accomplish so the company would nail its strategic KPIs.
Seeking out alignment with OKRs
In theory, KPIs sounded like a great idea for helping companies focus. In practice, they could easily morph into a giant rat's nest.
That's why a recent survey of executives at big companies found that "only 26% agree that their functional KPIs are aligned with the organization’s strategic objectives to a great extent."
If you're a ginormous multinational corporation that's been around forever, maybe you can afford to be this dysfunctional. But if you were a startup in the 1990s and 2000s, a rats nest of KPIs was as big of a recipe for disaster as Big Auto's narrow focus on financial metrics.
Back in the mid ‘70s, Intel’s Andrew Grove—who coined the phrase "only the paranoid survive"—experimented with a method of developing metrics focused on helping startups like Intel thrive. Fast forward to 1999. John Doerr was a former Intel staff person who'd been trained in Grove's method and who was now working at a VC firm. Doerr proposed to the founders of Google that their fledgling company adopt a variant on Grove's successful approach that he called Objectives and Key Results (OKRs). OKRs helped Google become a powerhouse, and over the next few decades OKRs spread first throughout the tech industry and then to other industries.
An OKR is made up of 2 parts:
- The What: a clearly defined goal, called an objective
- The How: 3 to 5 "key results" that everyone can use to track whether they are going to hit that objective.
Notice once again the use of the word "key." Like key performance indicators, OKRs are designed to get you to focus on what's most critical, but OKRs have two features that made them stand out:
- Tempo. KPIs are written with the assumption that you'll be using them for at least a year if not longer. OKRs assume you are living in a world where you need to adapt faster. As Doerr wrote, “measuring what matters begins with the question: What is most important for the next three (or six, or twelve) months?"
- Committed and aspirational goals. OKRs come in two flavors: committed goals, which you are absolutely committed to nailing, and aspirational goals, that are crazy ambitious. Doerr's argument was that the only way audacious startups were going to succeed was if they had at least some metrics that pushed them to shoot for the moon. That's also why he, Google, and many others who use OKRs argue that startups shouldn't tie OKRs to compensation.
While OKRs and KPIs serve different purposes, they aren't mutually exclusive. For example, it's not uncommon for companies to use OKRs for their overall direction and KPIs for projects and teams.
One Metric That Matters (OMTM), aka Your Company's North Star
Not surprisingly, OKRs can run into just as much trouble as KPIs—especially in the hyper-fast world of startups. As a result, in the early 2010s Alistair Croll and Benjamin Yoskovitz developed an approach which built on the concept of lean startups: Lean Analytics.
Lean Analytics has a lot to say about the types of metrics that a startup will want to track depending on the stage they’re in and their type of business. But what makes Lean Analytics truly unique is the concept of One Metric That Matters (OMTM).
Like metric wranglers before them, Croll and Yoskovitz argue that the biggest problem facing most startups is focus.
Founders are magpies, chasing the shiniest new thing they see. Many of us use a pivot as an enabler for chronic ADD, rather than as a way to iterate through ideas in a methodical fashion.
That means it’s better to run the risk of over-focusing (and miss some secondary metric) than it is to throw metrics at the wall and hope one sticks (the latter is what Avinash Kaushik calls Data Puking.)
And so, at any one time a lean startup needs to have One Metric That Matters. As Alistair Croll explains,
That doesn’t mean there’s only one metric you care about from the day you wake up with an idea to the day you sell your company. It does, however, mean that at any given time, there’s one metric you should care about above all else.
As a startup starts to get control of the problem that's driving their OMTM, new problems will arise that require a new OMTM. For example, suppose traffic to your site has finally seen a dramatic increase. Now it's time for OMTM to focus your startup on your new biggest challenge: maximizing conversion.
Although One Metric That Matters is a very different strategy than OKRs, it's hunting on familiar ground. For example, the book Lean Analytics has a list entitled, "Four reasons why you should use the One Metric That Matters":
It answers the most important question you have.
It forces you to draw a line in the sand.
It focuses the entire company.
It inspires a culture of experimentation.
...that sounds almost exactly like the advice John Doerr first gave to Google.
What the history of modern metrics can teach analysts
Why bother going through all this history? It's not just about untangling all those acronyms (although that helps). More importantly, the evolution of of modern metrics offers some important lessons:
1. Creating the right balance of metrics is tough, and that's okay
It's easy to be overwhelmed by the blizzard of acronyms, methods, and frameworks for using metrics. Once you understand the history of modern metrics, you'll realize that behind all of it there is a very simple story: the methods we use today are the result of a series of people trying to fix the metrics mistakes of the previous generation.
So don't get caught up in all the hype. Everybody who's tried to come up with a better metrics framework has made progress but has ultimately failed to produce a perfect solution for all startups—and while Lean Analytics hasn’t been replaced by a new framework just yet, you can be sure that at some point it will.
Once you understand that, you're already ahead of the game.
2. Every analyst needs to be agile in their usage of metrics frameworks, whether they work at a startup or not
Each iteration on a better way to use metrics was led by sharp cookies. And while none of them got it exactly right, each successive generation learned from the past and improved on it.
So instead of worrying about whether your startup is using the perfect metrics framework right now, focus on the process of iterating and getting rapid feedback on how well your startup's current metrics framework is working.
The other implication of this lesson is that you need to use a data platform that makes it easy to change up your metrics and the data that fuel them.
3. Your startup needs a metrics coach—why shouldn't it be you?
If you're in a big corporation, odds are you have coworkers with years of experience at figuring out how to find the right metrics. In fast-moving startups, even if there are a few folks with that experience, hiring happens so quickly and the world they operate in changes so frequently that it's hard to build up fluency in using metrics strategically.
If there was a simple, cookie-cutter way to choose a metrics framework that works for all startups, this wouldn't be such a problem. But as we can see from the history of KPIs, OKRs, and OMTM, there's no easy way to choose.
That's the bad news.
The good news: your startup almost certainly needs help making sure it's measuring what's most important today. And now that you understand how the world of modern metrics was put together, with a little work and a little more knowledge about how to coach your startup—the subject of our next article—you can build exactly the right skills to help people in your startup elevate their metrics game.
Conclusion
Many years ago, before dating apps and the Internet, young people in Britain and elsewhere entertained themselves playing a game called bobbing for apples:
In one set of rules, each apple was assigned to a potential mate. The bobber would then attempt to bite into the apple named for the young man she desired. If it only took her one try, they were destined for romance. If she succeeded with her second attempt, he would court her but their love would fade. If it took three tries, their relationship was doomed. Another approach to the game was a race to be the first to bite an apple; the first to emerge successful would be the first to marry.
For all of our sophistication with data, we are only a step or two away from bobbing for apples. Figuring out the path to a startup's success—of finding the right customers and building a lasting relationship—is incredibly difficult, and nobody has the perfect formula figured out.
But by understanding the history of modern metrics, we can pick up some pointers that can help us spend less time fruitlessly thrashing about in the water and more time sinking our teeth into the right apple.
