What Accident Statistics taught me about Metrics

I first took interest in goal setting & metrics nearly 5 years ago at a fledgling startup where we were trying to get our first product off the ground. The Lean Startup wasn’t out yet, High Output Management was hidden somewhere on Ben Horowitz’s bookshelf, & Goal was known only to Jeff Bezos’ inner circle. My research led me to “Out of the Crisis” by Edwards Deming, first published in 1982.

If you are unaware of Deming, he was a key management thinker (the anti-Drucker according to some, though they had more in common than distinct) who was ignored by American managers but embraced by the Japanese. American managers were big on the quota system, automation, & the compromise between quality & productivity. Deming on the other hand espoused the belief that the only way to achieve long term productivity is by improving quality (measuring it through statistical means & improving it constantly). Eventually, his principles & teachings led to the Japanese manufacturing revolution, & created The Toyota Way.

“Learning is not compulsory, neither is survival.” – Deming

The book is deeply insightful, and explains many management concepts in terms that are easy to grasp (the book is devoid of inaccessible management speak, & filled with inaccessible statistics instead). One takeaway of the book is that management requires one to roll up their sleeves, get immersed in the work to understand it well, & away from “dashboards”. In 1982, he cautioned that the PC revolution would encourage managers to rely to easy to access dashboards without understanding the actual business.

Outcomes vs. Actionable Metrics

One of the concepts the book explains is the difference between actionable metrics & outcomes. While outcomes ultimately matter, they are terrible at guiding day to day decisions.

An example Deming uses here is accident statistics. Let’s say that a city wants to reduce the number of deaths due to accidents, from 1,000 last year to 900 this year. Measuring this helps the city understand how its doing but it doesn’t inform what the city should do to move the needle. He calls this type of metric an outcome. He tells managers to reduce their reliance on metrics of this nature to inform their day to day decision making.

Instead, he encourages that managers dig deeper to find actionable metrics that represent the system — metrics that inform their actions but also connect to the outcome they are trying to impact.

To reduce accidents, actionable metrics would be number of intersections with cross walks, total length of paved side walk, number of speed limit signs posted, number of speeding tickets or DUI tickets issued, number of intersections with stop signs, etc. Prioritizing & measuring these would give the manager leverage on the outcome, & inform day to day actions & decision making. The manager could run many types of analyses & experiments to verify what impacts the ultimate outcome, what offers greatest leverage, & then focus on those areas.

To summarize, I think it is super important for everyone to understand what metric they can impact & how that connects to the ultimate outcome of their team & the business overall. If you haven’t done so already, now’s a good time.

 

What Accident Statistics taught me about Metrics

Leave a comment