Dangers of being Data Driven

Being data driven as a product leader is considered to be table stakes today. However, my view is that with the advent of easy access to data, ease of building software, and clever marketing (Hi Big Data!), we have veered too far on this view. There are strong limitations to being data driven. I try to highlight three ways in which being data driven is a bad idea.

no data for behaviors that are latent

One of the most successful innovations in the history of our civilization was the decimal system. It gave us an easy way to count. It unlocked a latent capacity – which was our ability to count. However, if you were asked to bring data to prove how successful the decimal system was going to be, you probably won’t be able to find any systematic data on this.

Fast forward to a few years ago, if you look at contemporary examples of some of the most successful products, like the iPhone, all data suggested that there was no need for an expensive phone that didn’t have a keyboard. The reality is that game changing products are borne out of a vision of how the world ought to be, & the products themselves become a means that realize the vision.

Instead of collecting data prior to building a product, what’s important is to articulate what sort of data will be collected to determine if the product is realizing the desired vision.

Poor decisions based on easily accessible data

Let’s assume that you actually have data to prove that the product decision you are going to make, but it’s hard to transform or not quite in the shape that would prove your hypothesis. As you go looking, you run into some data that points to a broken problem, which adjacent to your original hypothesis. At this point, the vividness of data that points to an adjacent problem will suck you in to making a set of decisions which will steer you away from the problem you originally intended to solve.

What’s worse is that after the product is launched and adopted, the data will show the problem is solved, and you’ll pat yourself on the back. All this while, what you did was solve a problem that data readily made visible, not the most important problem

This is one of the most important traps that product teams fall in to. That’s why it is very very important to articulate a hypothesis before looking for data to validate it, instead of digging through data to find all kinds of problems.

Game Changing products are a means of discovering the world

Like paradigm-defining science, game changing products often follow a path where the team working on the product notices that users are using the product in ways the team didn’t expect them to. This is followed by the team quickly digging a little bit deeper, and then iterating to expand on the functionality that their users loved. That is to say, the journey of a successful product is often filled with accidents.

The best way to increase these types of accidents is to cycle through product ideas quickly, and ensure that these ideas are in line with the original vision of what’s broken in this world.

On the Vocal Minority

In a product discussion yesterday, some of our team members touched upon how much weight to give to a vocal minority that finds flaws in the product. I think this varies from case to case. While the general view is that the vocal minority might not be representative of the overall view is true, I suspect there are valid reasons why one should pay attention to the vocal minority.

Vocal minority as a problem radar

Being vocal can be seen as a tipping point when problems aren’t small enough to tolerate. We are at risk of viewing a vocal minority as cranks (a political analogy would be birthers) and ignoring problems they raise. In reality, it is very important to probe deeper to understand the extent of the problem when a small section of the user base is vocal about it.

perception matters

In a hyper connected world where finding information is only a few clicks away, your product & business is often judged well before you have a chance to speak to a potential customer or user. Well before a user enters your acquisition channels, he or she has a well formed view of how good your product is. This view is formed & shaped by the vocal minority. If you don’t pay attention to this phenomena, its only a matter of time before the top of the acquisition funnel runs dry.

Confirmation bias is real

Your successful users are often influenced by the vocal minority as well whopaint a very vivid picture of the flaws of your product. The vividness of the flaws results in them being noticed by successful users of your product/ This inevitably sours their experience of the product. Confirmation bias kicks in, and gradually unbeknownst to you, your successful users begin to turn against you. These successful users might also look at how you treat the vocal minority to form their opinion about you & your product.

Unpacking the vocal minority

The best techniques to get under the loud voice of the vocal minority to speak to your users at scale using NPS surveys (wootric, promoter.io), support software (Intercom, zendesk, userdelight), & track product usage with tools like Totango, Mixpanel etc.

The most successful products today are a step ahead where they are using the vocal minority expressing favorable opinion to attain hyper growth. In fact, this is a big reason why products like Slack or Product Hunt will be difficult to catch up to. Vocal minority when developed right becomes a moat.

A Dozen and One books you ought to read

Thinking Fast and Slow. This masterpiece explains how we make decisions under uncertainty, & how even our higher order thinking is quite primitive.

Getting to Yes. Based on the Harvard Negotiation Project, this book describes principles for problem-solving negotiation.

Good Strategy Bad Strategy. Richard Rumelt’s book provides a sure footed & extensible framework for addressing high stakes challenges we encounter in business, filled with great case studies.

The Design of Everyday Things. Don Norman’s classic describes product design based on the principles of users’ needs and cognitive psychology.

Flow – The Psychology of Optimal Experience. Mihaly Csikszentmihalyi in his pioneering work describes how flow, a state of deep engagement, results in heightened creativity & fulfillment from life, including pathways to attaining it.

Genghis Khan & The Making of TheModern World. One of the greatest emperors ever, who took a contrarian view towards rules of war & ruling itself, and as a result, cast a long shadow on the modern world.

The Mythical Man Month. Unlike other complex projects, like marathon surgical procedures or large scale construction projects, software projects almost always go over budget. This book by legendary computer scientist Fred Brooks unpacks how software projects fail, and what we can learn from them.

High Output Management. A compendium for managers which builds a framework for how to manage, with specific details on day to day activities like goal setting, progress indicators, meetings, managing a calendar, 1 on 1s, matrix management, etc.

The Goal: A Process of Ongoing Improvement. A fictional account of a factory manager racing against time to turn his factory around. An illuminating discussion on the importance of goal orientation as well as the process of solving right problems at the right time, in pursuit of the goal.

Out of the Crisis. In this seminal text, W Edwards Deming, the father of the modern quality & productivity movement, discusses his management principles & their application to diverse industries.

The Innovator’s Dilemma. A book that provides the original definition of disruption, as well as many current silicon valley axioms like “come up from underneath”.

The Four Steps to the Epiphany. A playbook for how to validate your vision & increase likelihood of success, especially if you are building an enterprise product. Many would consider its teachings common sense today but this is a must read for entrepreneurs.

Soul of a New Machine. Wonder what its like in the early days of a tech startup, when its just a bunch of engineers trying to build something. This Pulitzer winner takes you inside on such project.

Design criteria for enterprise applications

When designing applications for teams that are organized around key outcomes, like hiring, or selling, there are two criteria that are enterprise applications should optimize for.

Reduce decision-distance

Lets consider recruiting as a process. The goal of recruiting is to hire great people that help the organization grow. If you wanted one role to make a decision about every candidate considered, it would be the hiring manager, given that they have the most insight. However, they don’t have enough time to do this. Therefore, they provide proxies to recruiters and sourcers to make the decision on their behalf. This distance between the person most qualified to make the decision and the person making the decision is what I call decision distance.

Applications that help reduce “decision distance” will help teams make better decisions, which is fewer false negatives & reduced loss in productivity due to fewer false positives.

Reduce work in process

Moving & managing a large number of candidates, or leads requires significantly more effort than moving a small number of leads. Given a fixed duration, reducing work in process will reduce the capacity required for the same amount of work accomplished.

The former results in a vicious cycle that slows organizations down. It particularly drains time which is the most scarce resource for a hiring manager. Further, the ability to get a hiring manager involved with decision making is inversely proportional to the # of decisions that have accumulated on their plate.

Applications designed to help make decisions “faster” enables small teams to have the impact equivalent to big teams, while staying more responsive.

Five principles of enterprise software design

Few would argue that enterprise software today is, by and large, unchanged from the 1990s, when the PC revolution first took hold. With the advent of mobile phones as well as mind-blowing consumer software, we are beginning to see enterprise software gain some much needed attention. It is too important to be left to those who brought us the original enterprise tools. However, “consumerization” alone is not enough to bring about much needed change in how people work.

I believe that the tools we use for work shouldn’t just meet our needs (workflow needs, compliance needs, security needs, project management needs) but they ought to reveal new possibilities of working together and making better decisions. Tools we use for work should facilitate a culture that fosters critical thinking & collaboration, not silos and politics.

With this in mind, I propose five principles of design that should be kept front & center when building enterprise software.

Transparency

Transparency is the 1st step for building enterprise software. Capturing & storing data in a manner than can be retrieved and queried is table stakes today. What is needed is that the information that is stored in silos is made transparent to all relevant constituents. Every function that I can think of at a workplace will gain from more people knowing, and thinking about it. Transparency, in fact, starts with creating & sharing goals across an organization. Tools should enable this behavior.

Critical Thinking

Once you have the information, it is very important that this information inspires critical thinking. Numbers shouldn’t promote vanity, or a sense of inflated / deflated self worth. More information shouldn’t be used to prove that one team is better than the other. On the contrary, information ought to be presented in a manner that allows people to think critically about what that information might mean. Our tools should anticipate our questions, or guide us towards better questions.

Inquiry & feedback

After we are able to think critically about the information presented, it is almost certain that further learning will come from asking thoughtful questions & providing feedback. Enterprise applications should enable this contextual inquiry and feedback, that allows people to think about the specific problem absent of “I don’t know what you are talking about”. It should also enable this to happen without the need for 5 people to gather in a room.

Constant Improvement

Once our tools enable inquiry and feedback, our tools should reveal how our decisions & actions are improving our performance against our goals. Our tools should help us understand how our decision making has improved as a result of our actions.

Excellence & Harmony

Finally, based on the last 4 steps, tools should help us achieve excellence & a state of harmony with our work. This is a state where people, and the tools, they use become one. While there are many cases of complete harmony between a worker and the tool, enterprise software designed on the principle I state above will result in complete harmony between teams and their work, thanks to their tools.

When applying these principles towards designing enterprise software, there are some values we should keep in mind. For every decision we make about designing enterprise software, we should ask the following questions

1. Does it increase trust between teams?
2. Does it create a sense of togetherness?
3. Does it make people feel thankful?

I believe great enterprise software must result in a trustful environment where people feel a sense of togetherness & purpose, and are thankful for their team & work.

Why is top of the funnel so difficult to improve?

This excellent article by Tom Tunguz proves the impact of top of the funnel identification & qualification to improving key SaaS metrics. I encourage you to read that article before continuing.

Investing in improvements at the top of the sales funnel, particularly in early prospect and lead qualification are so valuable: they save the time and energy of a startup’s go to market teams and can meaningfully improve unit economics.

However, improving top of the sales funnel is made difficult because of two key fundamental traits of all inside sales models (perhaps, even field sales models but their larger deal sizes change the dynamics a bit)

Lack of feedback loop between qualification & won deals

It is clear that people learn best when we get immediate feedback on our actions. However, the sales process after qualification is lengthy and when a deal is lost, the sales development rep often gets little to no feedback in order to improve their qualification process. Yes, sales leaders will deploy playbooks. However, the environment is so irregular that a playbook (which is a rules of thumb document, & historic on arrival) is a flailing attempt at solving this problem

Sales development reps are inexperienced

There is a clear hierarchy in SaaS sales organizations. SDRs are the bench, and often learning on the job. They aren’t astute enough to read the tea leaves of a lead, identify patterns, and make “good” decisions about qualifying the deal in or out. This exacerbates the problem of top of the funnel qualification.

I’ve been thinking about these two areas quite a bit, and starting to play with some approaches. More to come as I find time for my side project.