Why hypothesis-driven thinking CAN lead us astray in innovation and transformation efforts
In many strategy-centric disciplines (management consulting, product, data science), the hypothesis-driven mindset is a required, pivotal skillset. However, one must remember that it is only a tool.
“The one thing you must remember as future business executives is that you are WEIRD. You are a Kellogg student, and have had previous life circumstances, experiences, and perspectives that shape and bias your worldview to be a certain way. Your perspective may not be how your customers view the world. Be wary of thinking you know best.” - Kevin McTigue, during a Marketing Management course at the Kellogg School of Management
One of the first skillsets that is pivotal to learn how to master in any strategic-related role is hypothesis-driven thinking. It’s hammered onto us from Day 1 at Bain, and is a mainstay of the strategic skillset in many companies. So it goes: if you’re able to break down a problem into its potential root causes, and zone in on a hypothesis of the solution based on first principles and past experiences — you can theoretically solve any problem. More specifically, the hypothesis-driven approach is built off of hypothesis-driven scientific research, defined by JEI as the following:
“They seek to address a specific, measurable, and answerable question. A well-constructed hypothesis has several characteristics: it is clear, testable, falsifiable, and serves as the basis for constructing a clear set of experiments that will allow the researcher to discuss why it can be accepted or rejected based on the experiments”
And it rightfully deserves a place in terms of great problem-solving toolkits given its many benefits: (1) quick prioritization of root cause problems leading to efficient experimentation, (2) forcing leaders / strategists to think deeply about the potential problems that exist, and (3) in a world of big data, is an important way to filter out the noise. There’s a reason why McKinsey is so renowned for incorporating this in their famous problem-solving framework, and why their consultants (alongside the other consulting houses) are desired recruits for almost any company and role requiring sound business judgment and strategic problem-solving skillsets.
However, because I am the way that I am and enjoy challenging status-quo frameworks / mindsets (and because so [too?] many of my friends are consultants and have adopted this as their primary mental model to solve problems), my thesis in this short essay is that hypothesis-driven thinking is only a tool and has the potential to be the wrong tool and potentially cause significant failure to an organization’s efforts and decisions.
Fundamentally, hypothesis-driven thinking CAN (and this is a disclaimer because there are counterexamples that obviously exist that you can point to) go wrong when:
The problem at hand is ambiguous and has no clearly defined “right answer” but is force-fitted a potential solution just because a hypothesis is needed; this often comes in the form of company leaders directing top-down ideas of what could work based on their “past experiences” or when product teams are not empowered to be customer-centric through a genuine commitment to discovery because leadership is adamant on what they believe are the right solutions. Notice that this problem is often on the side of leadership.
The strategist (doesn’t have to be leadership) fails to develop a hypothesis for an ambiguous problem that is fully comprehensive. You can’t fail your car keys if they are in a place you don’t consider. This is because there are no general, algorithmic means for generating hypotheses; hypotheses are a byproduct of experience, extrapolation and imagination — and people are by nature biased (as shown by the first quote of this article)
The world has undergone a paradigm shift but first principles / hypothesis-driven pattern recognition causes a strategist (e.g. most common example is an investor) to make decisions that might not fit within the reality of the world. In fact, this is such a common bias that in data science teams we call this concept drift (when a model is inaccurate because the underlying data has changed). An example is when companies apply business principles that worked before COVID-19, or when crypto technical analysts see a random “two peak, one drop pattern” and assume things are going to play out the exact same as the last time they saw that pattern.
When there’s no commitment to experiment-driven solutions and the hypothesis is treated as the answer. Note that this is one of the fundamental tenets of the scientific method. The hypothesis is meant to be treated as a way to get to the answer. However, in many teams, the hypothesis is treated as a “best guess with available data”. This is absolutely warranted in certain, more deterministic situations, but this can fail especially product teams or startups that are building products where discovery needs to be the first priority as you are working in a yet unsolved domain with biases on hand.
When there is an effort to be customer-centric and discovery-oriented BUT the people running the discovery don’t know how to leave their biases at the door — instead applying a subtle and misguided form of hypothesis-driven thinking. This is why running user interviews is a skill and is tremendously difficult. We are naturally inclined to confirm our own biases (something I’ve seen over, and over, and over again)
When it fails — it does two things: (1) it wastes a depressing amount of effort and thus weakens the trust and the conviction of teams, and (2) we don’t actually get to the actual right solution to solve the actual problem at hand.
I’ll illuminate this with three hypothetical examples — one in product leadership, one in organizational transformation, and one in discovery — I’ve personally seen play out in other settings where hypothesis-driven thinking has failed.
1. When leadership applies their hypothesis-driven thinking to demand features rather than for the purpose of providing strategic context
There was an education-technology startup where the founder was adamant that the product should have all these features because his “hypothesis” based on his experience as an educator and as someone who previously worked with educators informed him that these features were incredibly important.
As a result, this mindset transpired to the product teams he led and as such the product team was not empowered to discover solutions through discovery (making a continuous effort to learn and understand customer problems and needs, working with reference customers). The product manager was a de-facto project manager, and designers / engineers were not motivated to be their best and be creatively involved. There was continuous tension between the product team and the founder. The product team actually knew and sought to create genuine impact — but it was impossible because the leadership was too set in their own thinking.
2. When an organizational transformation is top-down driven and chaos ensues after the fact because people don’t really agree at their core
An organization seeks to do a organization topology restructuring (aka shift teams around, make spans more efficient and aligned to business objectives at hand). They develop first principles of why these objectives are important and speak to one another on the leadership team with absolute confidence that this is the way to go. Fortunately, they make the effort to conduct group meetings inviting team leads explaining the first principles of the restructuring and attempting to get their buy-in and opinions — but the conversations are largely still top-down and there is more “convincing” rather than “conviction” being developed. Of course, no one says anything in these meetings because no one wants to be the bad guy.
A few weeks later, it’s discovered that many people in the team are in semi or complete disagreement to the restructuring effort and find flaws in the principles being advocated for as it does not align to the operations of their business at the core. The entire restructuring effort is delayed as there’s so much conflict despite the amount of time that has been spent strategically developing hypotheses and planning for it. When conducting the post-mortem, there is a realization that barely any 1-1’s were had with even the team leads, and team members that were not leads were not involved in any of the discussions at hand which directly affected them.
3. When you attempt a discovery phase but with people that don’t leave their biases at the door
Your team has this great idea for a new product to solve this genuine problem faced by the industry (for simplicity’s sake let’s say developing a B2B product to assess ESG risks of a company). The problem has been validated through data (yay!) and the team is now trying to design a solution to solve the problem of companies not being able to fully understand their ESG risks putting them in trouble with regulators. They go through the discovery phase and meet with potential customers to assess their pain points and dive deeper on the problems they face. However, they spend most of their time ideating about potential solutions with the customers rather than listening to the customer and letting them share their experiences. These folks get out of their conversations happy because it seems productive — but rather they’ve tainted the integrity of the discovery phase as they littered it with their own hypothesis-driven ideas rather than letting customers have the spotlight to share. As a result, they don’t really get into the root customer issue and their product fails to gain meaningful traction.
So what’s the solution to this? I’ll write a more fully-fledged article later on, but it boils down to: (1) leadership empowering teams to be bottoms-up driven at heart by inspiring clear objectives but allowing product / transformation teams to discover genuine solutions that work for the intended stakeholders, (2) developing radical empathy by instituting processes within the company that allow for discovery to be a prioritized value (e.g. product teams meeting with customers every single week, or in a decision involving a large-scale company transformation — having 1-1 conversations with the broader team that will be affected and not just the leadership), and (3) learning to recognize when you do not have the full picture and are making unfounded assumptions / inserting your biases.
Hypothesis-driven thinking can be a fantastic tool, but I hope that this article can illuminate and cause you to build another framework / mental model in your mind if you’re a leader or strategist responsible for decision-making in your organizations. Remember that tools should be fit to the situation at hand rather than be utilized as a crutch for everything.