Why Innovation Fails in Big Organizations
Innovation by committee, business cases, roadmaps and projects — four horsemen of failed innovation?
This blog post is a summary of a chapter from “Inspired” by Marty Cagan. I wrote this summary as reflection exercise on a topic which is critical to all Enterprises — challenges of innovation in large companies.
Innovation and Enterprise
Large enterprises leverage their services through the value and the brand created many years earlier. It is a natural thing to do. Groups of stakeholders focus on protecting what the company has created in the past. Unfortunately, this may mean putting so many obstacles to new ideas that almost nobody is willing or able to take the company to new directions.
When a company was young it likely had a compelling vision which is largely achieved now. And now the people may not be sure what’s next. Product and engineering teams complain about the lack of vision and empowerment. The fact that it takes forever to get a decision made, and the product work is turning into design by committee.
Company leadership may also not be happy too due the lack of innovation in the teams. They sometimes attempt to solve it by creating semi-independent “innovation centers” to facilitate new products in protected environment. Yet this rarely brings the innovation the company is desparate for.
So maybe large-scale innovation is not possible in enterprise companies? We see that it is not true. Companies such as Adobe, Amazon, Netflix continue innovate successfully even at high growth rate. The fact is that the other enterprises could do it too. But that requires making big changes.
Root Causes of Failed Products
In enterprise companies ideas for new products usually come from key company stakeholders or customers. To plan implementation of those ideas the companies want to prioritize and put them on a kind of products roadmap. They have two reasons for that. Firstly, stakeholders want product/engineering teams to work on the most valuable ideas first. Secondly, they want to know when the products will be ready.
Roadmaps are usually created in quarterly or annual planning sessions. Leaders discuss ideas and negotiate a product roadmap. And to prioritize the product roadmap, they need a business case for each idea.
Business cases are usually aim to answer two questions:
- How much money will a product make?
- How much money and time it will cost?
Based on the answers, the product roadmap is usually created and prioritized for 6 or 12 months ahead.
When the roadmap is confirmed, the product and technology teams get their work plans defined and communicated. Normally they will work according to that plan from the items of highest priority and down.
Once the most important idea is picked up by product manager, he usually meets the stakeholders to shape the idea and work on set of requirements.
Requirements often come as user stories. These user stories aim at explaining to engineers, designers and QA team what should be built.
Once requirements are ready, UX/UI team is tasked to provide interaction and visual designs.
When design specifications are ready, they reach engineers. The engineers use those designs and user stories to break all the work in some set of iterations. Often the idea is planned to be built as proof-of-concept (POC) or minimum-viable-product (MVP) first.
The work done by engineers and followed up by QA team. QA executes some testing to validate that the resulting application works as defined in the requirements.
This is how most of companies, large or small work. These companies, however very often aren’t satisfied with level of innovation. They also not happen that it takes a very long time to make it from the inception of an idea to a ready-to-use product.
All the steps can be illustrated in a simple diagram:
Although most of companies think they follow practices of Agile, the described process is actually waterfall. Waterfall process has been bashed for past 20 years, but how exactly is that a problem? Marty Cagan lists 10 reasons of how this process blocks innovation in the companies. I will dive-deep into four of them.
1. The source of ideas
Products developed in waterfall model products are usually stakeholder or client-driven. They focus around features facilitating sales. This may be not the source of best product ideas. Stakeholders rarely have enough time to dive deep into ideas of how products can solve particular problems of users. Another problem with stakeholder-driven innovation approach is the lack of team empowerment. The team is there just to implement. This is very similar to contractors, who focus on execution and (almost) never on innovation.
2. Business cases
Usually companies create their roadmaps based on business cases. Business stakeholders take ideas and ask two questions: how much money will it make and how much will it cost to build? Based on the answers the roadmap priorities are organized.
The problem with this approach at this stage is that nobody can really know how if a solution will be success or not. Best product teams know that half of their ideas about products and new features will fail. If the product team does a great job, then solution could be very successful and have an impact on a big part of the company. More often than not, however, solution ideas end up not making into successful products.
This is a key product development lesson. At this stage we cannot know how much money will a product make. In a same way we cannot know how much money will it cost.
The second challenge with business cases is knowing how much it will cost to build. As we don’t know how solution will look like. At this stage costs estimation is mostly educated guesses.
Summing up — business cases aim to answer two important questions: how much money will it bring and how much money will it cost to build. When an idea is at inception level, answer to these questions are guesstimates only.
3. Product roadmaps
They mostly consist of a list of products and features often prioritized according to business cases. We just seen that early business cases are guesstimates at best. Cagan mentions two insights he calls most important problem of product. Two “inconvenient truths”.
The first inconvenient truth is that not all our ideas are going to work. There could be many reasons for that. The usual one is that our customers aren’t just as exited about our idea as we are.
The second inconvenient truth with ideas is that even successful ones usually take several iterations with implementation till they start providing business value to users.
According to Cagan, there is no way to escape these inconvenient truths. Difference between successful product teams and others is how they approach these inconvenient truths.
4. Outputs vs outcomes
Entire process is project-centric. Companies usually fund, staff, and launch projects. But projects are output and products are about outcome. This results into many orphaned projects. Something got released, but if it doesn’t satisfy the goals, then why to keep working on it?
The biggest flaw is of such waterfall process is that we all risks remain to the end of a project. Validation with customers happen too late, when most of the budget is spent already. If the idea proved to be wrong, we can’t get our time or money back.
Summing up
Digital product innovation is hard. The way most enterprises think about the digital innovation seem to be sensible, however it doesn’t work that well.
The good news, that many companies already figured out how to make new digital products successfully. Their lessons learned are based on similar principles and ways of working. And Cagan’s “Inspired” is just about that.
