Software is Eating the World and Businesses Have Indigestion

Marc Andreessen, founder of Netscape and a famous Venture Capitalist, wrote an essay in the Wall Street Journal a few years ago that said “Software is Eating the World.”  In an earlier post, AI Rhymes with Cloud, I discussed how businesses need to approach adopting AI by first building a solid data foundation and moving to “cloudify” their IT infrastructure.  Only after adopting these measures can they progress to beginning the journey to adopting AI techniques and reaping the benefits of AI.

In the previous post, I talked about cloudification and the need to break down information silos, and I also mentioned the need to address organizational silos. Since writing that post, I have worked with customers in many countries who are trying to move forward on their digital transformation journey and I have seen them struggle with both the technical and organizational challenges of this journey. My overall conclusion is that, while the technical transformation is difficult, the organizational transformation is not only harder, but it is more important. If the IT organization is not set up to support the “to-be” vision of the digital enterprise, it will be almost impossible to create the technical synergies required to break down the digital silos.

What We Learned

When Huawei embarked on its digital transformation journey, we faced exactly that situation. After much work on our IT evolution, we found that we were not getting the benefits of digital transformation and the reason was our organizational silos. We got the IT organization to adopt Agile Development techniques but because of the organizational separation, we got fast implementation of siloed applications and no unified data platform that would allow us to move towards becoming an intelligent enterprise, a requirement in our hyper-competitive markets.

In fact, we actually regressed in some areas because of a lack of unified user interfaces, duplicate requirements for data entry, and different rules for information validation. It was only after we organized around our desired outcome that we began to make real progress. The results have been verified by our stable IT and employee headcount even as we have grown by more than 20% each year.

Read more: So What Does “Digital Transformation” Really Mean?

What Needs to Change?

To understand what has to change in the organization, we need to look at the underlying technology transformation. The foundation for an intelligent enterprise is having unified all of the information in the organization and developed methods of integrating this information with key external data in a Data Lake. If you will be adopting modern storage technologies, this may be a “virtual” Data Lake because you can access all of the information in-place without having to move it to a separate repository. This concept of being able to access all of the key information without having to move it to a separate repository is particularly important because one of the major pain points which businesses encounter in their journey to an Intelligent Digital Transformation is how to get all the information in the silos in one place. Huawei’s recently announced FusionStorage 8.0, which allows different kinds of access (block, file, object, HDFS) to the same underlying data storage, is an example of how evolving technologies can ease the creation of these information repositories.

One bank which I worked with spent three years of a dedicated organization’s time and only managed to move 20 of more than 200 data sources into their data lake. Sometimes having too much planning and too much organization can delay the benefits of unified information but in this case, a contributing factor seemed to be that the organization responsible for combining the data was completely separate from the organization that needed the combined data. Another bank I worked with was so convinced of the strategic value of the data that they created a separate subsidiary to not only create and manage the data lake but also to seek new external customers that could benefit from the information contained in the data, opening up a new source of revenue for the bank.

Being Who You Want to Be

Picking the right “to-be” organization for Digital Transformation will be different for all organizations, but there are some guiding principles

• Data belongs to the business, not to departments. The most common impediment to creating the unified data repository needed to move to “intelligent” is when the departments that “own” the data are reluctant to supply that data. The reasons often revolve around issues of confidentiality and privacy, but the real reluctance may be about power and control. In any case, there are many readily available tools which can anonymize and obfuscate sensitive information so that the information can be used to create a viable repository without creating security and/or privacy concerns. Huawei’s FusionData solution is a good example where using granular control of user and application access is paired with tools that make it easy to anonymize sensitive fields.
• Waterfall development processes can never be agile. Many IT organizations have spent years developing and honing processes for developing solutions. Virtually all of these were designed to optimize the traditional product development steps of Define, Scope, Develop, Test, Trial, Release. In today’s environment, that just won’t work because it is too slow and cumbersome. As much as it pains traditional managers, small teams have to be given much more autonomy related to tools and processes and measured on results of efficiency, effectiveness and user satisfaction. Managers need to refocus on providing guidance on overall business results rather than being experts in managing a process.
• Customer experience is more important than development efficiency. It might not be obvious, but the most efficient way to organize for agile development might result in developing solutions that are not optimal for the users and the overall organization. One company I am familiar with initially reorganized IT to align with its internal organizations, including HR, Sales, and Marketing. The result was the very fast development of solutions that met the needs of the individual organizations, but end-users were overwhelmed by overlapping capabilities, a lack of unified authentication, and confusingly diverse user interfaces. In this case, what was missing was a central group responsible for developing common architectures and standards for shared functionality like authentication and User Interfaces.
• Don’t forget employee development. One of the (few) advantages of traditional development cycles and development organizations was a shared experience of climbing the mountain, overcoming obstacles, and reaching a goal. At the end, everyone feels a shared sense of accomplishment and relief. “Let’s hold a party and give out T-Shirts”. 

How do you achieve that same sense of completion and satisfaction when small teams are putting out “products” every week or month? This is a major challenge that should not be underestimated when the organization is being restructured to support the Intelligent Digital Transformation goal. Employees still need to feel part of a bigger whole, get feedback on their performance and be celebrated as important milestones are achieved. This actually gives managers the opportunity to provide employees with a more fulfilling environment because of more individual recognition.

The journey to becoming an Intelligent organization requires the unification of information, the development of strategies and tools to extract intelligence from that information and finally the transformation of the organization to compete in today’s hyper-competitive markets. This transformation is heavily dependent on technology and tools, but it will fail if the organization is not optimized to support that transformation.

Software may be “eating the world” but with the right combination of organizational and technological transformation, your business can avoid indigestion and take a bigger bite out of your market.

Click the link for more information about Huawei’s Intelligent Data Lake solution.

Disclaimer: Any views and/or opinions expressed in this post by individual authors or contributors are their personal views and/or opinions and do not necessarily reflect the views and/or opinions of Huawei Technologies.