news

Huawei MetaERP's road to breakthrough: a heroic epic for mortals

2024-07-17

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina



On July 10, 2022, in a building under construction by Huawei in Songshan Lake, Dongguan, a somewhat lost Dong Xiangzui cast his gaze toward the mirror-like artificial lake outside the floor-to-ceiling glass windows of the office area.


After dinner, people walked around the lake in groups of two or three, chatting and laughing. The road outside the park had already started to become congested as usual, with bright red car taillights forming a long line.

It seemed to be an ordinary summer evening, and everything was the same as usual. But on this ordinary summer day, Huawei won a secret war that had lasted for three years:

In May 2019, after the sanctions by the U.S. Department of Commerce took effect, the Oracle ERP system that Huawei had used was suspended and supply was cut off. The system involved almost all of Huawei's businesses in 170 countries or regions around the world, and has since become a sword of Damocles hanging over the company's head.

Faced with almost no way out, Huawei decided to take a gamble and develop a full-stack self-developed ERP system to completely replace the Oracle ERP system that the company had used for more than 20 years.

This system is MetaERP.

July 10, 2022, this seemingly ordinary summer day, is the day when the MetaERP system was launched in China.


Dong Xiangzui is the "last insurance" in all solutions: she is responsible for the rollback plan for the system launch. If there are unsolvable problems when the system is switched to MetaERP, it is necessary to roll back to the old system through Dong Xiangzui's solution to minimize the impact on the business.

But that day, until the entire switching process was completely completed, the more than ten business experts who were originally scheduled to be on duty on site did not receive a single business question.

Dong Xiangzui, who seemed a little lost in the celebrating crowd, also felt relieved.

In less than three years, developing such a large and incredibly complex high-end ERP system is a field that no Chinese manufacturer has ever set foot in. From the day when Huawei decided to give it a try, it had already entered a kind of no man's land, and every step since then has been a pioneering effort.


The hidden worries revealed by Huawei’s predicament

On the afternoon of May 16, 2019, Huawei's then-CIO Tao Jingwen saw off the regional head of Oracle from the conference room. More than 30 on-site maintenance experts of Oracle's ERP system subsequently evacuated from Huawei. The usually crowded office area suddenly became a little empty, with only a few Huawei employees standing in the office, their eyes full of confusion and helplessness.

For quite a long time in the past, the Chinese people did not have a deep sense of being stuck in large-scale enterprise management systems and industrial software such as ERP. One of the reasons is that the market share of domestic ERP systems in small and medium-sized enterprises is not low. UFIDA alone can account for about 30% of the market share, and the localization rate has long exceeded 50%.

However, among large Chinese companies, high-end ERP systems are still almost monopolized by overseas companies. By 2023, the combined market share of ASP and Oracle, two overseas giants, will still be as high as 53% among large companies. (Data from the China Economic Industry Research Institute report)

The difficulties encountered by Huawei have completely exposed this hidden worry:There are still many technical barriers that Chinese domestic ERP systems have not been able to break through in the high-end field. The larger the enterprise and the more high-end the industry, the easier it is to be strangled by large management systems. Even some small and medium-sized enterprises, after developing to a certain extent, have begun to migrate from domestic systems to overseas systems due to growing demand. In the later stages of enterprise development, there will even be a phenomenon of "reverse localization" in the selection of ERP systems.


Results of the questionnaire survey of CAICT's "China ERP Value Innovation Research Report"

The situation is even more complicated in large enterprises whose business systems are already extremely complex and large.

The construction of digital systems for many large Chinese enterprises began around 1997, and Huawei is one of the most representative cases.

At that time, China was still very weak in both high-end manufacturing and information engineering industries that could develop supporting systems for manufacturing. It had no ability to build such a large software system as the ERP system. Therefore, purchasing overseas systems was both a last resort and a necessary path for China to grow and mature rapidly by standing on the shoulders of giants.

However, core systems like ERP that maintain the operation of the enterprise are difficult to replace once they are used.

Take Huawei for example. Since it introduced Oracle's ERP system in 1996, Huawei has used this system for more than 20 years and has accumulated 150T of data. The system contains the fulfillment of 5 million contracts worldwide, the delivery of more than 60,000 projects, more than 700 million purchase orders, more than 2.4 billion inventory transactions, more than 3.3 billion delivery orders, more than 1.7 billion invoices payable, and more than 1.5 billion invoices receivable...


During these 20 years, even Huawei's own customized development of the ERP system has been astronomical, with a staggering 4.9 million lines of code, more than 300 external operating systems, 3,950 business connection points, and more than 27,000 data imports and exports.

In 2019, Huawei sold 240 million mobile phones alone. The completion and roll-off of every mobile phone, the fulfillment of every purchase contract, and every inventory shipment and receipt must be recorded in the ERP system.

This system is almost the entire nervous system of the Huawei giant. To replace it, not only does it require building a complete and usable new system, but just separating the intricate old system from the original business is an extremely complex project.

This is why there are many large companies in the market that want to develop their own replacements for their ERP systems, but so far few have actually succeeded in doing so.

However, there is not much time left for Huawei.

Even an ordinary family car needs to be regularly serviced at a 4S shop. However, the ERP system that keeps Huawei, a giant international enterprise, running suddenly lost almost all the maintenance and official guarantee of its software and hardware. As a result, Huawei's internal ERP project was divided into two plans that were carried out simultaneously:

On the one hand, they found the elites who knew the original ERP system best, and used Huawei's own methods to maintain the old system first to ensure that this old car that lost official maintenance would not fall apart.

On the other hand, we will gather all our resources to independently develop a new Huawei ERP system to completely replace the old system in the future.


A heroic journey driven by the great times

In the eyes of many people, developing a new system and completely replacing the old ERP system before it becomes ineffective is an impossible task.

But Huawei did it.

Before being pushed to that point by force majeure, Huawei itself actually didn’t dare to think about it. But sometimes, ordinary heroes are pushed onto the stage of history by the great times.

I'm afraid the people who issued the sanctions order never imagined that it was this sudden sanction that prompted Huawei to break free from its shackles and do things that would have been impossible under normal circumstances.

Take the gathering of talent needed for the ERP system, for example. Under normal circumstances, this is an insurmountable problem.

If any large company wants to make system changes, there will be a lot of interest games and power and responsibility tug-of-war. Huawei was no exception in the past.

For an IT system like the ERP system that is closely integrated with the business, in order to clarify the logic and determine the final architecture, it is necessary to have very experienced business backbones with a global perspective to cooperate in understanding and design.

In the past, the IT department had to use coercion, threats and inducements to get the already short-staffed business departments to invest resources in system changes. However, after the ERP project was launched, almost all departments in the company, from finance and supply chain to procurement and service delivery, allocated resources without reservation to give priority to the project team.

Finance, supply, procurement, service delivery... Each field has a group of key business personnel with more than ten years of experience who are transferred to the ERP project team to help sort out these complex business rules and sort out the complicated business scenarios.

After deciding to build MetaERP on two fronts in parallel, namely maintenance and research and development, Huawei began to issue mysterious calls for action.

Under the calm surface, tens of thousands of employees around the world were quietly mobilized. Many were summoned to Shenzhen in the name of a confidential project. They met in a closed conference room, signed a confidentiality agreement, and began their three-year ERP project journey.

At that time, Huawei had already frozen social recruitment. During a communication with his superiors, CIO Tao Jingwen tore a page from his notebook and wrote an application report on the spot: "I am applying to add 100 high-end and special talents to win the battle of IT business continuity and security protection."

Then the leader approved 300 recruitment quotas for him on the spot. The human resources department of process IT specially formed a team for this purpose, which browsed more than 50,000 resumes and contacted more than 10,000 people.

The sense of crisis and mission brought about by the moment of life and death allowed Huawei to complete this assembly in just a few months with the help of almost the entire industry. It set up 12 sub-project teams and invested 1,800 people. At the peak, the team had more than 3,000 people, and began this heroic journey for every mortal in the project team.

In addition to internal support, there is also a lot of support from outside.

Several consulting companies that have long cooperated with Huawei helped Huawei find nearly 20 top consultants from all over the world to join in a short period of time. Software partners have tapped resources through various channels and transferred them to the ERP project.

Before the system switch, Huawei hoped to bring forward the production of urgent orders from customers to June in order to control risks to the greatest extent possible, and postpone non-urgent orders until after the system switch is completed. The key component suppliers involved are also trying their best to cooperate.

This is completely counter-intuitive to assembly speed and project efficiency.Under the crisis of sanctions, the weapon that was originally thought to be able to kill Huawei has become a huge opportunity imbued with a sense of destiny in the great era.

The traditional ERP system began in the 1960s and was built on two main foundations: one was the huge and mature industrial system accumulated over a long period of time in the West; the other was the very strong software and hardware capabilities of the information infrastructure, which already had the information technology capabilities to build a supporting system.

However, with the development of cloud computing technology and the rise of intelligence, the fourth industrial revolution of mankind, the old ERP architecture is actually already a patched-up old car. It has been several decades since the original system was invented, and it is no longer the optimal solution that can be achieved by the technology of this era.

However, in the past, few people tried to break through this barrier because the project was too huge and the difficulty was too high. However, the emergence of the crisis gave Huawei an opportunity to rebuild the new era ERP system from the bottom up:

They are not planning to just "escape" or take "conservative treatment" and do not want to simply copy the product architecture of Western manufacturers and make a replacement for Oracle ERP.

They want to take advantage of this gathering that will be difficult to replicate, and in the next three years, stand at the forefront of the digital and intelligent era, build a system with a more advanced and cutting-edge underlying architecture, and create a truly cloud-based, service-oriented, and autonomously controllable core business system!


ERP value innovation is an inherent need for high-quality development of enterprises. Chinese ERP is rising and gradually becoming the first choice.

They want to redefine ERP for this era in the era of intelligent empowerment of industry.


Pioneers of No Man's Land

To enter uncharted territory, to be a pioneer, and to do what others cannot do - after setting such a goal, almost every step Huawei takes is a pioneering effort.

The first MetaERP system switchover trial was conducted in a subsidiary in Malaysia, mainly to verify whether the system could operate. However, the business of the subsidiary in Malaysia only accounted for less than 1% of Huawei's overall business, while the business in China accounted for 70% of the global business. To achieve the launch in China, Huawei must achieve a huge breakthrough in software and hardware performance.

Among them, three technical challenges have become the top priority:

The first one is the technical architecture of metadata multi-tenancy. This is the first time that this technology has been used in an ERP system, which will bring MetaERP much higher scalability than the old system.

The second technical challenge is cloud-native technology. This technology requires not only software support but also hardware support. It means that the data base of MetaERP will no longer be fixed physical hardware, but will call on the computing power and resource elasticity of the cloud platform, which will bring powerful computing power and flexibility to MetaERP.

The third technical challenge is the breakthrough in database root technology.

Huawei has always had its own Gaussian database product, but its positioning has always been as a "technical reserve". In 2019, when the ERP replacement project was just launched, the project team tested the Gaussian database and concluded that it "could not meet the requirements". But in September 2021, when the project team was planning the switching plan for the China region, they found that there was no open source database on the market that could meet the requirements, so they turned their attention back to their own Gaussian.

The database joint research team signed a military order and increased the single capacity of the Gaussian database by 6 times in 8 months, from 4T to 24T; the availability increased from 99.99% to 99.999%; in terms of system resilience, the performance did not drop under 5 times the pressure, and did not crash under 10 times the pressure; the logical replication speed increased by 6 times, reaching 300MBa per second...

The ERP system established under the Western system represents the excellent management experience and best case practices of the Western business community.

But MetaERP is built on the foundation of China's industrial development and is the best case practice created by integrating all of Huawei's transformation and successful experiences over the past two decades.


The sanctions were originally intended to restrict Huawei, but they pushed Huawei to break free from the shackles of the old system and move towards greater freedom. It forced Huawei to throw away its burdens and directly base itself on the development of China's industry and the progress of contemporary technology, and reconstructed a more future-oriented ERP system that belongs to the intelligent era from the bottom up.


Conclusion

The Huawei Park in Songshan Lake High-tech Industrial Development Zone in Dongguan is called "European Town" by many people. But its real name is "Xiliu Beipo Village", which has a strong Chinese rural style.


On April 20, 2023, the MetaERP Commendation and Pledge Conference was held here. The conference awarded medals to thousands of key contributors, and there was a special session to award partners. Many external software partners are doing their best to help Huawei survive the ERP battle royale, and Huawei has lived up to it.


Huawei awards MetaERP partners

Three years ago, Tao Jingwen took the handwritten application report torn from the notebook and exchanged it for 300 special recruitment quotas approved by the company for the process IT department. On the day when MetaERP was successful, the project team brought 300 outstanding ERP project managers and technical backbones to the Chinese software industry. These people will become the backbone of Huawei's rise in the software field.


Tao Jingwen, Director of Huawei and President of Quality and Process IT Department, delivered a speech

The success of MetaERP is not just a victory for Huawei alone, but a common victory for China's entire software industry.

Even though MetaERP has not been put into commercial use today, Huawei's process of breaking through the blockade has set an example for large Chinese companies. All those who have experienced and witnessed this breakthrough,All the technical personnel who have worked hard for this have become the spark of China's ERP software industry, and even the spark of China's industrial software industry.


In recent years, when Huawei is mentioned, it seems that people always think of some grand narratives. But when reading the materials about MetaERP's breakthrough, what touched me the most was always the "ordinary people" in the project team. The ERP project manager was Zhang Xiaoyan. At the commendation meeting after the successful launch of MetaERP in Malaysia, she led everyone to recite a collectively created poem. The first sentence said:

Never thought about it

An ordinary person like me

Can also write a hearty story

But ordinary people writing exciting stories and embarking on a heroic journey that belongs to mortals are precisely the most common background in the story of China's rise.

May a single spark start a prairie fire.

The information in this article comes from the book "Journey Under the Starlight", which is attached to "Quality as the Guideline" in Huawei's four-part management series.

This book extracts original speeches and related documents from senior executives, interviews heavyweight executives, and includes practical cases of front-line employees. It restores the entire process of how Huawei broke through US sanctions, created the MateERP replacement miracle, and fought a good battle to defend quality. It is a comprehensive presentation of Huawei's more than 30 years of internal quality management experience and is well worth reading.


····· End ·····

Research on the Xinghai Intelligence Bureau System

Made in China andProduction Substitution


Join the media platform

36Kr/ Tiger Sniff/ Guanchazhe.com News Community/ NetEase News/

Himalaya/Xueqiu/Tencent News/Phoenix News