news

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

Once upon a time, the UDC underwater data center, like the root server soaked in water in "The Wandering Earth 2", was a "black technology" that only existed in science fiction movies, curious news, or was used by top overseas scientific research institutions to "show off".

Today, UDC is here and is accelerating.

In July this year, I went to the 2024 World Artificial Intelligence Conference, and in the "C" position of a Chinese computing manufacturer's booth, I saw a computer soaked in water and running.

It turns out that the special liquid developed by the company is compatible with circuit boards and will not corrode the circuits. Liquid-cooled servers built with related materials can resist the natural enemy of electronic components - "water", making underwater data centers possible.

According to on-site staff, the company has reached a cooperation agreement with Suzhou and is deploying an underwater data center in the lake. First, it can serve as a benchmark project for the local "new quality productivity" and demonstrate the regional strength in intelligent computing; second, it has a particularly good energy-saving, emission-reduction and consumption-reduction effect, and can provide green computing power for some local AI companies and applications.

So how can we use the computing power under water? The answer is the cloud.

Obtaining AI computing power from the cloud has become the first choice for AI startups due to its natural cost advantages and flexible leasing elasticity.In order to build a more energy-efficient, greener and lower-cost "cloud", UDC underwater data centers have also entered the vision of cloud computing vendors.

In 2014, Microsoft first proposed the concept of underwater data centers, hoping to provide high-speed cloud services to coastal populations. In 2018, Microsoft officially sank 855 servers into the sea to compare with Azure Cloud. my country currently also has submarine data centers, and its core business is computing power services, as well as large-scale direct cooperation with cloud manufacturers such as Tencent and Alibaba to provide them with more cost-effective cloud computing power.

This article will talk about how underwater clouds can “bring down” the price of computing power.

"The end of AI is electricity." The number of parameters in large models has reached trillions, driving the network scale of intelligent computing centers to evolve to 50,000 or even 100,000 cards. A cluster of 100,000 cards may exhaust the electricity of a city. In order to support large AI models, the whole city cannot turn on the air conditioner or watch TV. This is too cyberpunk.

Not only do we not want this to happen, but cloud vendors do too. A survey shows that water and electricity expenses account for more than 50% of data center operating costs. As the density of GPUs in intelligent computing centers increases further, the power load will increase exponentially, directly leading to a surge in costs for cloud vendors.

In order to reduce electricity bills, cloud vendors have tried every possible means.

Some people have placed data centers in Ulanqab and Qinghai, some have placed them by the lake (Alibaba Qiandao Lake Data Center), some have placed them in caves (Tencent and Huawei), some have moved them directly to the Arctic Circle (Facebook Node Pole Data Center, Google Data Center in Hamina Port, Finland), and some have placed them on the sea (Google's floating data center) and on the seabed (Microsoft Project Natick Data Center).

No matter how things change, the essence remains the same, which is to reduce the reliance on air cooling for cooling of electrical equipment such as air conditioners, and rely more on natural cooling through air or water in the natural environment.

Among these natural cooling solutions,The clouds under the water are undoubtedly the most competitive in terms of “power reduction”.

Obviously, compared with land-based data centers, the underwater environment is more complex, wiring and maintenance are more troublesome, and servers are afraid of water, which poses greater challenges to the waterproof and anti-corrosion performance of underwater data centers. Why is it still an option?

The first cut in the cost of underwater data centers has brought the natural cooling effect to the "floor price".

As the most extreme cooling solution, underwater data centers directly utilize the low temperature characteristics of water, absorbing more heat than other substances (such as air cooling and wind cooling), greatly reducing additional cooling needs.

At the same time, since the underwater environment itself provides effective cooling, the dependence on air conditioning systems and energy consumption can be reduced.

It can be said that the cost advantage of underwater data centers in energy saving and consumption reduction is most significant by using water flow to carry away heat.

Building a data center on land will naturally occupy land space. In addition to land costs, some cold inland cities and regions with lower natural gas temperatures generally have less developed infrastructure than coastal areas, such as the Arctic Circle, which leads to an increase in the total cost of ownership (TCO).

Especially with the arrival of super-10,000-card clusters, extremely high requirements are placed on the supporting facilities of data centers/intelligent computing centers on land, such as power supply, load-bearing capacity, computer room cleanliness, and cable rack design.

The "White Paper on New Intelligent Computing Technology for Super 10,000-Card Clusters (2024)" shows that due to the higher computing power density and power consumption density of super 10,000-card clusters, the amount of cables laid also increases. A 18,000-card intelligent computing cluster requires the laying of 100,000 cables, which will bring new challenges to the width and load-bearing capacity of the cable rack.

It can be said that the overall cost of land data centers/intelligent computing centers is rising.

In comparison, the construction cost of underwater data centers is decreasing, and the "scissors gap" between the two is getting smaller and smaller.

On the one hand, due to the larger space of lakes and oceans, builders have a wider range of site selection, and the land cost of underwater data centers is more controllable compared to inland areas.

In addition, underwater data centers often use highly integrated, tightly coupled, and sealed data cabins. Some equipment that is necessary in land-based data centers, such as cooling towers and compressors, is not required underwater. A properly designed submarine data warehouse has a lower failure rate and lower operation and maintenance costs.

In September 2020, Microsoft salvaged a data center that had been submerged in the sea for two years. An assessment found that the server failure rate of the underwater data center was lower than that of traditional data centers, and its reliability was 8 times higher than that of servers on land.

This may be because the underwater data cabin can be isolated from the changing environment and is rarely disturbed by accidents. In short, the vast waters make the price of underwater computing power, which does not need to compete with humans for land resources, more attractive.

Soak the server in water to save water. What is going on?

This brings us to an indicator: WUE (Water Usage Effectiveness), water resource utilization efficiency.

Many people have heard of PUE, but few have heard of WUE. However, if we want to promote environmental sustainability and achieve carbon neutrality, WUE must be reduced.

The key to reducing WUE is to reduce the molecule - the amount of water input into the data center, which mainly consists of production water such as refrigeration replenishment, cooling replenishment, and humidification replenishment.The less water is consumed in the cooling process, the higher the WUE。

Can soaking the data center in water reduce production water consumption? Yes, and even to zero.

On the one hand, underwater data centers directly use the low temperature of the surrounding water to cool the servers, without the need for evaporative heat dissipation, eliminating the cooling towers and cold water systems in traditional data centers, and the water resource consumption for cooling is equal to zero.

In addition, the water evaporated during the cooling process of underwater data centers returns directly to the water body, with almost no loss to public water sources. In some arid and water-scarce areas, if lakes, rivers, and groundwater are used for cooling, the evaporated water resources may not be able to return quickly to the local water system, which will have a negative impact on the local water environment.

In 2023, Arizona, USA, restricted the construction of local data centers due to concerns about water resource consumption. In China, Jining District, Ulanqab City, Inner Mongolia, also issued a "Notice on Prohibiting Jining District Big Data Enterprises from Using Groundwater for Cooling", prohibiting all big data enterprises in the area from using groundwater for cooling.

Even if underwater data centers generate some heat evaporation, it will quickly be metabolized by the vast lakes and oceans and will not affect the local water cycle.

Although underwater data centers have great cost advantages, if AI cloud services only reduce costs, the "cost reduction and profit increase" incident may occur.

For example, the storage and computing costs of data centers in the west are lower than those in the east, but some eastern companies would rather bear 50% more costs and store their data in the east, considering transmission delay, packet loss rate, business reliability, etc., especially for businesses such as autonomous driving that require high data real-time performance. This is also an important reason why many western data centers have low shelf rates.

The commercialization dilemma of underwater data centers is that, although the price is low, they are not fully adapted to the needs of AI, and AI is regarded by cloud vendors as the most critical new revenue point at present.

Underwater resources have limited scalability.Cloud services need to quickly expand resources based on user needs and increase computing power and storage space. The underwater data cabin takes a certain amount of time to leave the factory and launch (Microsoft's official website shows that it takes 90 days for Natick to go from the factory to operation). The current intelligent computing center on land uses modular construction and can be completed in about a month, which can meet AI computing power needs more quickly.

At present, Microsoft has also announced the suspension of the Natick project. It is said that it will strengthen research on installation and maintenance robots in the later stage. It seems that the underwater environment far away from human activities is still not suitable for large-scale deployment of data centers.

In addition, the management and administration of underwater computing power also poses challenges to cloud vendors.At present, the proportion of underwater computing power is still relatively small. Cloud vendors need to integrate and manage these heterogeneous resources, ensure that computing power from different sources can be seamlessly integrated to achieve optimal performance, and allow users to use these resources conveniently and simply. This requires investing a lot of resources in the research and development of technologies, systems, and platforms.

Some small and medium-sized cloud vendors that do not have sufficient accumulation may find that "the soy sauce (management platform) is more expensive than the chicken (computing power)", and cannot rely on cost advantages to improve their overall competitiveness.

Xu Zhimo once wrote in his poem "Occasionally": "I am a cloud in the sky, occasionally projected on the center of your waves." At present, AI cloud services and underwater data centers may not be able to stay together and become each other's only one.

In May 2024, the second phase of the Natick project has begun, reflecting Microsoft's continued pursuit of cloud data center solutions. It also makes "underwater data centers" a track that China cannot fall too far behind. It is more suitable as a business card for technological self-reliance and technological power, and is explored in pilot projects together with state-owned cloud, central state-owned enterprise cloud, and scientific research cloud.

Cloud service providers with stronger commercial attributes still need to focus on land. Fortunately, China has rich topography and perfect infrastructure, and there is a broad space to explore the "ultimate cost-effectiveness".

From this perspective, computing power has never been the business of a single computing company or industry, but a systemic project involving the whole world. The era of computing power being national strength has just begun.