news

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

Text: Semiconductor Industry Overview

​Industries related to "computing power" have continued to be popular recently, and the construction of intelligent computing centers is also flourishing everywhere.

In 2024, the Wuchang Intelligent Computing Center, China Mobile Intelligent Computing Center (Qingdao), South China Digital Valley Intelligent Computing Center, Zhengzhou Artificial Intelligence Computing Center, and Broad Data Shenzhen Qianhai Intelligent Computing Center have been started or put into operation one after another.

According to incomplete statistics, there are currently more than 30 cities across the country that are building or proposing to build intelligent computing centers, with an investment scale of over 10 billion yuan.

What exactly is an intelligent computing center? What is an intelligent computing center mainly used for? What are the characteristics of an intelligent computing center?

What is an intelligent computing center?

According to the definition of the Action Plan for High-Quality Development of Computing Infrastructure, an intelligent computing center refers to a facility that provides the required computing power, data, and algorithms for artificial intelligence applications (such as artificial intelligence deep learning model development, model training, and model reasoning) by using large-scale heterogeneous computing resources, including general computing power (CPU) and intelligent computing power (GPU, FPGA, ASIC, etc.).

It can also be said that the intelligent computing center is a data center that focuses on artificial intelligence computing tasks.

Data centers generally include three categories. In addition to intelligent computing centers, the other two are general computing centers that focus on general computing tasks, and supercomputing centers that focus on supercomputing tasks.

2023 is an important turning point in the development of artificial intelligence. AIGC technology has made breakthrough progress. New businesses such as large-model training and large-model applications are rapidly emerging. As a carrier of intelligent computing power, data centers have also developed from data rooms and general computing centers to the current supercomputing centers and intelligent computing centers.

What is the difference between an intelligent computing center and a general data center?

Intelligent computing centers are often closely related to cloud computing, emphasizing resource control and flexibility in infrastructure management. In a cloud environment, data center providers are responsible for the maintenance of hardware and certain software tools, while customers own the data. In contrast, traditional local data centers require enterprises to manage and maintain all data resources themselves.

The essential difference leads to huge differences between the two models in terms of capital investment, resource deployment and security.

In terms of capital investment, customers of intelligent computing centers can choose a service model that suits them, such as public cloud, private cloud or hybrid cloud, without spending a lot of hardware and software costs; while customers of traditional data centers need to invest a lot of money to purchase and maintain the servers, networks and storage devices they need.

In terms of resource deployment and security, customers of the intelligent computing center can remotely access and manage their data and applications through the Internet anytime and anywhere. At the same time, they can also enjoy the professional security protection provided by the data center provider, such as firewalls, encryption, backup and recovery, etc.; while customers of traditional data centers are restricted to offices/designated locations and need to protect and manage their data themselves.

In simple terms, the Intelligent Computing Center is a data computing center dedicated to artificial intelligence, which can provide the dedicated computing power required for artificial intelligence computing. Compared with traditional data centers, the Intelligent Computing Center can meet more targeted needs, as well as larger computing volume and faster computing speed, and provide AI computing power for vertical industry scenarios such as large model training and reasoning, autonomous driving, and AIGC.

What kind of chip is needed for AI intelligent computing?

In terms of hardware selection, the hardware architecture of the intelligent computing center is also different from that of the traditional data center.

What kind of computing power chip is needed for AI intelligent computing?

The hardware architecture of traditional data centers is relatively simple, mainly including servers, storage devices and network devices. Compared with this hardware architecture, the intelligent computing center will be more flexible, and different application scenarios will also choose different computing nodes.

Intelligent computing servers are the main computing hardware of intelligent computing centers. They usually adopt heterogeneous computing architectures of "CPU+GPU", "CPU+NPU" or "CPU+TPU" to give full play to the advantages of different computing chips in performance, cost and energy consumption.

GPU, NPU, and TPU have many cores and are good at parallel computing. AI algorithms involve a large number of simple matrix operation tasks, which require powerful parallel computing capabilities.

Traditional general-purpose servers use CPU as the main chip to support basic general-purpose computing such as cloud computing and edge computing.

What kind of storage chip is needed for AI intelligent computing?

Not only are the computing chips different, AI computing also has higher requirements for storage chips.

The first is the usage. The DRAM capacity of an intelligent computing server is usually 8 times that of an ordinary server, and the NAND capacity is 3 times that of an ordinary server. Even its PCB circuit board layers are significantly more than those of a traditional server.

This also means that intelligent computing servers need to be equipped with more storage chips to achieve the required performance.

As demand continues to rise, a series of bottleneck problems have also surfaced.

On the one hand, the traditional von Neumann architecture requires that data must be loaded into memory, resulting in low data processing efficiency, large latency, and high power consumption; on the other hand, the memory wall problem causes the processor performance to grow much faster than the memory speed, resulting in a large amount of data needing to be transferred between the SSD and the memory; in addition, the capacity and bandwidth limitations of the SSD mounted on the CPU also become performance bottlenecks.

Faced with problems such as the "storage wall" and the "power consumption wall", the computing storage architecture in the traditional computing system architecture urgently needs to be upgraded to organically integrate storage and computing, and to use its huge energy efficiency improvement potential to match the massive data storage needs in the era of intelligent computing.

For this series of problems, storage and computing integrated chips may be a good answer.

In addition to the different chips, in order to fully exert performance and ensure stable operation, the AI ​​server has also been enhanced in terms of architecture, heat dissipation, topology, etc.

Who is planning to design these chips?

Layout of computing chips

In terms of GPU, GPU is good at large-scale parallel computing. Huawei, Tianshu Zhixin, Moore Thread, Sugon, Suiyuan Technology, NVIDIA, Intel, AMD, etc. have launched related chips. For example, Huawei launched the Ascend series AI chips Ascend 910 and Ascend 310, etc. These chips are designed for AI training and reasoning, with high performance and low power consumption. The Ascend series has been widely used in data centers, cloud services, edge computing and other fields, providing powerful computing support for intelligent computing centers.

Nvidia has launched a number of GPU products for AI training and reasoning, such as A100 and H100. Intel has also launched a number of AI chip products, such as Habana Labs' Gaudi series chips, aiming to compete with Nvidia. AMD has also made some arrangements in the field of AI chips, launching the MI series of GPU and APU products.

In terms of FPGA, CPU+FPGA combines flexibility and high performance to adapt to rapid changes in algorithms. Xilinx and Intel are major market participants, and related products include: Xilinx's VIRTEX, KINTEX, ARTIX, SPARTAN product series and Intel's Agilex product series; major domestic manufacturers include Fudan Microelectronics, Unigroup Guoxin and Anlu Technology.

In terms of ASIC, CPU+ASIC provides high-performance customized computing suitable for specific needs. Foreign giants such as Google, Intel, and Nvidia have successively released ASIC chips. Domestic manufacturers such as Cambrian, Huawei HiSilicon, and Horizon Robotics have also launched ASIC chips for deep neural network acceleration.

In terms of NPU, NPU is a processor designed specifically for artificial intelligence and machine learning scenarios. Unlike CPU and GPU, NPU has been optimized in hardware structure and focuses on performing AI-related computing tasks such as neural network reasoning. The combination of the versatility of CPU and the specialization of NPU enables the entire system to flexibly respond to various AI application scenarios and quickly adapt to changes in algorithms and models.

There are many mass-produced NPUs or chips equipped with NPU modules on the market, including the well-known Qualcomm Hexagon NPU and Huawei's Ascend series. It is worth noting that each major manufacturer has a unique strategy in the design of chip computing cores.

In terms of TPU, TPU is a chip developed by Google specifically to accelerate the computing power of deep neural networks. It is more focused on processing large-scale deep learning tasks, with higher computing power and lower latency. TPU is also a type of ASIC chip.

In terms of DPU, DPU is specially designed for data processing tasks, with a highly optimized hardware structure, suitable for computing needs in specific fields. Unlike CPU for general computing and GPU for accelerated computing, DPU is the third main chip in the data center. The DPU products of the three major international giants NVIDIA, Broadcom, and Intel occupy most of the domestic market. Xilinx, Marvell, Pensando, Fungible, Amazon, Microsoft and many other manufacturers have also produced DPU or similar architecture products in the past 2-5 years. Domestic manufacturers include Zhongke Yusu, Xinqiyuan, Yunbao Intelligence, Dayu Zhixin, Alibaba Cloud, etc.

How far have domestic computing chips come?

At the 2024 Beijing Mobile Computing Network Conference, the Beijing node of China Mobile Computing Center was officially put into use, marking a new stage in the construction of my country's intelligent computing center. As Beijing's first large-scale training-push integrated intelligent computing center, the project covers an area of ​​about 57,000 square meters, deploys nearly 4,000 AI acceleration cards, and the localization rate of AI chips reaches 33%, with an intelligent computing power scale of over 1000P.

Zhen Yanan, CTO of Beijing Beilong Super Cloud Computing Co., Ltd., the operating entity of Beijing Super Cloud Computing Center, recently said that it only takes about 15 days to "graft" domestic large models with domestic chips. He believes that computing power sharing will be a major trend in the industry, and high-end GPU computing power resources require efforts from all parties.

In recent years, the market structure of China's artificial intelligence computing chips has been mainly dominated by NVIDIA, which has occupied more than 80% of the market share.

Zhen Yanan said, "We are also very concerned about the development of domestic chips. It is understood that domestically developed large models and even some open source large models are constantly being transplanted to domestic chips. From the perspective of chip use, some models can already run smoothly, and the areas that need to catch up are mainly high performance such as GPUs."

"The entire localization process is layered. Chips belong to the hardware layer, and in addition there is the software ecosystem. For domestic chips, both the framework and the ecosystem require a certain cultivation cycle." Zhen Yanan called on the final users to give enough confidence in domestic chips.

Layout of memory chips

The storage of intelligent computing centers needs to have high capacity, high reliability, and high availability. Storage devices usually use high-performance hard disks or solid-state drives and are equipped with redundant storage architectures to ensure data security and accessibility. Samsung, Micron, SK Hynix, etc. have related chips that are widely used in data centers, cloud computing and other fields, providing high-performance storage solutions for intelligent computing centers.

In recent years, domestic manufacturers have also achieved rapid development in catching up with DRAM and NAND technologies.

In addition to traditional storage chips, the intelligent computing center also needs the new storage-storage and computing integrated chips mentioned above to play a greater role.

From the perspective of the development history of integrated storage and computing, since 2017, major manufacturers such as NVIDIA, Microsoft, and Samsung have proposed integrated storage and computing prototypes. In the same year, domestic integrated storage and computing chip companies began to emerge.

The demand of large companies for storage-computing integrated architecture is practical and can be implemented quickly. As the technology closest to engineering implementation, near-memory computing has become the first choice of large companies. Large companies with rich ecosystems such as Tesla and Samsung, as well as traditional chip manufacturers such as Intel and IBM, are all deploying near-memory computing.

Domestic start-ups focus on in-memory computing without considering advanced process technology. Among them, start-ups such as Zhicun Technology, Yizhu Technology, and Jiutian Ruixin are betting on PIM, CIM and other storage and computing technology routes that are closer to "storage" and "computing". Yizhu Technology and Qianxin Technology focus on AI high-computing scenarios such as large model computing and autonomous driving; Shanyi, Xinyi Technology, Pingxin Technology, Zhicun Technology, etc. focus on edge small-computing scenarios such as the Internet of Things, wearable devices, and smart homes.

Yizhu Technology is committed to designing AI high-computing power chips with a storage-computing integrated architecture. It combines memristor ReRAM and the storage-computing integrated architecture for the first time. Through a fully digital chip design approach, based on the current industrial structure, it provides a new development path for AI high-computing power chips that is more cost-effective, has higher energy efficiency, and has greater computing power development space.

Qianxin Technology focuses on the research and development of high-computing storage-computing integrated computing chips and computing solutions for the fields of artificial intelligence and scientific computing. In 2019, it took the lead in proposing a reconfigurable storage-computing integrated technology product architecture, which can increase computing throughput by 10-40 times compared to traditional AI chips. Currently, Qianxin Technology's reconfigurable storage-computing integrated chip (prototype) has been tried or implemented in cloud computing, autonomous driving perception, image classification, license plate recognition and other fields; its high-computing storage-computing integrated chip product prototype has also been the first in China to pass the internal test of major Internet companies.

Zhicun Technology's solution is to redesign the memory, utilize the physical characteristics of Flash memory storage units, transform the storage array and redesign the peripheral circuits so that it can accommodate more data, and at the same time store the operators in the memory so that each unit can perform simulated calculations and directly output the calculation results, so as to achieve the purpose of storage and computing integration.

The scale of intelligent computing accounts for more than 30%, and computing power construction is in full swing

In early July, the Tianfu Intelligent Computing Southwest Computing Center was officially put into operation in Chengdu, Sichuan. According to reports, the center will use computing power to support Chengdu in building a core industry of artificial intelligence worth hundreds of billions of yuan, and enable artificial intelligence innovation in the fields of industrial manufacturing, natural sciences, biomedicine, and scientific research simulation experiments.

This is not an isolated case. In the past month, Yinchuan Green Intelligent Computing Center projects have been launched; Beijing Mobile has built the first large-scale integrated training and push intelligent computing center in Beijing to support the training and reasoning of large models with high complexity and high computing requirements of tens of billions and hundreds of billions; Zhengzhou Artificial Intelligence Computing Center has started construction with a total investment of over 1.6 billion yuan... New digital infrastructure represented by intelligent computing centers is accelerating its construction and implementation.

Data released by the National Bureau of Statistics on July 15 showed that as of the end of May, 460,000 new 5G base stations had been built across the country; more than 10 intelligent computing centers with high-performance computer clusters were planned, and intelligent computing power accounted for more than 30% of the total computing power.

According to incomplete statistics from the China IDC Circle, as of May 23, 2024, there are 283 intelligent computing centers in mainland China, covering all provinces, autonomous regions and municipalities in mainland China. Among them, there are 140 intelligent computing center projects with investment statistics, with a total investment of 436.434 billion yuan. There are 177 intelligent computing center projects with planned computing power scale statistics, with a total computing power scale of 369,300 PFlops.

These "intelligent computing centers" have different standards and scales. The computing power scale is generally 50P, 100P, 500P, 1000P, and some even reach 12000P or more. Although the AI ​​wave has brought broad development prospects to intelligent computing centers, supply and demand mismatch, high prices, and duplicate construction are still the problems facing my country's computing power construction.

At the same time, many places have also issued special plans to clarify the construction goals for the next few years and improve support measures in terms of technology, application, funding, etc. For example, Jiangsu has issued a special plan for the development of provincial computing power infrastructure, proposing that by 2030, the total computing power in use in the province will exceed 50 EFLOPS (EFLOPS refers to 100 trillion floating-point operations per second), and the proportion of intelligent computing power will exceed 45%; Gansu has proposed to provide policy support for the new computing power network infrastructure in terms of land use, municipal supporting facilities construction, talent introduction, and funding.

"The explosive development of applications such as artificial intelligence big models has led to a surge in demand for intelligent computing power." Shan Zhiguang, director of the Information Technology and Industry Development Department of the National Information Center, said that intelligent computing has developed rapidly and has become the fastest growing type in my country's computing power structure. Among them, big models are the largest demander of intelligent computing power, accounting for nearly 60% of the demand. It is expected that by 2027, the annual compound growth rate of China's intelligent computing power will reach 33.9%.