news

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

By Li Anqi

Editor: Li Qin

“Either embrace end-to-end or leave the intelligent driving industry in a few years.”

After working in the intelligent driving industry for several years, Qin Feng (pseudonym), an intelligent driving engineer, had originally adapted to the fierce internal rhythm of the intelligent driving industry. But when the new technology "end-to-end big model" came, he felt that the first person to be impacted might not be human drivers, but himself as an engineer.

Qin Feng is not the only one who has this anxiety. Many intelligent driving engineers told 36Kr that in order to learn new technologies, they read the latest industry papers and attended classes on Bilibili during their spare time at work, and some even started learning from graduate textbooks.

"End-to-end big model" is the latest technological bomb in the intelligent driving industry this year.

In January this year, Tesla officially released the test version of FSD (autonomous driving software package) v12 to ordinary users. This version adopts an end-to-end network solution, and many users say that the effect experience is much more human than the previous v11 version.

Musk once introduced the end-to-end capability, calling it "image input, control output". Although many industry insiders told 36Kr that they did not believe that Tesla's end-to-end solution was so radical and amazing in practice, the end-to-end solution still drives domestic peers like honey. Domestic players gradually believe that driven by large models, large computing power, and massive data, AI systems will drive like humans.

Sensing the new technology trend, domestic automakers and leading intelligent driving companies have already taken action. Leading players such as Huawei, Wei, Xiaoli, and BYD have invested teams and resources to promote end-to-end solutions; Ideal and Weilai, two automakers, have also established dedicated end-to-end large model departments to promote technology implementation faster.

The competition for high-end talents is also surging in the light and dark. When the first car was launched on the market, Xiaomi Motors recruited Wang Naiyan, the former CTO of Tucson China, to catch up with the progress of intelligent driving. Some people in the intelligent driving industry told 36Kr that Huawei's intelligent driving even uses relevant patents to anchor talents and conduct targeted mining.

The new solution is indeed igniting the domestic market. But the other side of the coin is that the end-to-end solution relies heavily on data-driven rather than a large number of engineers. Tesla's team size of about 300 people is used as a model to spread among the top players.

However, the reality of the industry is that the current intelligent driving teams of the top players have a talent scale of nearly 1,000 people. BYD, a veteran automaker catching up with the intelligent driving industry, has a software team of 3,000 people, and Huawei's intelligent driving team is not far behind. When the market is good, engineers can generally get a salary package of one million yuan per year.

However, many intelligent driving engineers believe that if the effectiveness of the end-to-end solution is further verified, layoffs in the intelligent driving teams of auto companies will be a high probability event.

"200-300 people are not enough." A former intelligent driving backbone of a new power car company firmly told 36Kr. Even fresh graduates with a deep learning background may have more advantages than some intelligent driving engineers in entering end-to-end projects.

Some intelligent driving headhunters have also clearly felt that the industry is overflowing with talent: car companies' intelligent driving teams are no longer releasing new positions, and staff are beginning to be streamlined. "Many of the HCs that are still online are zombie positions." In the latest update from a headhunter, he has switched tracks and turned to recruiting talent for robotics companies.

Engineers blocked outside the door

Intelligent driving engineer Tian Wei (pseudonym) told 36Kr that in this new technological revolution, engineers working on the planning and control modules will be more impacted than those working on the perception and prediction modules.

This is mainly because the end-to-end solution is significantly different from the traditional intelligent driving solution. The traditional solution is divided into multiple modules such as perception, positioning, mapping, prediction, planning and control, and the module function implementation is basically driven by the engineer's code. The personnel of the two major departments of perception and planning and control often account for the majority of the number of people in the intelligent driving team.

However, the characteristic of the end-to-end solution is that it has changed from being driven by engineers’ code to being driven by data. The most ideal way is to input images into the system, and the system can directly output the vehicle control, and the intermediate links are completed by the AI ​​neural network.

Judging from the progress of domestic leading players, after the introduction of end-to-end solutions, multiple modules of traditional solutions are being transformed through AI neural networks and integrated into two large networks: the perception model and the prediction and decision model. "Many current solutions are based on the perception model and connected to a prediction and planning model."

A further solution will integrate perception, prediction, decision-making and planning into one, which the industry calls “One Model”.

The new technology route has also created a new talent profile for automobile companies’ intelligent driving teams.

An intelligent driving expert told 36Kr that the number of people needed for the end-to-end team has decreased, but the talent threshold has become higher. The large model itself requires the team to have a strong deep learning background. "In the solution building stage, strong infra (infrastructure) talents are needed. They have a deep understanding of each module of perception, planning and control, and understand the support strength of different chip computing platforms, different AI reasoning frameworks, etc."

However, only a small number of people are responsible for model building and training. “Perhaps 90% of the team is providing end-to-end data and data closed-loop tool chain support.”

"The big model itself is a very capable team." said an intelligent driving person. This is also the reason why AI technology companies such as OpenAI, which only had 200-300 people in the early days, were able to launch the large language model ChatGPT and change the global AI process.

For engineers, the impact of end-to-end technology varies.

An intelligent driving expert told 36Kr that in the two modules of perception and regulation and control, the perception model originally relied on deep learning technology. Although the visual detection route has shifted from the previous CNN convolutional neural network to the Transformer-based BEV, the impact on engineers is not significant.

But for regulatory control engineers, if they want to join the end-to-end, it is almost like switching tracks again. An intelligent driving person told 36Kr that traditional regulatory control engineers mainly have several directions: path prediction, path optimization, rule post-processing, and vehicle control. "They are all very niche disciplines and basically unrelated. Except for the path prediction module, engineers in other directions basically have no deep learning background."

Tian Wei, an intelligent driving engineer, told 36Kr that if people in regulatory control want to switch to end-to-end, one direction is model training itself, but it requires a strong background in deep learning. "It is possible that fresh graduates who study deep learning understand the model better than you."

Secondly, data mining and processing provide data nutrients for end-to-end. "But once the infrastructure of the tool chain is built and the model structure is stable, people may no longer be needed." Finally, there is model post-processing. The output trajectory of the end-to-end large model is unreliable, and a small number of engineers are still needed to write rules to cover it.

This is also where the engineers' anxiety comes from. "On the one hand, the end-to-end large model itself does not require so many people. On the other hand, everyone wants to do end-to-end, but the company's mass production business needs people to run it."

An employee of Zhijia was also upset because he missed the opportunity to join the end-to-end project team because of the company's current mass production project. But he was also entangled: even if he joined the end-to-end solution, he would be a backup for the new solution, but this was not a core large model position;

If you stay in your current mass production project position, you can accumulate a complete set of experience in intelligent driving mass production projects, and you can also move to traditional car companies in the next few years.

But another danger will also come. Once the end-to-end solution is popularized throughout the industry, the technology stack he has accumulated over the years will also face the risk of being eliminated in a few years. "I may have to leave the intelligent driving industry."

Technology division, resource game

In order to switch to the end-to-end project team, engineer Tian Wei started directly with the graduate course on deep learning.

He found a classic course on deep learning and a graphics card, and used the practical courses in the textbook to implement some simple image recognition algorithms. "At least you need to thoroughly understand the knowledge points before you can understand how the model itself works."

After two months of reading and practical training, Tian Wei finally felt that he could understand some end-to-end large model open source code. He has applied to the company to be transferred to the end-to-end project team.

In fact, Tian Wei is not the only one who is anxious. The company where Tian Wei works is even more anxious than him. He told 36Kr that his company is working with a car company to develop a mass production solution for intelligent driving, but there is also a team within the car company that is promoting end-to-end. "The whole company is very anxious and has already started the end-to-end plan."

Tian Wei said that according to the company's understanding, an end-to-end demo can be trained with just 2,000 hours of video data, and this amount of data can be completed in one or two months using 50 vehicles.

But Tian Wei knew that with the company's existing resources, the best they could do was train an end-to-end demo to prove the solution was feasible. There was still a big gap between demo and mass production.

This game of dividing the field of new technology will first be reflected in the division of resources.

Tesla CEO Elon Musk once emphasized the importance of data for end-to-end: "Training with 1 million video cases is barely enough; 2 million is slightly better; 3 million is amazing; and 10 million is incredible."

On the other hand, Musk has also purchased a large number of Nvidia graphics cards for training, saying that by the end of the year, its AI training computing power will be equivalent to 90,000 Nvidia H100 graphics cards. The reserves and demand for computing power are staggering.

This threshold is quite high. For intelligent driving companies that still have difficulty making money, on the one hand, they do not cooperate with car companies, and it is difficult for intelligent driving companies to collect training data on their own; on the other hand, cloud training chips are hard to find in China, and many car companies are buying them at high prices. "Mass production projects and financing are still unclear, and it is difficult to invest in end-to-end for a long time."

Another intelligent driving engineer also felt helpless. After developing the end-to-end project for half a year, he received a notice from the company to suspend the end-to-end project. The reason was that the company had to focus its energy and resources on developing the current urban mapless intelligent driving solution, and "end-to-end consumes too many resources."

The engineer felt sorry that the end-to-end demo made by his team was already ready for the road. The team was originally aiming to compete with Tesla's FSD, and even spent a lot of effort to build infrastructure such as tool chains. But with the suspension of the company's end-to-end strategy, the team's research and development focus has shifted to the field of robotics.

The new end-to-end technology has not yet been truly implemented in China, but its impact on the reshaping of the talent structure and ecological landscape of the intelligent driving industry has begun to emerge.

Despite this, the leading players will still try their best to squeeze into this disruptive express train, and the era of giants that control data resources, chip resources, and talent resources will come.

(Qin Feng and Tian Wei in this article are pseudonyms.)