news

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

Some companies are starting to build the “brain” of smart cars

Author | Cao Siqi
edit| Jingyu

"A successful smart electric vehicle company must be a successful AI company."

This is the most fashionable expression in the automotive field in the past two years. The most recent one came from Li Bin, CEO of NIO. Li Bin now often hides a series of titles such as entrepreneur and investor, and regards himself as a self-media person. The core purpose is to let more people know the real NIO. We should not think that NIO's technology is bad just because it has been labeled as "good service" in the past.

He said: "Innovation is the foundation of Weilai."

NIO is not the only one to have expressed similar views. In the past, every car company had its own label, such as control, luxury, power, economy... But now, everyone has a new consensus: label themselves as AI.

Li Bin and Qin Lihong share NIO's latest thinking in the field of technological innovation | Image source: Geek Park

Making cars smarter is the first key issue and also an important capability that car companies need to demonstrate to users as soon as possible at this stage.

But the word "becoming smarter" is a difficult description to quantify. What is a smart machine? Mira Murati, CTO of OpenAI, once tried to use the human intelligence level to compare the capabilities of AI models in a podcast: GPT-4 is equivalent to a smart high school student, while GPT-5 is expected to reach the level of a doctorate.

Although this statement is easy to understand, it is actually difficult to apply directly to cars.Because compared to the big model itself, smart cars, as a huge smart hardware carrier, not only need the "intelligence" to understand problems, but also need to solve problems through specific "abilities".

If we compare smart cars to robots, then in the process of this "person" becoming smarter, a "smart brain" is needed - not only must it have the intelligence of the model itself; it must also be responsible for developing a reasonable task management and allocation system for the "body" to determine how different "organs" need to handle a series of different tasks at different times and with what priority.

For example, active suspension is a very popular new feature in the past two years. Its core is to adjust the suspension parameters of the vehicle during driving based on memory and calculation to cope with the bumps of different roads and bring a better driving experience. This involves the coordination between different fields such as chassis and intelligent driving. If there is no unified brain coordination, due to technical problems such as bandwidth and delay, the coordination is easy to be uncoordinated and the desired experience cannot be achieved.

On July 27, at the 2024 NIO IN Innovation Technology Day, NIO released the "brain of the whole vehicle" - SkyOS, a global operating system for the whole vehicle. This represents NIO's thinking about the AI ​​era: it is reported that based on this system, a better experience can be provided in cross-domain task execution and computing power allocation.

And all these efforts are aimed at the same goal: making smart cars smarter.

01

world model,

Let smart driving "reason"

Intelligent driving is the most popular scenario for AI in cars, and it is also the area where technology routes have changed most rapidly in recent years.

In the past few years, intelligent driving has undergone several iterations of its technology routes, including switching from high-precision map solutions to map-free mode, and some companies represented by Tesla trying the pure vision (no lidar) route.

However, these route changes are essentially just to reduce costs, and do not change the nature of intelligent driving as "rule-driven". This system still requires a large number of programmers to spend time writing code and fixing bugs. In other words, this is just a process in which humans translate their understanding of driving to machines through programming languages, and the intelligent driving system itself does not generate intelligence.

The emergence of end-to-end technology has changed this set of rules. Rule-driven is evolving towards data-driven, and humans are trying to replicate the Scalling Law that has been proven successful in the field of large language models. By feeding a large amount of video data, the intelligent driving system is trained into a large neural network model that can generate intelligence.

The most ideal result of this model is the human driving thinking mode that Musk once described, namely: input video (road information seen), output control (pedal and steering control).

End-to-end intelligent driving architecture diagram | Image source: NIO

Applying the theory of "Three Cornerstones of AI", to achieve this goal, we need to solve three key problems: computing power, algorithms, and data.

NIO uses a "swarm intelligence" approach to data, that is, to obtain a large amount of real data based on a large number of mass-produced vehicles pre-installed with high-computing power and high-configuration perception hardware.

Computing power is more of an external issue and is not determined by the car companies themselves.

Therefore, the focus of the next stage is to start from the algorithm itself and improve the model's own training efficiency and reasoning ability. The next answer given by NIO is: Nio World Model (hereinafter referred to as NWM).

You can think of NWM as a variant of Sora, and to some extent, a video-generated product. The slight difference is that Sora is still mainly based on "video from text", while Weilai's world model is a "video from video": according to the official introduction, NWM can use 3 seconds of real driving video as a prompt to generate a video of up to 120 seconds.

NIO describes the NWM’s role as “deducing thousands of parallel worlds” | Image source: NIO

But video generation is obviously not NIO’s purpose, and NIO does not intend to use the videos generated by NWM as training data.

Let’s use the human thinking model as an analogy: before a human driver chooses to overtake, he or she will judge the different lane changes and choose different overtaking times based on the real-time road conditions. This decision-making and reasoning process is concretely expressed by NWM through a generated video.

According to the official introduction, NWM can deduce 216 possible scenarios within 100 milliseconds and find the optimal solution.

Therefore, the emergence of NWM actually found a new path for intelligent driving systems to become smarter in a new way of working (end-to-end) through the technical route of generative AI.

02

Discover the “human touch” in the cockpit

Another in-depth application scenario of AI in automobiles is the smart cockpit.

If you have experienced a "traditional car" and an "excellent smart car", you will not think that the various functions in the smart cockpit are just fancy tricks. I have always believed that the safest way to interact with the car during driving is through voice - it can ensure that your eyes and hands continue to focus on driving.

It can be said that in terms of cockpit intelligence, new car manufacturers with Internet genes are clearly ahead of many traditional car companies.

Now, when the AI ​​wave arrives, they have begun to face the next choice, just as Zhang Yong, former chairman of Alibaba, said:

Use AI methods to remake all products.

NIO hopes to use AI to redefine the paradigm of human-vehicle interaction | Image source: NIO

At present, NIO has basically moved all the mainstream AI software and hardware on the market to its cars.

For example, AI+music, through the addition of AI-powered sound quality algorithms, optimizes the sound sources that lack panoramic sound, and then plays them using the hardware built into the car.

With the help of AI, the parking guard function becomes a product similar to video conference minutes. AI will filter and generate key moments worthy of attention. If you are unfortunate enough to encounter scratches while parking, this product can become a tool to improve efficiency.

There is also a travel memory function that will be available soon. The press conference showed a vlog of the scenery captured by the on-board camera after a self-driving tour, which was automatically edited and generated. Some iconic peaks were marked. Similar AI editing functions have been used in sports cameras and other scenarios in the past, and now NIO has also brought it to the car.

In addition, NIO also demonstrated the evolution of NOMI with the support of multimodal capabilities: you can see the "hush" gesture and automatically lower the volume; you can also express appropriate emotional concern based on different environments.

NIO uses "emotional intelligence" to describe the evolution of AI in the cockpit field. However, in my opinion, these single functions do not represent a huge breakthrough in the capabilities of AI itself. Instead, it is the actual application scenarios that deserve attention, and the decisions made when they are put on the vehicle. In the cockpit field, perhaps the current exploration of application scenarios precedes the evolution of model capabilities.

On the evening of July 27, Li Bin, who had just finished his media interview, immediately went into another online live broadcast. Just like countless other times he spoke online, he encountered some embarrassing or sharp questions, and he chose to read them out directly:

"Stop messing with TikTok and focus on managing the company."

Li Bin smiled and explained that this was only his fifth live broadcast.But just like those little ingenious ideas hidden in the smart cockpit that may not be widely used now, he thinks that NIO’s thinking and efforts in innovation need to be introduced to everyone, “otherwise everyone will say that NIO’s service is good every day, as if our technology is not good.”

Fortunately, this expression received a lot of positive feedback. Li Bin joked at the media communication meeting that in the past, people said that they would not buy a car for one person (referring to Li Bin himself), but now many people also say that they will buy a car for one person, and he is very happy that he can do this.

Qin Lihong on the side added: In the past, what everyone looked forward to most was NIO DAY (NIO Day, the biggest event for NIO owners) at the end of the year. We hope that NIO IN can become the most favorite event for the media and car owners in the future.

Hope this day can come as soon as possible.

*Header image source: NIO

This article is an original article from Geek Park. For reprinting, please contact Geek Jun on WeChat: geekparkGO

Geek Question

What are your thoughts on smart driving?What are your expectations?

Musk: Social media is fake and it is gradually destroying your self-confidence.

Like and followGeek Park Video Account，