news

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

Smart Things
Author: Vanilla Chengqian
Edit Mo Ying

Zhidongxi reported on August 14 that early this morning, at the 9th Made by Google event, AI became the keyword for the launch of Google's entire hardware family.AI was mentioned nearly a hundred times, Google has seamlessly integrated Gemini intoFolding screen mobile phones, smart watches, TWS headphonesAnd other types of hardware.

This time, Google, the global giant in big data and AI, launched its AI hardware family bucket that is deeply integrated with big data, before Apple officially released the iPhone 16 series equipped with Apple smart phones.Android becomes first mobile OS to feature large-scale on-device multimodal AI models。

▲Google Pixel 9 real shot

It is worth mentioning that when it comes to end-side AI, Google, Apple, and Samsung have basically the same idea, which isRebuilding the operating system with AI as the core and redefining the functions of mobile phones。

The updates of Google AI hardware are as follows:

1) Gemini Live, which is comparable to GPT-4o’s voice function, is now available on mobile phones;

2) Gemini can be called directly regardless of applications and services, and multiple applications can be called with one click to meet the needs;

3) Google released four AI phones: Pixel 9, two sizes of Pro models Pixel 9 Pro, Pixel 9 Pro XL, and foldable screen phone Pixel 9 Pro Fold;

4) The Pixel 9 camera is fully upgraded, with AI capabilities that can provide 8K resolution videos, automatic framing to save waste films, and can also turn concerts into artifacts in seconds;

5) Pixel Watch 3 can use AI to detect pulse loss and automatically send out an alarm;

6) Pixel Buds Pro 2 is the first to be equipped with the Tensor A1 chip, and the noise cancellation effect is twice as good as the previous generation.

It can be seen that Google’s current focus is to make the user’s AI experience smoother, which is also a concrete manifestation of its core advantages: from research and models that implement new features and capabilities, to the Android operating system that allows billions of people to use applications and services every day, to various high-end hardware manufactured using customized Tensor chips.

With Google's entry, the AI ​​mobile phone war is becoming more and more intense. However, the latest data from global market research firm IDC shows that the share of Pixel phones in the US market in 2023 will be about 4.6%. It is still unclear how much variable value AI can bring to Google Pixel phones.

1. Deep integration with Android, no need to call any Google apps, Gemini can take over with one click

Google's most important announcement this time is undoubtedly the new capabilities and full integration of its AI assistant Gemini.

Previously, Google Gemini, the "official smart assistant", had already appeared on Samsung phones. Today, its own Pixel not only finally deeply integrated Gemini, but also brought many updates.

first,Gemini Live, a voice-based alternative to GPT-4o, is now fully available in English to Gemini Advanced subscribers, and will be expanded to iOS and more languages ​​in the coming weeks.

Gemini directly becomes a chat partner, and the scope of the chat with users can range from solving complex problems to exploring innovative ideas. Users can freely talk to Gemini Live, not only can they interrupt at will in the middle of the answer, but also can pause the conversation and come back to continue later, and can also use it hands-free or in the background.

Google demonstrated a conversation with Gemini Live, which was very fast overall with almost no latency.

//oss.zhidx.com/uploads/2024/08/66bbc5d07d599_66bbc5d07073e_66bbc5d07071a_Live Demonstration 2.mp4

To make conversations more natural, Google has introduced 10 new voices, allowing users to choose the style and tone that best suits them.

//oss.zhidx.com/uploads/2024/08/66bbc442e4f76_66bbc442dfc3e_66bbc442dfc15_Gemini-Live Voice Type.mp4

Gemini is now deeply integrated with all Google apps, making the experience of calling Gemini on Pixel very smooth.

Users can long-press the power button or say "Hey Google" to summon Gemini to provide help anywhere, such as bringing up Gemini's overlay on top of the app they're using to ask questions about what's on the screen.

For example, when watching a video on YouTube, you can click "Ask this screen" and ask questions about the video content, or let Gemini extract information such as the address in the video into a list and add it to other applications such as Google Maps; you can also summon Gemini when writing an email, let it generate an image, and drag and drop the generated image directly into the text box.

▲Gemini interactive demonstration

Gemini can also integrate with all the Google apps and tools you currently use, without having to switch between apps and services.

For example, if a user is planning to host a dinner party, they can do it in one step: ask Gemini to find a lasagna recipe that a friend sent in Gmail, ask it to add the ingredients to the shopping list, and then ask it to make a "playlist of songs that remind us of the late 90s."

In the future, Gemini will also be integrated into applications such as calendars. Users will be able to directly take photos of concert flyers and chat with Gemini to confirm their time on the day and set reminders to purchase tickets without opening other applications.

▲Gemini demonstration

In terms of security, Gemini can help users by connecting relevant personal data with all the knowledge Google has collated and made public, with their permission. For example, Gemini can help you create a daily exercise plan based on your personal trainer's email, or write a job resume using your resume in Google Drive.

With the Gemini Nano, users can compile audio from calls with Call Recording, or save and organize images with Pixel Screenshots.

But whether that data is processed in the cloud or on the device, it exists only within Google’s secure end-to-end architecture, ensuring that the information remains safe and private.

To make users' AI experience on mobile devices smarter and more efficient, Google has introduced new models such as Gemini 1.5 Flash with higher response quality and faster speed. In the coming months, it will continue to launch deeper integration with Google Home, calls and messages.

2. Google's largest and thinnest foldable phone, with greatly improved AI graphics capabilities

Starting with phones, let’s take a look at how Google is upgrading its hardware devices around AI today.

Google released four new mobile phones, namely Pixel 9, two sizes of Pro models Pixel 9 Pro, Pixel 9 Pro XL, and foldable screen phone Pixel 9 Pro Fold, priced from US$799, US$999, and US$1799 respectively.

▲Pixel 9 series mobile phone prices

The Pixel 9 Pro Fold has the largest display ever on a Google phone and isThe thinnest foldable phoneIt features a Super Actua Flex display that's 80% brighter than the Pixel Fold.

▲Pixel 9 Pro Fold thickness

The Pixel 9 phone has an all-new look that puts the camera front and center, and it features a 6.3-inch Actua display that’s 35% brighter than the previous-generation Pixel 8.

For the first time, the Pro models of Pixel 9 are available in two different sizes, including the 6.3-inch Pixel 9 Pro and the 6.8-inch Pixel 9 Pro XL, both of which are equipped with Super Actua displays.

▲Two Pixel 9 Pro models

All four phones are powered by Google's latest custom chipsTensor G4Designed by Google DeepMind, Tensor G4 is optimized to run the most advanced AI models and is the first processor to run the multimodal Gemini Nano model, allowing phones to understand text, images, and audio.

In terms of AI functions, in addition to the deep integration of Gemini, PixelAI graphics capabilitiesIt has also been significantly upgraded.

Pixel launches its first image generatorPixel StudioIt is composed of a device-side diffusion model running on Tensor G4 and an Imagen 3 model in the cloud, providing fast text-to-image capabilities on mobile phones.

▲Pixel Studio

Many people have a lot of screenshots on their phones, but it is difficult to find the required information accurately and quickly when they need it. Pixel 9 launched a dedicated app Pixel Screenshots, which can save, organize and call important information in screenshots.

For example, if you are preparing a birthday gift for a friend who likes squirrels, you can first collect a large number of alternative gifts or related information in your browser and save them as screenshots. Pixel Screenshots will analyze the content of all the pictures and provide further suggestions.

▲Pixel Screenshots

Pixel also adds AI capabilities to apps like Weather and Calls. For example, the Clear Calling feature further improves audio quality, and the new Call Recording feature sends a private call summary and full call log immediately after hanging up the call. To protect privacy, the call recording feature runs entirely on the device.

3. From hardware, software to algorithms, nine major camera capability upgrades

In addition to AI capabilities, the camera is another focus of the Pixel 9 release, with nine main capability upgrades, including hardware upgrades such as lenses, HDR+ imaging pipeline, ultra-high-resolution zoom video, real-time augmented reality technology, Reimagine, automatic framing, panoramic night vision, zoom enhancement, and child-assisted shooting functions for parents.

First of all, at the hardware level, the two Pro models are equipped with professionalTriple rear camera system, the new 48MP 5x telephoto lens has an upgraded sensor for improved autofocus, and the front camera is 42MP for clearer images even in low-light conditions.

The Pixel 9 Pro Fold features an ultra-thin advanced triple rear camera system, including an ultra-wide camera that supports macro focus for photos and videos.

The Pixel 9 has the same main and ultra-wide lenses, which collect 110 percent more light and can focus as close as two centimeters.

Secondly, the Pixel 9 series phones use newHDR+ Imaging Pipeline, optimizes exposure, tone mapping, sharpening, contrast, and more to better capture elements such as texture, shadows, and skin tones in everyday scenes.

When shooting videos with the Pro model, users can use up to 20x ultra-high-resolution zoom capabilities, which combines the details of the upgraded telephoto camera with machine learning to capture high-resolution videos with rich details regardless of the lighting conditions - perhaps it will become another "concert artifact".

▲Ultra-high resolution zoom video

In addition, the Video Boost functionLeveraging AI capabilities to provide 8K resolution video, which can be upscaled 2x in post-production and retain 4K resolution.

▲Ultra-high resolution zoom video

When taking group photos, how to get the photographer involved has always been a problem. Now based on real-time augmented reality technology,Add MeYou can instruct the photographer on how to compose the photo after taking it to match the first photo, ensuring that no one is missed.

▲Add Me function

GoogleAI Image Editing Tool Magic EditorTwo new features have been added: Reimagine and Auto-Framing.

Reimagine can generate images based on text and edit them, such as changing the appearance of grass, sky, trees, interior decoration, etc. in a photo.

▲Reimagine

Automatic framing is suitable for "saving waste films". It uses AI to generate a better framing frame, allowing the photo to be recomposed, while generating a background to fill the blank space around the subject and highlight the subject of the photo.

▲Automatic framing

The Pixel's panorama mode has a simpler interface and higher quality, allowing for excellent low-light panoramas of landscapes and cityscapes.

▲Panoramic night vision function

Based on the zoom enhancement function, users can capture part of the photo and intelligently increase the resolution based on AI enhancement to obtain high-quality post-shooting zoom effects.

▲ Zoom enhancement

Targeting children, Pixel 9 Pro Fold also launchedMade You Look Features, interesting animations will be played on the flipped side to attract the child's attention and make it convenient for parents to take photos. To be honest, this function is quite suitable for pet owners.

▲Made You Look

4. AI detects pulse disappearance, TWS headphones are first equipped with Tensor A1 chip

Google's Pixel Watch 3 is called the biggest leap forward for its flagship smartwatch. In terms of size, the Pixel Watch 3 provides users with a new 45mm watch face option.

In addition to size, AI is the key update of this smartwatch.

First, users no longer need to manually turn on sleep mode. The Pixel Watch 3 uses machine learning toAutomatically detect sleep and enable sleep mode, which saves power and turns off notifications and the always-on display. Based on the same technology, it can also automatically detect when the user wakes up and exit sleep mode.

Secondly, like Apple and Samsung, Google's smartwatches focus on users' fitness and sports. To help users find a balance between fitness and rest, Pixel Watch 3 has launched three new tools: daily readiness (analyzing the user's readiness for daily training), aerobic load (tracking heart load during training), and target load (planning training based on personalized goals, cardiopulmonary load, and physical status).

▲Google Pixel Watch 3

In terms of running, Pixel Watch 3 also provides new features to help users plan runs, guide them to complete runs and reflect their running progress. In Fitbit Premium, the premium subscription model for Google Watch, Google AI will also provide automatic daily running suggestions. These suggestions are designed to help users reach their target load and can be personalized based on previous running situations.

The third highlight of the Pixel Watch 3 update is——The first pulse loss detection function, this feature combines Pixel Watch 3 sensor signals, AI and signal processing algorithms to detect pulse loss events.

The so-called pulselessness refers to the sudden cessation of the heart beating. This condition can affect people of any age and may be caused by various problems inside or outside the heart, such as primary cardiac arrest, respiratory or circulatory failure, drug overdose or poisoning.

The Pixel Watch 3's heart rate sensor detects the user's pulse. Once it cannot be detected, it will simultaneously activate the motion sensor to look for signals, and then combine the pulse and motion signals based on AI to confirm the pulse disappearance event.

If so, a check is triggered to see if the user reacts and actions are taken such as false positives or alarms.

Finally, Google has also moved the AI ​​​​scanning spam calls on Pixel phones to the Pixel Watch 3.

Let’s look at TWS headphones.Google calls the Pixel Buds Pro 2 earphones designed specifically for Gemini.

The most outstanding performance is that users can talk to Gemini without a mobile phone when wearing Pixel Buds Pro 2. In other words, with Gemini, users are like having a hands-free, no-eyes virtual AI assistant. It will proactively ask users if they need to walk to the nearest public transportation station? Do they need to be reminded of an upcoming appointment? Do they need to recommend a new song?

In addition, with Gemini Live, Pixel Buds Pro 2 provides a mobile conversation experience, allowing users to have long conversations with Gemini.

Users can even use Gemini Live on the go by simply unlocking their phone and saying “Hey Google, let’s chat in real time” while wearing the headset.

▲Google Pixel Buds Pro 2

In terms of hardware performance, Pixel Buds Pro 2 is equipped with a new chip designed for audio processingTensor A1 chipThis is the first time Google has expanded its Google Tensor chip beyond phones and tablets.

The Tensor A1 powers Google’s next-generation active noise cancellation technology, which adjusts to the user’s environment up to 3 million times per second to eliminate two times more noise than the previous-generation Pixel Buds Pro.

Conclusion: The battle for AI phones has begun, and large models are competing for landing positions

Following Apple and Samsung, Google is also making full efforts in AI mobile phones, which undoubtedly brings new vitality and competition to the technology industry. With Gemini as the core, Google is using AI to completely reconstruct the entire hardware bucket from mobile phones, watches to headphones.

The battle for AI phones has already begun. Google is leveraging its advantage in the Android market to expand the reach of Gemini and compete with OpenAI's ChatGPT for market share.

In addition to launching its own Pixel, Google has also cooperated with Chinese mobile phone brands such as Xiaomi and OPPO to introduce Gemini into more models. On the one hand, this reflects the status of domestic mobile phones in the global AI mobile phone market, and on the other hand, it also demonstrates the advantages of Google's AI capabilities.