news

author of this article: li dan

source: hard ai

microsoft made a big move this tuesday, with its personal artificial intelligence (ai) assistant copilot ushering in a major upgrade. microsoft began rolling out a series of new copilot features to all windows users, including new tools that can understand and respond to computer screen problems. at the same time, microsoft's online search engine bing officially launched the ai-generated summary summary function, which is a head-on challenge to google's similar generative ai search function ai overviews.

starting from tuesday, october 1st, eastern time, microsoft launched a new version of the copilot app for apple ios, google android, windows and the web, saying that the styles of all these apps are more "unique" and "heartwarming". microsoft has also introduced chatbots to meta's social media whatsapp, allowing users to chat with copilot through direct messages (dm), similar to the experience of using other bots on meta's messaging platform.

copilot vision interprets the website content and deletes the data immediately after the conversation

copilot vision is the most prominent feature of copilot. as the name suggests, it can see the content that users view on their personal computers (pcs). specifically, it can view the websites that users visit using microsoft edge, the microsoft ai browser.

copilot vision is a new experimental optional feature exclusive to copilot pro. users can let it analyze text and images on web pages and answer user questions about the content of those web pages. for example, users can ask how to cook the food shown in the picture and have copilot provide recipes.

users can call it up by typing "@copilot" in the address bar of edge, similar to the search technology provided by google in android and chrome browsers. but microsoft says copilot vision is more powerful and privacy-focused than past screen analytics.

microsoft says copilot vision can advise users on what steps to take next, answer their questions, help guide users to whatever they want to do, and assist in completing tasks by simply talking to it in natural language. for example, the user wants to decorate a new apartment. copilot vision can help search for furniture, find the right color palette, consider everything from rugs to blankets, and even suggest how to arrange the items the user is looking at.

in terms of privacy, microsoft emphasized that users can delete conversation data immediately after talking to copilot vision. audio, images, or text processed by this feature will not be stored or used to train the model, at least not in this preview version. moreover, copilot vision is also limited in the types of websites it can parse, and can only analyze and interpret pre-approved "popular" websites. currently, microsoft blocks this feature from processing paid content and "sensitive" content on websites behind paywalls, but it has not disclosed what is considered sensitive content.

copilot vision is currently only available in the united states. microsoft said the feature will respect the site's "machine-readable controls on ai," such as rules prohibiting bots from scraping data for ai training. but the company hasn't made it clear which controls vision will respect; several are currently in use. we've asked microsoft for clarification.

copilot daily offers audio summary of current events microsoft pays partner publishers

microsoft on tuesday launched a news feature called copilot daily that provides users with voice summaries of weather and current events. to support the feature, microsoft will pay publishers to purchase content that appears in copilot daily.

alexa and google assistant, the voice assistants of amazon and google, have already provided similar daily briefings. microsoft says copilot daily "can defuse the familiar feeling of information overload" and that it is "simple, simple and easy to understand. it will only pull content from authorized content sources" and will provide reminders and customization over time. options.

microsoft currently only launches the copilot daily function in the united states and the united kingdom. reuters, axel springer, hearst magazines, usa today network and the financial times have partnered with microsoft to provide sources for this feature. microsoft said it plans to add paid publishers "soon" and expand copilot daily to more countries.

think deeper infers complex problems or supports openai o1 model

like vision, copilot's new feature think deeper is designed to make microsoft's ai assistant more flexible.

microsoft says think deeper allows copilot to reason about more complex questions, thanks to "inference models" that take more time to reason and then give step-by-step answers. microsoft did not disclose the specific inference models, but said that it uses "openai's latest model, which microsoft has fine-tuned." according to the media’s judgment, it should refer to the customized version of openai’s o1 model.

starting tuesday, think deeper will be available to a limited number of copilot labs users in australia, canada, new zealand, the united states and the united kingdom.

copilot voice is similar to chatgpt advanced voice mode

a new feature of copilot is copilot voice, first launched in english in new zealand, canada, australia, the united kingdom and the united states. microsoft has added four synthesized voices, allowing users to choose which voice to use when talking to copilot.

like the advanced voice mode provided by openai for chatgpt, copilot voice can recognize the user's tone during the conversation and respond accordingly. users can also interrupt at any time when copilot voice responds, just like when talking to humans. .

however, copilot voice has time limits. microsoft said copilot pro subscribers can get more copilot voice conversation time, but the specific time is "variable" and depends on demand.

personalization uses past interactions to recommend copilot usage

microsoft says copilot will soon be more responsive to user preferences with new personalization settings enabled. it will use the user's past interactions and history, as well as interactions with other microsoft apps and services, to recommend ways to use copilot. this helps users get started with copilot, "providing both a handy guide to copilot's practical features and a conversation starter."

copilot's personalization feature can be turned off in the copilot settings menu in windows and will not be available in the uk and eu countries in the near future.

bing's generative search launches in the u.s. on tuesday

in july this year, microsoft launched a trial version of bing's generative search. on tuesday, the service was officially launched for all us users. the easiest way to invoke it is to search for "bing generative search" on bing.

microsoft said it is rolling out an option to make it easier for users to trigger bing-generated searches for "information queries."

bing’s generative search is seen as a product of microsoft’s head-on battle with google’s ai search.google launched three major innovations in the search model at the 2024 google i/o developer conference that month. the first of them is an ai-generated summary function called ai overviews, which displays the generated summaries at the top of the search results.

microsoft believes that bing's generative search can do more than just find answers. microsoft says it understands search queries, scrutinizes millions of information sources, dynamically matches content, and generates search results with new ai-generated layouts to more effectively meet the intent of user queries.