news

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

A year ago, Amazon, Microsoft, Google, Meta,OpenAISeven artificial intelligence companies, including , Anthropic and Inflection, have reached eight voluntary commitments with the White House on how to develop artificial intelligence in a safe and trustworthy manner.

The commitments include increasing testing and transparency of AI systems and sharing information about potential harms and risks.

On the first anniversary of the voluntary pledge, MIT Technology Review asked the AI ​​companies that signed it for some details on their work so far. Their responses show some promising progress for the tech industry, but also some major caveats.

These voluntary commitments areGenerative AIThe announcement came as the frenzy was “perhaps at its frothiest”, with companies racing to launch their own models and make them bigger and better than their competitors’. At the same time, we began to see debates around issues such as copyright and deep fakes. A lobby of influential tech figures such as Geoffrey Hinton also raised concerns that AI could pose an existential risk to humanity. Suddenly, everyone was talking about the urgent need to ensure the safety of AI, and regulators everywhere were under pressure to act.

Until recently, the development of artificial intelligence has been like the "Wild West". The United States has traditionally been reluctant to regulate its tech giants, relying instead on them to self-regulate. The voluntary commitments are a good example: these are some prescriptive rules for the field of artificial intelligence in the United States, but they are still voluntary and cannot be enforced. The White House subsequently issued an executive order that expanded these commitments and applied them to other tech companies and government departments.

“A year later, we’re seeing some good practices from some companies around their products, but they’re nowhere near as far along as we need them to be in terms of good governance or protecting fundamental rights,” said Merve Hickok, president and research director of the Center for AI and Digital Policy, which reviewed the companies’ responses in response to MIT Technology Review’s request. “Many of these companies continue to make unsubstantiated claims about their products, such as claims that they can surpass human intelligence and capabilities,” she added.

One trend that emerged in the tech companies’ responses is that they are taking more steps to seek technological solutions, such as red-blue adversarial networks (humans exploring the flaws of AI models) and adding watermarks to AI-generated content.

It’s unclear what has happened to those pledges or whether the companies will follow through on them, said Rishi Bommasani, director of the Center for Fundamental Modeling at Stanford University, who also reviewed the responses for MIT Technology Review.

A year is a long time in the field of artificial intelligence. Since signing the voluntary pledge, Inflection AI founder Mustafa Suleyman has left the company to join Microsoft to lead its AI efforts. Inflection declined to comment.

“We appreciate the progress leading companies have made in fulfilling their voluntary commitments beyond the requirements of the executive order,” said White House spokeswoman Robyn Patterson. “However, the president continues to call on Congress to pass bipartisan legislation on AI.”

Brandie Nonnecke, director of the CITRIS Policy Lab at the University of California, Berkeley, said that without comprehensive federal legislation, all the U.S. can do now is require companies to meet these voluntary commitments.

But it’s important to remember that “these companies are essentially studying for the exams they’re taking,” Brandie Nonnecke said. “So we have to look carefully to see if they’re actually validating themselves in a really rigorous way.”

Here’s our assessment of the progress these AI companies have made over the past year.

Commitment 1. The companies commit to internal and external security testing of their AI systems before their release. This testing, which will be carried out in part by independent experts, guards against some of the most significant sources of AI risks, such as biosecurity and cybersecurity, as well as its broader societal effects.

Commitment 1:Conduct internal and external safety testing of AI systems before they are released. Part of this testing will be performed by independent experts to protect against some of the most important sources of AI risk, such as biosafety, cybersecurity, and broader societal impacts.

All of the companies (except Inflection, which chose not to comment) said they conducted red-blue duels, in which internal and external testers explored the flaws and risks of their models. OpenAI said it has a separate preparedness team that tests models for cybersecurity, chemical, biological, radiological and nuclear threats, and complex AI models that can do or persuade a person to do things that could cause harm. Anthropic and OpenAI also said they conduct these tests with external experts before launching new models. For example, to launch its latest model, Claude 3.5, Anthropic conducted pre-deployment testing with experts from the UK AI Safety Institute, and Anthropic also allowed the nonprofit research organization METR to test Claude 3.5.AutopilotGoogle said it also conducted an “initial exploration” of its model, Gemini, in internal red-blue duels to test the boundaries of election-related content, social risks, and national security issues. Microsoft said it has worked with third-party assessors from NewsGuard, an organization that promotes news integrity, to assess the risks and mitigate the risk of abuse of deep fakes in Microsoft’s text-to-image tool. Meta said that in addition to red-blue duels, it also evaluated its latest model, Llama 3, to see how it performs across a range of risk areas, including weapons, cyberattacks, and child exploitation.

“On the testing side, it’s not enough to just report that companies are taking action,” said Rishi Bommasani. Amazon and Anthropic, for example, said they have teamed up with the nonprofit Thorn to address the risks that AI poses to child safety. He wants more details on how the interventions companies are implementing actually reduce those risks.

“We should clearly recognize that it is not just the company that is doing things, but that these things are producing the desired effects,” said Rishi Bommasani.

result:That’s good. Driving red-blue and testing a variety of risks is important work. However, Hickok would like to see broader access to the company’s models by independent researchers.

Commitment 2. The companies commit to sharing information across the industry and with governments, civil society, and academia on managing AI risks. This includes best practices for safety, information on attempts to circumvent safeguards, and technical collaboration.

Commitment 2:Share information on managing AI risks with industry and government, civil society, and academia. This includes best security practices, information on attempts to circumvent safeguards, and technical collaboration.

After signing the voluntary pledge, Google, Microsoft, Anthropic, and OpenAI formed the Frontier Model Forum, a nonprofit organization to promote discussion and action on AI safety and responsibility. Amazon and Meta later joined.

Rishi Bommasani said that working with nonprofits funded by AI companies themselves might not be in the spirit of a voluntary commitment. In his view, the Frontier Model Forum could be a way for these companies to collaborate with each other and pass on safety information, which is often difficult for them as competitors.

“Even if they don’t disclose information to the public, you might hope that they can at least collectively figure out ways to reduce the risk,” Rishi Bommasani said.

All seven signatories are also members of the AI ​​Safety Institute Consortium (AISIC), established by the National Institute of Standards and Technology (NIST) to develop guidelines and standards for AI policy and AI performance assessment, a large consortium of public and private sector players. Google, Microsoft, and OpenAI are also represented on the United Nations High-Level Advisory Group on Artificial Intelligence.

Many companies also highlighted their research collaborations with academia. For example, Google is part of MLCommons, where it works with academics to conduct cross-industry AI safety benchmark research. Google also said it actively contributes tools and resources such as computing credits to programs such as the National Science Foundation's National AI Research Resources pilot program, which aims to democratize AI research in the United States.

Many companies also contributed to the Partnership on AI, another nonprofit founded by Amazon, Google, Microsoft, Facebook, DeepMind and IBM, which is responsible for the deployment of the underlying models.

result:Still, more needs to be done. As the industry works together to make AI systems safe and reliable, sharing more information is certainly an important step in the right direction. However, it’s unclear how much meaningful change the efforts that have been announced will actually lead to, and how much will just be cosmetic.

Commitment 3. The companies commit to investing in cybersecurity and insider threat safeguards to protect proprietary and unreleased model weights. These model weights are the most essential part of an AI system, and the companies agree that it is vital that the model weights be released only when intended and when security risks are considered.

Commitment 3:Invest in cybersecurity and insider threat protections to protect proprietary and unpublished model weights. These model weights are the most important part of the AI ​​system, and companies agree that it is critical to only release model weights when it is intentional and takes into account the security risks.

Many companies have implemented new cybersecurity measures in the past year. For example, Microsoft launched its "Secure Future Plan" to respond to the growing scale of cyber attacks. Microsoft said that its model weights are encrypted to reduce the potential risk of model theft, and strong authentication and access control are applied when deploying highly customized models.

Google has also launched an AI cyber defense program. In May, OpenAI shared six new measures it is developing to supplement its existing cybersecurity practices, such as extending encryption protection to AI hardware, and it also has a cybersecurity grant program that allows researchers to use its models to build cybersecurity defenses.

Amazon said it has also taken specific measures against attacks unique to generative AI, such as “data poisoning” and “cue word injection,” which might use hints to guide a language model to ignore previous instructions and safety protections.

Days after signing the voluntary commitment, Anthropic published details about its protections, which include common cybersecurity practices, such as controlling who has access to models and model weights, and checking and controlling third-party supply chains. The company is also working with independent assessors to evaluate whether the controls it has designed meet cybersecurity requirements.

result:All companies said they had taken additional steps to protect their models, though there didn’t seem to be much consensus on the best way to protect AI models.

Commitment 4. The companies commit to facilitating third-party discovery and reporting of vulnerabilities in their AI systems. Some issues may persist even after an AI system is released and a robust reporting mechanism enables them to be found and fixed quickly.

Commitment 4:Facilitate third parties to discover and report vulnerabilities in their AI systems. Even after the AI ​​system is released, some problems may still exist, and a strong reporting mechanism can enable problems to be discovered quickly and fixed in a timely manner.

One of the most popular ways to fulfill this commitment is to implement a "bug bounty" program, which rewards individuals who find flaws in AI systems. Google, Microsoft, Meta, Anthropic and OpenAI have all launched such programs for AI systems. Amazon and Anthropic also said they have established forms on their websites where security researchers can submit vulnerability reports.

In Brandie Nonnecke's view, it may take years to figure out how to do a good third-party audit. "This is not only a technical challenge, but also a socio-technical challenge. It will take us years to figure out not only the technical standards for artificial intelligence, but also the socio-technical standards, which is complex and difficult," she said.

Brandie Nonnecke said she is concerned that the first companies to implement third-party audits could set a bad precedent for how to think about and address the sociotechnical risks of AI. For example, the audits might define, assess, and address certain risks but ignore others.

result:Still, more needs to be done. Bug bounties are a good way to go, but they don’t go far enough. New laws, such as the EU’s AI Act, will require tech companies to conduct audits, and it would be even better if tech companies shared success stories from such audits.

Commitment 5. The companies commit to developing robust technical mechanisms to ensure that users know when content is AI generated, such as a watermarking system. This action enables creativity with AI to flourish but reduces the dangers of fraud and deception.

Commitment 5:Developing robust technical mechanisms to ensure users know which content is generated by AI, such as “watermarking systems,” allows AI creativity to flourish while reducing the risk of fraud and deception.

Many companies have built watermarking systems for AI-generated content. For example, Google launched SynthID, a watermarking tool for images, audio, text, and video generated by Gemini. Meta has developed an image watermarking tool called "Stable Signature" and a voice watermarking tool called "AudioSeal". Amazon now adds an "invisible watermark" to images generated by its Titan image generation model. OpenAI uses watermarking in its custom voice model Voice Engine and built an image detection classifier for images generated by DALL-E 3. Anthropic is the only company that has not yet built a watermarking tool because watermarking is mainly used for images, which the company's Claude model does not support.

All companies except Inflection, Anthropic, and Meta are also members of the Content Provenance and Authenticity Alliance (C2PA), an industry alliance that embeds information into image metadata about when content was created and whether it was created or edited by AI or humans. Microsoft and OpenAI automatically attach C2PA’s provenance metadata to images created with DALL-E 3 and videos created with Sora. While Meta is not a member of the alliance, it announced that it is using the C2PA standard to identify AI-generated images on its platform.

“The six companies that signed the voluntary commitment naturally gravitated toward technological approaches to address risk, and this is particularly true of watermarking systems,” said Rishi Bommasani.

“The question is, will ‘technical solutions’ make meaningful progress and address the underlying societal issues that prompt us to wonder whether content is machine-generated?” he added.

result:Very good. Overall, this is an encouraging result, and while the watermarking system is still experimental and still unreliable, it is still good to see the research around it and the commitment to the C2PA standard. It's better than nothing, especially in a busy election year.

Commitment 6. The companies commit to publicly reporting their AI systems’ capabilities, limitations, and areas of appropriate and inappropriate use. This report will cover both security risks and societal risks, such as the effects on fairness and bias.

Commitment 6:Publicly report on the capabilities, limitations, and appropriate use of their AI systems. This reporting will cover safety risks and societal risks, such as impacts on fairness and bias.

The White House's commitment leaves a lot of room for interpretation, for example, companies could technically meet this public disclosure requirement as long as they take steps in that direction, and the level of transparency can vary widely.

Here, the most common solution offered by tech companies is what are called "model cards." While each company calls them slightly different, they essentially serve as a product description of an AI model. They can cover everything from the model's capabilities and limitations (including how to measure fairness and interpretability benchmarks) to realism, robustness, governance, privacy, and security. Anthropic says it also tests models for potential safety issues that may arise later.

Microsoft released an annual Responsible AI Transparency Report that provides insight into how the company builds applications that use generative AI, makes decisions, and oversees the deployment of those applications. Microsoft also said it clearly indicates where and how AI is used in its products.

result:Still, more work needs to be done. Hickok said greater transparency into governance structures and financial relationships between companies would be an area for improvement for all companies, and she would also like to see companies be more open about the sources of their data, their model training processes, security incidents and energy use.

Commitment 7. The companies commit to prioritizing research on the societal risks that AI systems can pose, including on avoiding harmful bias and discrimination, and protecting privacy. The track record of AI shows the insidiousness and prevalence of these dangers, and the companies commit to rolling out AI that mitigates them.

Commitment 7:Prioritize research into the societal risks that AI systems may pose, including avoiding harmful biases, discrimination, and protecting privacy. The track record of AI shows how insidious and pervasive these dangers are, and these companies are committed to launching AI to mitigate them.

Tech companies have been busy conducting security research and incorporating findings into their products. Amazon built “guardrails” for “Amazon Bedrock” that detect hallucinations and apply security, privacy, and authenticity protections. Anthropic said the company employs a team of researchers focused on studying social risks and privacy, and in the past year has launched research on deception, jailbreaking, discrimination reduction strategies, and emerging capabilities of models to tamper with their own code or persuade. OpenAI said it has trained its models to avoid “hateful content” and refuse to generate hateful or extremist content, and it has trained GPT-4V to reject many requests that need to be answered based on stereotypes. Google DeepMind has also released research reports on assessing dangerous capabilities and conducted research on the abuse of generative AI.

All companies are investing heavily in research in this area. For example, Google has invested millions of dollars to create a new AI safety fund and promote research in this area through the Frontier Model Forum. Microsoft said it has pledged $20 million to study social risks through the National AI Research Resource and launched an AI model research accelerator program, the "Accelerating Foundational Model Research" program, and the company has also hired 24 researchers focusing on AI and sociology.

result:Excellent. This is an easy pledge to achieve, since the signatories are some of the largest and wealthiest corporate AI research labs in the world. While more research into how to ensure the safety of AI systems is a welcome step, critics have pointed out that the focus on safety research takes attention and resources away from AI research, which focuses on more immediate harms, such as discrimination and bias.

Commitment 8. The companies commit to develop and deploy advanced AI systems to help address society’s greatest challenges. From cancer prevention to mitigating climate change to so much in between, AI—if properly managed—can contribute enormously to the prosperity, equality, and security of all.

Commitment 8:Developing and deploying advanced AI systems to help solve society’s greatest challenges, from cancer prevention to climate change mitigation and many other areas, if managed well, AI can greatly advance human prosperity, equality, and security.

Since the promise was made, tech companies have been tackling a variety of problems. For example, Pfizer used Claude to assess trends in cancer treatment research after collecting relevant data, while US biopharmaceutical company Gilead used Amazon Web Services' generative AI to assess the feasibility of clinical studies and analyze data sets.

Google DeepMind has a strong track record of launching AI tools that can help scientists. For example, AlphaFold 3 can predict the structure and interactions of almost all molecules of life. AlphaGeometry solves geometry problems at a level comparable to that of an excellent high school student. GraphCast is an AI model capable of medium-term weather forecasting. Meanwhile, Microsoft is using satellite imagery and AI to improve the response to wildfires in Maui, Hawaii, and to map populations vulnerable to climate change, which is helping researchers uncover risks such as food insecurity, forced migration, and disease.

At the same time, OpenAI announced collaborations and funding for multiple research projects, such as one on how educators and scientists can safely use multimodal AI models in lab settings. The company also funded a “hackathon” to help researchers develop clean energy on its platform.

result:Very good. Some of the work in areas like using AI to advance scientific discovery or predict the weather is really exciting. AI companies are not yet using AI to prevent cancer, which is a pretty high bar to cross.

Overall, there have been some positive changes in how AI is built, such as red-blue confrontation, watermarking systems, and new ways to share best practices across industries. However, these are just some of the neat technical solutions found to the messy socio-technical problem of AI harms, and there is still a lot of work to be done. A year later, the commitment still overemphasizes a particular type of AI safety that focuses on "hypothetical risks" such as bioweapons, and completely ignores consumer protection, deep fakes, data and copyright, and the environmental footprint of AI. These omissions seem very strange today.

Original link:

https://www.technologyreview.com/2024/07/22/1095193/ai-companies-promised-the-white-house-to-self-regulate-one-year-ago-whats-changed/