news

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

• Mengchen Cressy from Aofei Temple
  Quantum Bit | Public Account QbitAI

Just now, there was a major earthquake at the top of OpenAI:

Co-founder John SchulmanResign and run away,Greg Brockman, President and Co-founderLong-term leave,Peter Deng, Vice President of ProductsIt was also revealed that he had resigned.

John Schulman, who has led the OpenAI reinforcement learning team for many years, is known as theChatGPT architect"。

It was also he who temporarily took over the Super Alignment Team after Ilya Sustkever left.

Now he has announced that he has "made the difficult decision" to join Anthropic next door, aka Team Claude.

At the same time, OpenAI President Greg Brockman also announced that he would take a long vacation until the end of the year.

OpenAI plans to hold developer days and other events around the world starting October 1, and he will also be absent.

Brockman said that this was the first time he relaxed after working continuously for 9 years, but he was vague about his plans after the end of the year. Some netizens believed that it was due to a confidentiality agreement.

20 minutes after Schumann's resignation was officially announced, Altman responded by thanking Schumann for his contribution and recalling the first time the two met in 2015.

But as of now, Altman has not responded to Brockman's leave.

Altman normally had the habit of typing sentences without capitalizing the first letter of the letter and the “i”, but whenever an important colleague chose to leave, he would act more formally.

Apple Brother, a well-known OpenAI insider whistleblower account, said: The leadership has now become a shell, you should just release the product as soon as possible.

ChatGPT architect resigns

Schumann studied undergraduate at Caltechphysics, studied briefly at UC BerkeleyNeuroscience, and later studied underReinforcement LearningPieter Abbeel, a great guy, completed his PhD in Computer Science.

When he became a co-founder of OpenAI, he had not yet graduated with his PhD.

OpenAI is the first and only company he has ever worked for outside of an internship.

His representative works and most cited papersPPO, which is the reinforcement learning algorithm used in ChatGPT's core technology RLHF.

Later, he led the alignment/post-training team in a series of works from GPT-3.5, GPT-4 to GPT-4o.

The reason he gave for leaving this time was also related to this:

Hoping to deepen my focus on AI alignment and start a new chapter in my career where I can return to actual technical work.

However, he also gave OpenAI enough face and added:“Not leaving OpenAI due to lack of support for safety alignment work”, just out of a personal desire to focus.

In his speech at ICML2023, Schumann described one of his next research directions:How to avoid over-optimization。

• Some annoying behaviors in ChatGPT (e.g. excessive apologies) are the result of over-optimization.
• RLHF can easily over-optimize a narrow metric; for example, the model may learn shortcuts that lead to reward without true understanding.
• Proper training for dialogue tasks requires reasoning, perspective-taking, and knowledge, and researchers must carefully design the training process, reward functions, and datasets.
• The reward signal must assess the quality of the entire conversation, not just individual feedback.

In an interview with podcast host Dwarkesh Patel in May this year, Schumann also said,Computing power should be shifted more from pre-training to post-training。

OpenAI continues to lose talent

Due to differences in ideas about AI safety with Ultraman, OpenAI suffered a large loss of talent.

In May this year, two leaders of the Super Alignment Team - Chief ScientistIlya Sutskeverand one of the inventors of RLHFJan Leike, left OpenAI on the same day.

At that time, it had been less than a year since the establishment of the Super Alignment Team.

After Jan Leike left, he sent more than a dozen tweets in a row, bombarding OpenAI with criticism, accusing it of failing to deliver on its promise of 20% computing power and paying far less attention to security than to products.

Later, Jan Leike chose to join OpenAI's competitor Anthropic to continue his research on super alignment.

Co-led alignment work with Jan Leike on the GPT-4 projectRyan Lowe, also left OpenAI at the end of March.

And one of the members of Super AlignmentWilliam Saunders, and those who have participated in GPT-4 adversarial testingDaniel Kokotajlo, and have also resigned from OpenAI.

Not to mention next doorDario and Daniela Amodei, the founders of Anthropic, also from OpenAI.

Dario was the vice president of research at OpenAI before he left, and Daniela was the deputy director of security and strategy.Tom Brown, lead engineer of GPT-3Then a dozen employees were brought to Anthropic.

One of the reasons why they chose to leave and start their own business was that OpenAI directly released GPT-3 without resolving security issues, which aroused their dissatisfaction.

The emphasis on safety, coupled with the background of the founder, has made Anthropic one of the main destinations for OpenAI employees, including Jan Leike and Schumann who left this time.

OpenAI has also made some moves in security work recently——

At the end of July, OpenAI appointed its senior director of safetyAleksander MadryHe was transferred from the security position and reassigned to the "AI reasoning" job.

But OpenAI told the media that Madry will continue to work on AI safety in his new position.

CEO Altman also emphasized that OpenAI has always attached great importance to security work and insisted on the commitment of "giving at least 20% computing power to the entire security team."

But sharp-eyed netizens soon discovered that Ultraman had played a trick of changing the concept.The original promise was to give 20% of the computing power to the Super Alignment Team, but it was secretly changed to the entire security team.。

Therefore, after Ultraman’s tweet, netizens ruthlessly added this background information and attached the official link of OpenAI at the time.

emm...the Internet still has memory.

One More Thing

While a large number of talents are leaving the company, Musk, OpenAI’s “old enemy”, has also made new moves.

Musk just took OpenAI to court again, also defendants are Ultraman and President Brockman.

The reason for the lawsuit is that Musk believes he was "deceived and blackmailed" by Altman and Brockman when he participated in the founding of OpenAI.

Musk said that he participated in and invested in OpenAI because he believed what Ultraman said. At that time, Ultraman promised that OpenAI was a safe and open non-profit organization.

But later OpenAI became a for-profit organization and sold part of its shares to Microsoft, which made Musk very dissatisfied. He felt that he had been betrayed and denounced Ultraman for "betrayal."

Musk's lawyer Marc Toberoff said the lawsuit is very different from the previously withdrawn one, holding the defendants "accountable for intentionally making false statements to Musk and the public" and seeking "large-scale recovery of their ill-gotten gains."

At the same time, the indictment also requests the revocation of the license granted by OpenAI to Microsoft.

Reference Links:
[1]https://www.bloomberg.com/news/articles/2024-08-06/openai-co-founder-john-schulman-departs-for-ai-rival-anthropic
[2]https://www.cnbc.com/2024/07/23/openai-removes-ai-safety-executive-aleksander-madry-from-role.html
[3]https://x.com/sama/status/1818867964369928387
[4]https://www.documentcloud.org/documents/25031741-elon-vs-openai-again
[5]https://www.nytimes.com/2024/08/05/technology/elon-musk-openai-lawsuit.html