news

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

Jin Lei's west wind blows from Aofei Temple
Quantum Bit | Public Account QbitAI

Tsinghua Yao ClassYao Shunyu, officially announced that it had joined OpenAI.

And it was just such a short message, but it attracted the attention and blessings of many people in the circle. Let’s feel this feeling:

Among them are OpenAI's cutting-edge research director, American IOI coachMark Chen, as well as professors and investors in the field of AI.

So why does Yao Shunyu attract so much attention?

From his past resume, we can extract the following key words:

• Tsinghua Yao Class
• Chairman of Yao Class Joint Committee
• Co-founder of Tsinghua University Student Rap Club
• PhD in Computer Science from Princeton

△Yao Shunyu, source: personal homepage

But in addition to his outstanding resume, what really made Yao Shunyu enter the public eye was his many scientific research achievements:

• Thinking Tree(Tree of Thoughts): Allow LLM to think repeatedly and greatly improve reasoning ability.
• SWE-bench: A large model capability evaluation dataset.
• SWE-agent: An open source AI programmer.

It is no exaggeration to say that almost every study has caused a lot of ripples in the circle; and it is very obvious that they are all deeply rooted in theLarge ModelAnd unfold.

This may also be in line with what Yao Shunyu said in this official announcement:

It’s time to turn research vision into reality.

As for this “research vision”, let’s continue to learn more about it.

Research Keywords: Language Agents

If you look at Yao Shunyu's homepage, especially the research section, you will find a phrase that appears very frequently:Language Agents。

Including the introduction on his X homepage, the first sentence is also Language Agents:

This is also the title of his doctoral dissertation:Language Agents: From Next-Token Prediction to Digital Automation。

Language Agents, that isLanguage Agent, Yao Shunyu proposed a new category of intelligent agent.

Unlike traditional intelligent agents, this method uses language models for the reasoning and actions of intelligent agents, focusing on enabling them to achieveDigital Automation（Digital Automation）。

As for the specific implementation method, there are three key technologies (each with its own independent paper), which are:

• ReAct: A method that combines reasoning and action, generating reasoning trajectories and actions through language models to solve various language reasoning and decision-making tasks.
• Thinking Tree: A tree search-based approach that solves complex problems by generating and evaluating multiple thinking paths and improving the reasoning ability of language models.
• CoALA: A conceptual framework for organizing and designing linguistic agents, including aspects of memory, action space, and decision making.

Taking ReAct as an example, the research is to expand the action space of the language model into the union of the action set and the language space.

Actions in language space (i.e., thoughts or reasoning trajectories) do not affect the external environment, but can update the context by reasoning about the current context, which can support future reasoning or actions.

For example, in the conversation shown in the figure below, the ReAct method can be used to guide the agent to cycle the process of "generating ideas → taking actions → observing results".

In this way, the reasoning trajectory and operations can be combined to allow the model to perform dynamic reasoning, making the agent's decision and final results better.

If we reduce the ReAct method to letting the agent“reason to act”, then the next method, that isThinking Tree, then the focus is on making the intelligent agent“reason to plan”。

The thinking tree represents the problem as a search on a tree structure. Each node is a state, representing a partial solution, and the branches correspond to operations that modify the state.

It mainly involves four issues:

• Thinking decomposition: Break down complex problems into a series of intermediate steps, each of which can be seen as a node in the tree.
• Thought generation: Use the language model to generate potential thoughts for each node, which are the intermediate steps or strategies to solve the problem.
• State evaluation: The state of each node is evaluated through the language model to determine its progress and potential in solving the problem.
• Search algorithm: Use different search algorithms (such as breadth-first search BFS or depth-first search DFS) to explore the mind tree and find the optimal solution.

Applying the mind tree to the "24 Points" game has significantly improved the accuracy compared to the previous mind chain (CoT).

As for the last key technology in Language Agents,CoALA, is a conceptual framework for organizing and designing language agents.

From the structural diagram below, it can be roughly divided into three modules: information storage, action space and decision making.

Information storage refers to the storage of information by language agents in multiple memory modules, including short-term working memory and long-term memory (such as semantic memory, episodic memory, and procedural memory).

These memory modules are used to store different types of information, such as sensory input, knowledge, experience, etc., and play a role in the agent's decision-making process.

In addition, CoALA divides the action space of the agent into external actions and internal actions; external actions involve interactions with the external environment, such as controlling robots, communicating with humans, or performing operations in a digital environment.

Internal actions interact with the agent's internal state and memory, including operations such as reasoning, retrieval, and learning.

Ultimately, the language agent will choose the action to perform through a decision-making process; this process will also find the optimal solution based on various factors and feedback.

In addition, there are jobs like open source AI programmersSWE-agentAnd so on, which are also widely circulated in the circle.

But from Yao Shunyu's many scientific research topics, in addition to Language Agents, we can also see another keyword he pursues:Computational Thinking。

In fact, this was already revealed when he was an undergraduate.

Before heading to Princeton University to pursue a Ph.D. in computer science, Yao Shunyu, as a senior of the class of 2015, shared his learning and growth experience at Tsinghua with the re-examination candidates at the opening ceremony of Tsinghua's 2019 various types of independent selection re-examinations.

The relevant content is recorded in an article he wrote himself titled "What did you learn in Tsinghua's Yao Class? Yao Shunyu: Enough to change the world."

At that time, he focused on sharing computational thinking from both theoretical and practical aspects, and revealed that after four years,The biggest gain is computational thinking：

Theoretically, we now see many things that are impossible to do. When it comes to so-called theory guiding practice, I think it means that we have to understand the limits of a system’s capabilities and the difficulty of things from a high level, and then choose to do things that are possible and meaningful.

With the tag of sunny and cheerful big boy, Yao Shunyu also shared his experience of going to Argentina for Tsinghua Southern Immersion Program:

I met a group of Argentinian kids... English is not universal, Argentinians speak Spanish. I tried to learn Spanish, but I gave up because I studied computer science, so I took out Google Translate. I told them about the Forbidden City and the Great Wall in Beijing...

△Source: Tsinghua Admissions Official Account

In his opinion, in this era, computing can be combined with any subject, and the world is big, and you can do whatever you want at Tsinghua.

After talking about Yao Shunyu, who else in Yao's class is working on big models?

Who else in Yao Class is working on this popular big model?

It must be mentionedMa TengyuandChen Danqi。

The two were classmates and alumni of Tsinghua University's Yao Class in 2008. They both later won the Sloan Prize, which is known as the "barometer of the Nobel Prize."

Dr. Ma Tengyu studied at Princeton University, where his advisor was Professor Sanjeev Arora, a theoretical computer scientist and two-time Gödel Prize winner.

After graduating with a Ph.D., top universities such as MIT, Harvard, and Stanford all offered him an assistant professor position, but Ma Tengyu ultimately chose Stanford.

At the end of last year, Ma Tengyu officially announced his big model startup - Voyage AI, and revealed that he would lead the team to create the best embedding model currently available, and would also provide customized models focusing on a certain field or enterprise.

Three professors, including Christopher Manning, director of the Stanford Artificial Intelligence Laboratory, and Fei-Fei Li, a well-known Chinese scholar in the field of AI, serve as academic advisors to Voyage AI.

As for Chen Danqi, after completing his undergraduate studies at Tsinghua University's Yao Class, he obtained a Ph.D. from Stanford University in 2018, majoring in NLP. He eventually became an assistant professor in the Department of Computer Science at Princeton University, deputy director of the Princeton Language and Intelligence Project, and co-led the Princeton NLP group.

According to his personal homepage, he is "mainly attracted to developing large models these days" and is currently researching the following topics:

• How retrieval can play an important role in next-generation models, improving realism, adaptability, interpretability, and trustworthiness.
• Low-cost training and deployment of large models, improved training methods, data management, model compression, and downstream task adaptation optimization.
• I am also interested in work that truly advances the understanding of the capabilities and limitations of current large models, both empirically and theoretically.

QuantumBit also continues to pay attention to the large-scale model work of Chen Danqi's team.

For example, the proposed large-model cost-reduction method - data selection algorithm LESS, only selects the 5% of data most relevant to the task to fine-tune instructions, which is better than using the entire data set.

And instruction fine-tuning is the key step to make the basic model become a ChatGPT-like assistant model.

He proposed the popular "alpaca shearing" method - LLM-Shearing large model pruning method, which achieved SOTA with only 3% of the computational effort and 5% of the cost, and dominated the open source large models of 1B-3B scale.

In addition to these two, there are many Yao Class alumni in the industry and academia who are working on large models.

The popular large model native application "It's Over! I'm Surrounded by Large Models" and its sequel "I Broke the Large Model" were developed by the Yao Class academic team.

Game AuthorFan Haoqiang, Megvii's No. 6 employee. He was hailed as a genius boy for his legendary achievements such as the IOI gold medal, admission to Tsinghua University's Yao Class, and internship in his second year of high school. Now he is the general manager of Megvii Technology Research and an industry leader with a Google Scholar h-index of 32.

Musk xAI’s first research result, Tensor Programs VI, also had Yao Class alumni as a co-author.

Tensor Programs VI is a continuation of the previous Tensor Programs series work by Greg Yang, a founding member of xAI and a disciple of Qiu Chengtong. The paper focuses on "how to train infinitely deep networks."

It is said that the results related to Tensor Programs have been applied in GPT-4. In order to interpret the paper, Young himself also held a live broadcast sharing on X at that time.

Co-authorDingli YuDingli Yu graduated from Tsinghua University's Yao Class with a bachelor's degree. Currently, he is about to graduate with a Ph.D. in Computer Science from Princeton.

There are many more……

Back to the time when Yao Shunyu was recruited to OpenAI, OpenAI's recruitment efforts are still ongoing.

OpenAI engineer Karina Nguyen published the latest recruitment post:

• OpenAI Model Behavior Team is hiring! This is a dream position that combines design engineering and post-training research, and it is also the rarest job in the world❤️
• We use alignment methods such as RLHF/RLAIF to define model core behaviors to reflect fundamental values ​​and enhance the creative intelligence of AGI. Through these achievements, we work with product + model design and engineering teams to create new AI interface and interaction models that will affect millions of users...
  

Interestingly, Karina Nguyen was actually a researcher at Anthropic AI (Claude team) before. In May last year, she had a prompt word duel on X (formerly Twitter) with Jason Wei of OpenAI, the first author of the "pioneering paper" of Siwei Chain.

I didn’t expect Karina Nguyen to jump to OpenAI so quickly…

By the way, just yesterday there was news that Google DeepMind researcher Thibault Sottiaux was also poached by OpenAI.

You should know that Thibault Sottiaux is a core contributor to papers such as the first generation of Gemini and Gemini 1.5.

This shows how popular the large model track is currently, with everyone scrambling to grab the track and to grab talent.

One More Thing

There were two other Yao Shunyus who graduated from Tsinghua University in the same year as Yao Shunyu!

When the three Yao Shunyu graduated in 2019, Tsinghua University officially posted a Weibo post and shared a group photo of the three of them.

In addition to Yao Shunyu who has now joined OpenAI, there is another Yao Shunyu fromschool of HumanitiesA girl majoring in Japanese.

Another Yao Shunyu is Yao ShunYu, fromDepartment of PhysicsHe is the winner of the 2018 Undergraduate Special Award. During his undergraduate studies, he published two papers in the top physics journal PRL (Physical Review Letters) and one paper in PRB (Physical Review B) as the first author.

Reference Links:
[1]https://x.com/ShunyuYao12/status/1818807946756997624
[2]https://ysymyth.github.io
[3]https://x.com/karinanguyen_/status/1819082842238079371
[4]https://weibo.com/1676317545/HCR7yuXAl?refer_flag=1001030103_