news

Text: Web3 Sky City·City Lord

In this latest Stanford class interview, former Google CEO Eric Schmidt dives into the direction of artificial intelligence, the future of global tech competition, and how these technologies will have a profound impact on society, the economy, and national security in the short term.SchmidtHe reviewed his years of experience in the technology industry, shared his foresight on breakthroughs in artificial intelligence technology in the coming years, and discussed how these breakthroughs will shape the competitive landscape among companies and countries, especially in the competition between the United States and China.

SchmidtSpecial emphasis is placed on the complex factors behind technological progress, such as the increase in computing power, the development of new algorithms, and the market's endless pursuit of intelligent systems. He also discusses the potential impact of artificial intelligence on the labor market, data privacy, antitrust, and national security, and offers suggestions on how to maintain a competitive advantage in this rapidly changing environment.

The city lord especially pointed out that as Schmidt has served the US Department of Defense for a long time, his remarks and positions need to be identified, and I believe that readers will be able to understand them on their own.

Short-term breakthroughs and far-reaching impacts of artificial intelligence：

SchmidtIt is predicted that in the next one to two years, artificial intelligence will usher in important breakthroughs, especially in the combination of context window expansion, AI agents and text-to-action. These technological advances will enable artificial intelligence systems to handle complex tasks more effectively and surpass current limitations. This progress will not only be limited to the field of technology, but will profoundly affect all levels of society, including education, healthcare, government and business. He emphasized that the transformative impact that the development of these technologies may bring may even be more far-reaching than the impact of the rise of social media on society.

The Sino-US Game in the Global Science and Technology Competition：

SchmidtThe fierce competition between the United States and China in the field of artificial intelligence was analyzed in detail. He pointed out that the United States currently leads in technology, talent and resources, but to maintain this advantage, it needs continued high investment and international cooperation, especially cooperation with allies such as Canada to ensure a sustainable supply of energy and resources. He stressed that the future of AI is not just a technological competition, but also a strategic game between countries involving national security, economic competitiveness and global leadership. Schmidt warned that the United States needs to increase investment to cope with China's rapid rise in the field of AI and maintain its global dominance in this field.

Monopoly and innovation challenges of technology giants：

When discussing the current dominance of tech giants,SchmidtHe pointed out that the monopoly of companies such as NVIDIA in the field of AI is due to their strong technical capabilities and capital advantages. He mentioned that although there are competitors in the market, challenging the status of these technology giants requires huge investments and technological innovation. He also expressed concerns about how these giants will continue to promote technological innovation in the future, believing that capital-intensive AI development may lead to fundamental changes in the software development model, from open source to closed source, thereby further consolidating the monopoly of the giants.

The impact of artificial intelligence on society and the labor market：

SchmidtHe discussed the potential impact of artificial intelligence on society, the economy, and the labor market. He believes that while AI technology may replace certain repetitive jobs, it will also enhance the importance of high-skilled jobs and drive people's productivity in complex tasks. He also expressed concern about the social inequality that artificial intelligence may bring, pointing out that wealthy countries will benefit more from AI, while poor countries may be left behind. In addition,SchmidtCalls for greater regulation of AI to address issues such as data privacy, intellectual property and the spread of misinformation.

Adversarial AI and Security Challenges：

SchmidtThe potential threat of adversarial artificial intelligence is particularly mentioned, predicting that in the future there may be specially designed AI systems used to attack and destroy other AI systems. This development will bring new challenges to security and ethics.SchmidtIt is suggested that the scientific and technological community and the government need to work together to study how to prevent these risks and formulate corresponding regulations and technical standards to ensure the safety and credibility of artificial intelligence. He also mentioned that research in this field will become an important direction for future scientific and technological development and may receive more attention in universities and research institutions.

Former Google CEO talks about global AI competition: US chips have a 10-year technological advantage, and China is the most important competitor

=Web3 Sky City Full Text Edition=

Host Professor:

Today's guest needs no introduction. I first met Eric about 25 years ago when he came to Stanford Business School as the CEO of Novell. Since then, he has held important positions at Google, starting in 2001, and joined Schmidt Futures in 2017. In addition, he has been involved in many other projects, and you can check out the relevant information. So Eric, if I can, I will start with that.

First, where do you see AI going in the short term, which I guess you define as the next one to two years?

Schmidt:

Things are changing so fast that I feel like every six months I need to give a new talk about what's going on. I'm a computer science major, can anyone explain to the rest of the class what "a milliontokenWhat is a "context window"? Please name it and tell us what it does.

student:Basically, it allows you to use a million markers or a millionWord prompts.

Schmidt:

So you can ask a million-word question. Anthropic is 200,000 tokens, going to 1 million, and so on. You can imagineOpenAIThere is a similar goal.

Can anyone here give a technical definition of an AI agent? Again, computer science.

student:An AI agent could be something that acts in a certain way. It could be calling something on the web to look up information on your behalf. It could be a number of different things along those lines. There are all sorts of things going on in one process.

Schmidt:So an agent is something that performs a task. Another definition is that it is aLLM, state, and memory. Next, computer scientists, can any of you define "text to action"?

student:Instead of converting text into more text, let the AI ​​trigger actions based on this.

Schmidt:

Another definition is the Python language. There is a programming language that I never want to see survive. Everything in AI is done in Python. A new language just came out called Mojo and it looks like they have finally solved AI programming, but we will see if it can really survive the dominance of Python.

Here’s another tech question. Why is NVIDIA worth $2 trillion while other companies are struggling?

student:The technical answer is that most code will need to run using CUDA optimizations which are currently only supported on NVIDIA GPUs, so other companies can do whatever they want, but unless they have 10 years of experience in software, you won't have machine learning optimizations.

Schmidt:

I like to think of CUDA as the C programming language for GPUs, and I find this idea satisfying. CUDA was founded in 2008. Even though I always thought it was a terrible language, it has become dominant. There is also a noteworthy insight: there is a set of open source libraries that are highly optimized for CUDA that are not done by any other library. This is completely missed in the discussion by all the people building these technology stacks. These libraries are technically called VLLMs, and there are many similar libraries that are also highly optimized for CUDA, making it difficult for competitors to copy.

So what does all this mean? In the next year, you will see very large context windows, agents, and text-to-action applications. When these technologies are delivered at scale, they will have a huge impact on the world, far beyond the impact of social media. Here’s why: In the context window, you can use it as short-term memory, and I was shocked by the length of the context window. The technical reason has to do with the difficulty of serving and computing. The interesting thing about short-term memory is that when you input information and ask questions, like read 20 books and use the text of the book as a query, and then ask them what they are about, it forgets the middle parts, which is similar to how the human brain works.

With regard to agents, there are people now building LLM agents where they read and understand knowledge in areas like chemistry, then test it and add it back into their understanding. That's very powerful. The third aspect is text to action. I'll give you an example: let's say the government is trying to ban TikTok. If TikTok is banned, I suggest you say the following to your LLM: copy TikTok for me, put my preferences in it, make this app and publish it in the next 30 seconds, and then within an hour, if it doesn't go viral, do something similar. That's command. You can see how powerful this is.

If you can convert from an arbitrary language to an arbitrary numeric command, that's essentially what Python is in this scenario. Imagine that everyone on Earth had their own programmer who actually did what they wanted to do, instead of programmers who didn't work as required. The programmers here know what I'm talking about. Imagine a programmer who wasn't arrogant and actually did what you wanted to do, without you having to pay a high price. And the supply of these programmers is unlimited.

professor:Will all this happen in the next year or two?

Schmidt:

Very soon. All three of these things, I am sure, will happen simultaneously in the next wave. So you ask what else will happen. I have a swing every six months, so we are in an odd-even oscillation. Right now, the gap between the leading edge models (there are only three now) and the rest seems to be getting bigger. Six months ago, I was sure that the gap was closing, so I invested heavily in some small companies. However, now I am not so sure about this.

I'm in conversations with some of the big companies who are telling me they're going to need $10, $20, $50, or even $100 billion. The Stargate project is going to be $100 billion, and it's going to be very difficult. My good friend Sam Altman thinks it's going to be about $300 billion, maybe more. I pointed out to him that I've calculated the amount of energy that's going to be needed.

In the interest of full disclosure, I went to the White House on Friday and told them that we need to be best friends with Canada. Because the Canadian people are really nice, they helped invent AI, and they have a lot of hydroelectric resources. Because we as a country don't have the power to accomplish this. The alternative is to have an Arab country fund this project. I personally like the Arabs, I've spent a lot of time there, but they probably won't play by our national security rules. And Canada and the United States are one of the big three that we can all agree on.

So in these $100 billion to $300 billion data centers, power starts to become a scarce resource.

By the way, if you follow this line of reasoning, you might ask why am I discussing CUDA and NVIDIA? If all $300 billion went to NVIDIA, you know what to do in the stock market. However, this is not a stock recommendation and I am not a licensor.

professor:

Part of the reason is that we're going to need more chips, but Intel gets a lot of money from the US government. AMD is trying to build a fab in South Korea.

Schmidt:

Raise your hand if you have an Intel chip in any of your computing devices. So much for the monopoly.

professor:

But that's the point. They did have a monopoly at one time, and now NVIDIA has a monopoly. So are these barriers to entry?

Speaking of CUDA, are there any other options? I was talking to Percy Lange the other day. He's switching between TPUs and NVIDIA chips, depending on what he can get his hands on. That's because he has no choice.

Schmidt:

If he had unlimited funds, he would choose NVIDIA's B200 architecture today because it is faster. I am not implying that - having competition is a good thing.

I've talked to Lisa Su from AMD at length. They've built something that can convert the CUDA architecture to their own architecture, called Rokam. It's not fully functional yet, but they're working on it.

professor:

You worked at Google for a long time, and they inventedTransformerArchitecture. Thanks to the great people over there, like Peter, Jeff Dean and all. Currently, OpenAI seems to have lost the initiative. In the latest leaderboard I saw, Claude from Anthropic is at the top. I asked Sundar about this, but he didn't give me a very clear answer. Maybe you have a more pointed or objective explanation of the situation there.

Schmidt:

I'm no longer a Google employee. Google's work-life balance is more about letting employees go home early and work from home than about winning. In contrast, startups succeed because their employees work their ass off. Although it may be a bit blunt to say this, the fact is that if you start a company out of college and want to compete with other startups, you can't have employees come in only one day a week.

Google's former CEO publicly complained that employees "weren't working hard enough" and fell behind in the AI ​​race because they "only came one day a week" and also hacked TikTok

professor:The same was true for Microsoft in the early days of Google.

Schmidt:

Now it seems that in our industry, for a long time, companies have always won by being really creative and dominating a space rather than making the next pivot. This is well documented. I think founders are special and they need to be in control, even though they can be difficult to work with because they put a lot of pressure on their employees. As much as we don’t like Elon’s personal behavior, look at what he gets from his employees. I had dinner with him once and he was flying. I was in Montana and he had to fly at 10 o’clock that night for a midnight meeting with x.ai. Think about that.

Different places have different cultures. I was impressed by TSMC. They had a rule that freshly graduated PhD students, excellent physicists, had to work in the basement of the factory. Can you imagine having a physics PhD in the United States do that? Not likely. It's a different work ethic.

The reason I work so hard is because these systems have network effects, so time is very important. In most businesses, time is not that important, you have a lot of time. Coke and Pepsi will still exist, they will still compete with each other, everything is cold and icy. When I was dealing with telecom companies, the typical deal took 18 months to sign. There is no reason to take 18 months to do anything, it should be done as soon as possible. We are in a period of growth and maximizing revenue, but it also requires crazy ideas.

For example, when Microsoft made the deal with OpenAI, I thought that was the dumbest idea I had ever heard. Outsourcing your AI leadership to OpenAI and Sam and his team? That's crazy. No one would do that at Microsoft or anywhere else. Yet today, they are becoming the most valuable company, certainly going toe-to-toe with Apple. Apple didn't have a good AI solution, but it looks like they made it work.

student:

How will AI play out in terms of national security or geopolitical interests, especially in the competition with China?

Schmidt:

As the chair of an AI commission, I took a deep dive into this. We wrote a report of about 752 pages, and the summary was this: We are leading, and we need to maintain that lead, and this requires a lot of funding. Our main clients were the Senate and the House, and this led to the CHIPS Act and other similar policies.

If the frontier model and some open source models continue to develop, there may be only a few companies that can compete in this field. Which countries have such capabilities? These countries need to have sufficient funds, abundant talents, strong education systems, and the will to win. The United States and China are two of the main countries. As for whether other countries can participate, I am not sure. But what is certain is that in the future, the competition between the United States and China for intellectual hegemony will be a major struggle.

The U.S. government has essentially banned NVIDIA chip exports to China, even though they won't admit it publicly. We have about a 10-year technology advantage in sub-DUV chips, sub-5nm chips. That advantage puts us several years ahead of China, which makes China very unhappy. This policy was set by the Trump administration and supported by the Biden administration.

professor:Whether Congress listens to your advice and makes large investments, obviously the CHIPS Act is an example of this.

Schmidt:

In addition, we need to build a massive AI system. I lead an informal, ad hoc, non-legal group of the usual industry players who last year made the case for what became the Biden administration’s AI bill, the longest presidential directive in history.

We talked about a core problem: How do you detect danger in a system that has learned something but you don't know what to ask? In other words, the system might have learned something bad, but you don't know how to ask it. For example, it might have learned how to mix chemicals in some new way, but you don't know how to ask it. To solve this problem, we proposed in our memo to the government to set a threshold that we called 10 to the 26th power, which is a technical computing measure. Above this threshold, companies must report their activities to the government. The European Union, to make sure they are different, put 10 to the 25th power of 10. But these numbers are close enough. I think all of these distinctions will disappear because the existing technology will be obsolete. The technical term is called co-training, which basically means that the parts can be merged together. So we may not be able to protect people from these new things.

professor:

Rumor has it that OpenAI has to train this way in part because of power consumption. There is no single place they do it this way.

Let’s talk about a real war going on right now. I know you were actively involved in the war in Ukraine, especially regarding the White Stork program and your goal of destroying a $5 million tank with a $500 drone. How does this change the war?

Schmidt:

I worked for the Secretary of Defense for seven years, trying to change the way we run the military. I didn't particularly like the military, but it was very expensive, and I wanted to see if I could help. I basically failed, I think. They gave me a medal, so they probably give medals to losers or something. But my self-criticism is that nothing really changed, that the American system doesn't lead to real innovation.

So I decided to start a company with your friend Sebastian Thrun, who was a faculty member here, and a bunch of Stanford people. The idea is to basically do two things: use artificial intelligence in sophisticated and powerful ways in these wars, which are essentially robot wars, and the second thing is to reduce the cost of robots. Now you might be thinking, why would a well-meaning libertarian like me do this? The answer is that the whole theory of the military is tanks and artillery and mortars, and we can eliminate them. We can make it essentially impossible to invade a country, at least by land, with penalties. It should eliminate that kind of land warfare.

professor:

That's a very interesting question. Does it give the defense an advantage over the offense? Can you tell the difference?

Schmidt:

Because I've been doing this for the last year, I've learned a lot about war that I really don't want to know. One thing you need to know about war is that there's always an advantage to the offense because you can always overwhelm the defense. So from a defense strategy perspective, you better have a very powerful offense that you can use when you need to. The system that I and others are building will do that. Because of the way the system works, I'm now a licensed arms dealer. So I'm a computer scientist, businessman, arms dealer. I'm sorry to say that. Is it an improvement? I don't know. I don't recommend you do this in your career. I'll stick to AI. Because of the way the law works, we do this privately, with the support of the government, and it's all legal. It goes right into Ukraine, and then they start a war. Without going into all the details, it's pretty bad. I think in May or June, if the Russians build up as they're supposed to, Ukraine will lose a chunk of territory in the process of losing the entire country. It's pretty bad. If you know Marjorie Taylor Greene, I recommend you remove her from your contact list. Because she's the one who blocked billions of dollars in funding to save an important democracy.

professor:

Next, I want to discuss a somewhat philosophical question. Last year, you co-authored an article with Henry Kissinger and Dan Hertenloch about the nature of knowledge and how it develops. I discussed this question the other night as well. For most of history, humanity's understanding of the universe was mysterious until the scientific revolution and the advent of the Enlightenment. In your article, you mentioned that models are now becoming so complex and incomprehensible that we don't really know what's going on inside them. I'm quoting Richard Feynman: "What I can't create, I don't understand." I saw this quote the other day. But now people are creating things that they can create without really understanding their inner workings. Has the nature of knowledge changed in some way? Do we have to start taking these models at face value and not explaining them to us?

Schmidt:

I want to use the example of a teenager. If you have a teenager, you know they're human, but you can't quite figure out what they're thinking. Yet we've managed to accommodate teenagers in society, and they'll eventually grow out of that. It's a serious problem. So we might have knowledge systems that we can't fully describe, but we understand their boundaries and the limits of what they can do, and that's probably the best we can get. Do you think we'll ever learn about those limits? If we can do that, that's great.

The consensus of my group in weekly meetings is that eventually there will be what is called adversarial AI, where there will actually be companies that hire you and pay you to break your AI system. Just like red teams. Instead of today’s human red teams, you’re going to have entire companies and an entire industry of AI systems whose job is to break existing AI systems and find vulnerabilities in them, especially the ones that we can’t figure out. That makes sense to me. It’s also a great project for Stanford. If you have a grad student who has to figure out how to attack one of these large models and understand what it does, that’s going to be an important skill to build the next generation. So it makes sense to combine the two.

professor:

Now, let's answer some questions from the students. There is a student in the back, please tell me your name.

student:

You mentioned earlier, and this is related to the current comment, getting AI to actually do what you want. You mentioned adversarial AI a little while ago, and I wonder if you could elaborate on that a little bit more. It seems like, aside from the fact that computing power obviously will increase, and you can get higher performance models, the question of getting them to do what you want, seems to be partially unanswered.

Schmidt:

Well, you have to assume that the current hallucination problem will decrease as the technology improves and so on. I'm not saying it will go away. And then you also have to assume that there is efficacy testing, so there has to be a way to know if this thing is successful. In the example I mentioned about the TikTok competitor, I was not suggesting illegally stealing other people's music. What would you do if you were a Silicon Valley entrepreneur? Hopefully, you are all Silicon Valley entrepreneurs. If your product is successful, you hire a bunch of lawyers to deal with the follow-up. But if no one uses your product, then it doesn't matter if you stole all the content. Of course, don't quote me on that.

Silicon Valley typically does these tests and handles the follow-up. That's common practice. I think you're going to see more and more performance systems and even better tests, and eventually adversarial testing, which will keep it within a framework. The technical term for this is called chain of thought reasoning. People believe that in the next few years, you will be able to generate a thousand-step chain of thought reasoning, like making a recipe. You can run it and actually test whether it produces the right result, and that's how the system works.

student:

You're very optimistic about the potential for advancement in AI in general. I'm curious, what's driving that advancement? Is it more computing power? Is it more data? Is it a fundamental or practical shift?

Schmidt:

The answer is all of the above. The amount of money being invested is incredible. I invest in basically everything because I don’t know who’s going to win and the amount of money I follow is so huge. Part of it is that the early money has been made and the ones that don’t know as well must have an AI component. Everything is AI investing now and they can’t tell the difference.

I define AI as learning systems, systems that really learn. I think that's one of them. The second point is that there are some very sophisticated new algorithms coming out now that are a little bit post-Transformer. I have a friend who is also my long-term partner who invented a new non-Transformer architecture. A group I funded in Paris claims to have done the same thing. There is a lot of invention there, and there is a lot of research at Stanford. The last point is that the market believes that there is an infinite return on the invention of intelligence. Let's say you put $50 billion of capital into a company, you have to make a lot of money from intelligence to pay it back. We may go through some huge investment bubbles and then it will resolve itself. It has been true in the past and it may be true now.

professor:You mentioned earlier that leaders are distancing themselves from others.

Schmidt:

Now, there's a company in France called Mistral, and they're doing really well. I'm obviously an investor. They've already made a second version, and their third model will most likely be closed because it's too expensive. They need the revenue and can't give their model away for free. The debate in our industry about open source vs. closed source is very intense. My entire career has been built on people being willing to share software in an open source way. Everything I've done is based on open source. Much of Google's foundation is built on open source. The work I've done has been mostly in the technology space. However, the huge capital costs could fundamentally change the way software is built.

My view of software programmers is that they will at least double in productivity. There are three or four software companies that are trying to do this right now, and I've invested in all of them over the time. They're all trying to make software programmers more productive. I recently came across a very interesting company called Augment. I often think of a programmer and they say that's not our goal. Our goal is those 100-person software programming teams that have millions of lines of code and nobody knows what's going on. This is a really great application of AI. Will they make money? I hope so, but there are a lot of questions here.

student:

At the beginning, you mentioned that the combination of context window extensions, proxies, and text-to-action will have incredible impact. First of all, why is this combination important? Secondly, I know you are not a prophet and cannot predict the future, but why do you think it is beyond our imagination?

Schmidt:

I think it’s mainly because context windows allow you to solve the recency problem. Current models take about 18 months to train, six months of preparation, six months of training, and six months of fine-tuning, so they’re always out of date. With context windows, you can feed in what’s happening recently and ask questions about the Hamas-Israel war in context, which is very powerful and makes it as up-to-date as Google.

In the case of an agent, I can give you an example. I started a foundation that funds a nonprofit. I don't know much about chemistry, but there's a tool called ChatCrow, which is a system based on a large language model that learns chemistry. They run this system to generate chemical hypotheses about proteins, and then the lab tests them overnight, and the system learns. This is a huge accelerator for fields like chemistry, materials science, and so on. This is an agent model.

I think the concept of text-to-action was understood as long as there were lots of cheap programmers. I don't think we understand what happens when everyone has their own programmer. This is also your area of ​​expertise. I'm not talking about simple tasks like turning a light on and off. I imagine another example, let's say you don't like Google, you could say, build me a competitor to Google. Yes, you personally could do that. Build me a competitor to Google, search the web, build a UI, make a good copy, and add in interesting ways.Generative AI. Do it in 30 seconds and see if it works. A lot of people think that incumbents, including Google, are vulnerable to this attack.

professor:

Now, let's take a look. Slido sent a number of questions, some of which have been uploaded. Last year we discussed how to stop AI from influencing public opinion and spreading misinformation, especially during the upcoming election.

Schmidt:

We need to think about both short-term and long-term solutions. In the upcoming global elections, most of the misinformation will appear on social media, and social media companies are not currently organized enough to effectively regulate this information. For example, TikTok has been accused of favoring certain kinds of false information, although I have no evidence. I think we have a mess on our hands.

The nation needs to learn critical thinking, and that can be a difficult challenge for the U.S. Just because someone tells you something doesn't mean it's true.

professor:

Are we going so far that some things are no longer believed to be true? Some people call this an epistemological crisis. Now, Elon Musk says he never did something, but how can he prove it?

Schmidt:

We can use the example of Donald Trump to illustrate this. I think we have a trust problem in our society and democracy may fail because of it. The biggest threat to democracy is misinformation because we have become very good at it.

When I ran YouTube, the biggest problem we had was people uploading fake videos where people died. We had a no-death policy to try to fix that, which was shocking and scary. This was before generative AI.

professor:

I don't have a good answer, but there is a technology that seems to mitigate this, and that is public key authentication. When Joe Biden speaks, why not use digital signatures like SSL? Can celebrities, public figures, or others have public keys?

Schmidt:

It's a form of public key that provides a certainty, just like when I send my credit card to Amazon, I know it's Amazon.

I did a paper with Jonathan Haidt that didn’t have an impact. He’s a very good communicator, and I’m probably not. My conclusion is that the system is not organized the way we say it should be. CEOs are generally maximizing revenue, and to do that they maximize engagement, and the way to maximize engagement is to maximize anger. The algorithm selects anger because it generates more revenue, so people tend to support extreme things. This is a problem on all fronts, and it has to be addressed.

In a democracy, my solution for TikTok is based on something we've discussed privately before. When I was a kid, there was something called an equal time rule. TikTok is not really a social media platform, but more like a form of television. The average TikTok user in the United States uses the app for 90 minutes a day and makes 200 videos, which is a lot of usage. While the government doesn't implement an equal time rule, it might be a direction worth considering, and some form of balance is needed.

student:

First, the economic impact on the labor market. The impact has been slower than initially expected, especially on the labor market. And then there's the question about customer service. Do you think academia should get subsidies for AI, or should it work with large companies?

Schmidt:

I've been pushing universities to build data centers. If I were a computer science faculty member, I'd be frustrated that I couldn't work with graduate students to develop algorithms for my doctoral research because I was forced to work with companies. And the companies aren't very generous in this regard. A lot of faculty spend a lot of time waiting for Google Cloud credits, and that's terrible. We want American universities to be successful in this regard, so I think it's the right thing to do to allow them to get these credits.

Regarding the labor market impact, I would listen to the real experts. As an amateur economist, I believe college education and high-skill tasks will have a good future as people use these systems. I don't think these systems are fundamentally different from previous waves of technology. Dangerous jobs and jobs that don't require human judgment will be replaced.

student:

What about the shift from text to action and its impact on computer science education? I think computer science education should adapt to the changing times.

Schmidt:

I assume that computer scientists in undergraduate programs always have a programmer partner. When you learn your first for loop, there is a tool that becomes your natural partner. The professor will explain the concepts, and you will participate in it that way.

Student: Regarding the discussion of non-Transformer architectures, I think state models are a direction that is discussed, but now there is more focus on context.

Schmidt:

I don't understand the math well enough, but I'm glad this is creating jobs for mathematicians because the math here is very complex. Basically, these areGradient DescentAnd a different way to do matrix multiplication, with the goal of being faster and better. As you know, Transformers is a systematic way to do multiplication simultaneously. Here's my idea. It's similar to this, but the math is different. Let's take a look.

student:

In your paper on national security, you talk about the situation today with China, the United States, and other countries. The next ten countries down from the next cluster are either allies of the United States or have the potential to become allies of the United States. I'm curious about your thoughts on these ten countries. They are kind of like middlemen, not formal allies. How likely are they to join in the effort to keep us safe? What's stopping them from joining?

Schmidt:

The most interesting country is India, because the top AI talent is coming from India to the United States. We should let India keep some of the top talent, not all, but some. And they don't have the rich training facilities and programs that we have here. In my opinion, India is a swing country in this regard. Japan and South Korea are clearly in our camp. Taiwan software is terrible, so this won't work. The hardware is great. And in the rest of the world, there aren't a lot of other good options. Europe is screwed because of Brussels, this is nothing new. I spent 10 years fighting them. I fought very hard to get them to change the EU Act. They still have all kinds of restrictions that make it very difficult for us to do research in Europe. My French friends spend all their time fighting Brussels. Macron, who is a personal friend of mine, is fighting hard for this. So I think France has a chance. I don't think Germany will come, and other countries are not strong enough.

student:

I know you're an engineer by training, I think you're called a compiler. Given the capabilities that you envision these models to have, should we still spend time learning to code?

Schmidt:

Yes, because at the end of the day, it's the same old question, why would you learn English if you can speak it? You'll learn it better. You do need to understand how these systems work, and I've experienced that.

student:

I'm curious if you've explored distributed settings. I ask this because, of course, it's hard to make a large cluster, but MacBooks are powerful. There are a lot of small machines around the world. So do you think that Fold at Home or similar ideas would be applicable to training these systems?

Schmidt:

Yes, we've looked at this very carefully. So the way the algorithm works is you have a very large matrix and you have essentially a multiplication function. So think of it as going back and forth. And these systems are completely limited by the speed of memory to the CPU or GPU. In fact, the next generation of NVIDIA chips have all of these functions packed into a single chip. The chips are so large now that they're all glued together. In fact, the packaging is so sensitive that both the packaging and the chip itself are assembled in a clean room. So the answer looks like supercomputers and the speed of light, especially the memory interconnect, really prevail. I think it's not possible to segment large language models (LLMs) at this time.

professor:

Jeff Dean mentioned in a speech last year that the model can be divided into different parts, trained separately and then combined.

Schmidt:

But to achieve this, you need to have tens of millions of these models, and the speed of asking questions will become very slow. He mentioned that you need 8, 10 or 12 supercomputers to achieve this goal, but it is not at his level.

student:

Regarding the issue of data privacy, I learned that the New York Times sued OpenAI for using their work for training.

Schmidt:

I think there will probably be a lot of lawsuits like this in the future, and eventually some kind of agreement will be reached, like ASCAP and EMI in the music industry that require a certain percentage of the revenue to be paid for the use of certain works. This model may seem a bit outdated, but I think it will eventually work this way.

student:

In the AI ​​space, there appear to be a few companies that dominate the market and overlap with the large companies that antitrust regulations focus on.

Schmidt:

I worked on Microsoft's breakup in my career, which was unsuccessful, and I worked on Google's unsuccessful breakup. So I think the trend is toward no breakup. As long as these companies don't become monopolies like John Rockefeller, the government is unlikely to take action.

These big companies dominate because they are the only ones who have the capital to build data centers. My friends Reed and Mustafa made the decision to spin off their business to Microsoft because they couldn't raise tens of billions of dollars. As for the specific numbers, you may need to let Reed tell you.

student:

Finally, I wonder what impact these developments will have on countries that are not involved in cutting-edge model development and calculations.

Schmidt:

Rich countries will get richer, while poor countries can only do their best. This is actually a game for rich countries, which requires huge capital, technical talents and strong government support. Globally, many countries are facing various problems, especially when resources are scarce. They need to find partners and work with others to solve these problems.

professor:

I remember the last time we met, you wereAGI House to participate in hackathons. I know you spend a lot of time helping young people create wealth and are passionate about it. What advice do you have for people who are writing business plans for classes or writing policy proposals or research proposals in their careers?

Schmidt:

I teach a course on this at the business school and you should come and listen. I am amazed at the speed at which you present new ideas.

At one hackathon I attended, the winning team was tasked with flying a drone between two towers. They generated code in Python in a virtual drone space and successfully completed the task in a simulator. A good professional programmer might take a week or two to do this. I think the ability to prototype quickly is very important because part of the problem entrepreneurs face is speed. If you can't prototype in a day with these tools, you need to rethink it because that's what your competitors are doing.

So my biggest advice is that it's OK to write a business plan when you start thinking about starting a company. In fact, you can have a computer write it for you, as long as it's legal. It's very important to prototype your idea as soon as possible using these tools, because there may be people doing the same thing in other companies, universities, or places you haven't been.

professor:Thank you so much.