news

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

Machine Heart Report

Editors: Yali, Dapanji, Du Wei

The debate over open source and closed source has been going on for a long time, and may have reached a new climax now.

When it comes to open source big models, the Llama series has been a typical representative since its birth. Its excellent performance and open source characteristics have greatly improved the applicability and accessibility of artificial intelligence technology. Any researcher and developer can benefit from it, making research and application more extensive.

Now, Meta Llama 3.1 405B is officially released. In the official blog, Meta said: "Until today, open source large language models mostly lag behind closed models in terms of functionality and performance. Now, we are ushering in a new era led by open source."

At the same time, Zuckerberg, the founder and CEO of Meta, wrote a long article to explain the significance of open source to all developers, Meta, and the world. He said that open source is a necessary condition for the positive development of AI. Taking the development of Unix and Linux as an example, open source AI will be more conducive to innovation, data protection and cost-effectiveness.

He also believes that the open source Llama model can build a complete ecosystem to ensure technological progress and not lose advantages due to competition. Meta has a successful open source history, and through the open source AI model, Zuckerberg hopes to promote equal and safe application of global technology.

Original link: https://about.fb.com/news/2024/07/open-source-ai-is-the-path-forward/

The following is the original content:

Zuckerberg injects open source genes into Meta

In the early days of high-performance computing, when major technology companies invested heavily in developing their own closed-source versions of Unix, it was hard to imagine any other way to develop such advanced software.

However, open source Linux gradually became popular: initially because it allowed developers to modify the code freely and was more affordable, but later it became more advanced, more secure, and had a wider ecosystem and supported more features than any closed source Unix. Today, Linux has become the industry standard foundation for cloud computing and the operating system that runs most mobile devices, and everyone benefits from its superior products.

I believe that the development of artificial intelligence will follow a similar trajectory.Today, some tech companies are developing leading closed-source models, but open source is rapidly closing the gap.

Last year, Llama 2 was only comparable to the model one generation older. This year, Llama 3 is already competitive with or even ahead of the industry's leading models in some areas. Starting next year, we expect future Llama models to be the most advanced large models in the industry. Until then, Llama is already leading in openness, modifiability, and cost-effectiveness.

Today, we’re taking the next step — making open source AI an industry standard.We released the first cutting-edge open source AI model, Llama 3.1 405B, as well as the improved Llama 3.1 70B and 8B models. Compared with closed-source models, these open source models are significantly more cost-effective, especially the open source nature of the 405B model, making it the best choice for fine-tuning and distilling small models.

In addition to releasing these models, we are working with multiple companies to expand the broader ecosystem. Amazon, Databricks, and NVIDIA are launching full services that enable developers to fine-tune and distill their own models. Innovators like Groq have built low-latency, low-cost inference services for all new models.

These models will be available on all major cloud platforms including AWS, Azure, Google, Oracle, and more. Companies like Scale.AI, Dell, Deloitte, and more are already ready to help enterprises adopt Llama and train custom models with their own data. As the community grows and more companies develop new services, together we can make Llama the industry standard and bring the benefits of AI to everyone.

Meta is committed to open source AI, and here are the reasons why I think open source is the best development platform, why open sourcing Llama is good for Meta, and why open source AI is good for the world and will be around for the long term.

Open Source AI for Developers

When I talk to developers, CEOs, and government officials around the world, I often hear a few themes:

• We need to train, fine-tune, and distill our own models.Every organization has different needs that are best met by using models of different sizes that are trained or fine-tuned with specific data. On-device and classification tasks require small models, while more complex tasks require large models. Now you can use state-of-the-art Llama models, continue to train them on your own data, and then distill them to the model size that best suits your needs — without us or anyone else seeing your data.
• We need to take control of our own destiny and not be “locked in” by closed source vendors.Many organizations don't want to rely on models they can't run and control themselves. They don't want closed-source model vendors to be able to change the model, change the terms of use, or even stop service altogether. They also don't want to be locked into a single cloud platform that has proprietary rights to the model. Open source makes possible a broad ecosystem of compatible toolchains that you can easily switch between.
• We need to protect our data.Many organizations handle sensitive data that needs to be protected and cannot be sent to closed-source models via cloud APIs. Some organizations simply don’t trust closed-source model vendors with their data. Open source solves these problems because it allows you to run models wherever you want. Open source software is known to be more secure because the development process is more transparent.
• We need a model that is efficient and economical.Developers can run Llama 3.1 405B on their own infrastructure for inference at a cost of about 50% of using closed-source models (such as GPT-4), suitable for user-side and offline inference tasks.
• We want to invest in an ecosystem that will become the long-term standard.Many see that open source is advancing faster than closed models, and they want to build their systems on the architecture that will provide the greatest advantages in the long term.

Open Source AI for Meta

Meta's business model is to build the best experiences and services for people. To achieve this, we must ensure that we always have access to the best technology and not be locked into a competitor's closed ecosystem so that they cannot limit what we develop.

I want to share an important lesson: while Apple allows us to build content on its platform, we are still restricted when it comes to building services. Whether it’s the taxes they put on developers, the arbitrary rules they impose, or all the product innovation they prevent, it’s clear that Meta and many other companies will be able to provide better services to people if we can build the best version of our products and competitors can’t restrict what we build. On a philosophical level, this is a big reason why I believe so strongly in building open ecosystems for the next generation of computers in the areas of AI and AR/VR.

People often ask me if I'm worried about losing technical advantage by open sourcing Llama, but I think that misses the big picture for several reasons:

First, to ensure that we can maintain our technological leadership in the long term and not be locked into a closed source ecosystem, Llama needs to grow into a full ecosystem, including tools, efficiency improvements, hardware optimizations, and other integrations. If only our company uses Llama, this ecosystem will not grow and we will be no better off than a closed source variant of Unix.

Second, I expect AI development to continue to be highly competitive, which means that at any given moment, open sourcing a model will not cause us to lose a huge advantage in the race against the next best model.Llama became the industry standard by remaining competitive, efficient, and open generation after generation.

Third, a key difference between Meta and closed source model providers is that selling access to AI models is not our business model. This means that releasing Llama publicly will not undermine our revenue, sustainability, or ability to invest in research, while closed source providers will be affected. (This is one reason why some closed source providers have been lobbying public administrators against open source.)

Finally, Meta has a long history of success with open source projects. We have saved billions of dollars by sharing our server, network, and data center designs with the Open Compute Project and standardizing our supply chain. We have benefited greatly from the innovation of the ecosystem by open sourcing leading tools such as PyTorch and React. This approach has been very effective for a long time.

Open Source AI for the World

I believe open source is necessary for the future of AI. AI has the potential to increase human productivity, creativity, and quality of life more than any other modern technology, and to advance medical and scientific research while accelerating economic growth.Open source will ensure that more people around the world can reap the benefits and opportunities of AI developments, that power is not concentrated in the hands of a few companies, and that the technology can be more evenly and safely deployed across society.

There is an ongoing debate about the safety of open source AI models. My view is that open source AI will be safer than the alternatives. I think governments will eventually come to the conclusion that they support open source because it will make the world more prosperous and safer.

In the security framework as I understand it, there are two types of harm we need to protect against: accidental and intentional.

• Unintentional harm refers to the possibility that an AI system could cause harm unintentionally while operating. For example, modern AI models could inadvertently give incorrect health recommendations. Or, in future scenarios, there are concerns that models could inadvertently replicate themselves or over-optimize objectives, causing harm to humans.
• Intentional harm occurs when bad actors use AI models with the intent to cause harm.

It’s worth noting that unintentional harms encompass most of the concerns people have about AI — from the impact of AI systems on billions of users to the most truly catastrophic science fiction scenarios. In this regard, the safety benefits of open source are even more significant, as systems are more transparent and can be widely scrutinized.

Historically, open source software has been more secure for this reason. Similarly, using Llama and its security systems like Llama Guard will likely be more secure and reliable than a closed source model. As a result, most discussions about open source AI safety focus on intentional harm.

Our security process includes rigorous testing and red team assessments to see if our models have the potential to cause substantial harm, with the goal of mitigating risk before release. Since these models are open source, anyone can conduct their own testing. We must remember that these models are trained on information that is already available on the internet, so when considering harm, the starting point should be whether the model can cause more harm than information that can be quickly obtained from Google or other search results.

Reasoning about intentional harm would benefit from distinguishing what individuals or small-scale actors can do from what large-scale actors with significant resources, such as states, can do.

At some point in the future, individual malicious actors may be able to exploit the intelligence of AI models to create new types of harm from information already available on the internet. At this point, the balance of power is critical to AI safety.

I think it would be better to live in a world where AI is widely deployed, because that would allow large actors to check and balance small malicious actors. This is also how we manage security on social networks, with more powerful AI systems identifying and blocking less sophisticated actors who often use small-scale AI systems.

More broadly, large institutions will promote the safety and stability of society when they deploy AI at scale. As long as everyone has access to similar generational models, governments and institutions with more computing resources will be able to check and balance malicious actors with fewer computing resources.

When thinking about future opportunities, remember that most of today’s leading tech companies and scientific research are built on open source software. If we collectively invest in open source AI, the next generation of companies and research will have access to it. This includes startups that are just getting started, as well as people in universities and countries that may not have the resources to develop SOTA AI from scratch.

In summary, open source AI represents the greatest global opportunity we can have to use this technology to create economic opportunity and security for all.

Cooperation is steady, open source is far-reaching

In past Llama models, Meta released them for its own development, but didn’t focus on building the broader ecosystem. With this launch, we’re taking a different approach. We’re building teams internally to make Llama available to as many developers and partners as possible, and we’re actively building partnerships so that more companies in the ecosystem can provide unique capabilities to their customers.

I believe the release of Llama 3.1 will be a turning point for the industry where most developers will start using open source technologies, and I expect that approach will start with our open source.

I hope we can work together to bring the benefits of AI to the world.

You can access these models now at llama.meta.com.

mark Zuckerberg