GPU inventory will reach 600000! Zuckerberg Confirms New Goal: Creating General Artificial Intelligence

On January 19, it was reported that in the field of science and technology, an upsurge was sweeping. It originated from a firm belief that the science and technology industry was on the way to achieving superhuman and divine intelligence. Flagmen leading this trend include OpenAI , they claim to create this universal artificial intelligence （AGI）。 and Google Demis Hassabis, the director of AI project, also has the same dream.

Today, Meta CEO Mark Zuckerberg (Mark Zuckerberg) also jumped into this fierce battle field. Although he did not give a specific timetable for the realization of general AI, or even a clear definition, he is still determined to create and open source general AI. At the same time, Zuckerberg also integrated the artificial intelligence research team FAIR of Meta and merged it with the team responsible for building generative AI products in Meta applications. Meta's ambition is clear: they hope that every breakthrough in AI will directly reach billions of users.

In an interview, Zuckerberg said: "We have reached a consensus that in order to build the product we have always dreamed of, we must work hard for general intelligence. I think it is very important to convey this, because many top researchers are eager to challenge more ambitious goals."

However, this feast of artificial intelligence is not for everyone. The battle for talent behind the scenes has been smoldering. Every technology giant is struggling for a limited number of researchers and engineers. Those who hold scarce professional knowledge can easily obtain amazing annual salary, even up to one million dollars. Industry leaders like Zuckerberg have to step down in person to attract key talents, or do their best to prevent their research team from being poached by competitors.

Zuckerberg sighed: "We have been used to this extremely fierce competition for talent. But the situation here is different. Many companies are pursuing the same goal, and a large number of venture capital and other funds are constantly pouring into this field, which makes it easy for people to try different things from the outside."

In addition to competing for talents, the most scarce resource in the field of artificial intelligence is the computing power required to train and run large-scale models. In this regard, Zuckerberg seems to have made adequate preparations. He revealed that by the end of this year, Meta will have more than 340000 Nvidia H100 GPUs - they are the industry's preferred chips for building generative AI.

External research shows that in 2023 alone, Meta will purchase 150000 H100, which is comparable to that of Microsoft, at least three times that of other companies. Zuckerberg further said that if Nvidia's A100 and other AI chips are included, Meta's GPU inventory will reach nearly 600000 by the end of 2024. This is undoubtedly an eye popping figure, which demonstrates Meta's ambition and strength in the field of artificial intelligence.

Zuckerberg said: "We have built such a capability, which may be larger than any other company. I think many people may not realize this."

Training Llama 3

Although scientific and technological leaders, including Zuckerberg, are full of expectations for general artificial intelligence, an embarrassing fact is that so far, general artificial intelligence still lacks a clear and unified definition, and its implementation schedule is still unknown.

Zuckerberg said frankly: "It is not easy to give a simple and clear definition of general artificial intelligence. Some people may argue that general artificial intelligence should reach the level of human intelligence, or surpass humans, or it is some kind of super intelligence in the distant future. But for me, the key lies in its wide applicability - that is, intelligence can have many different abilities, including reasoning and intuition. "

He believes that the arrival of general artificial intelligence will be a gradual process, rather than an overnight success. Zuckerberg said, "In fact, I'm not sure whether there is a specific threshold that can make people feel 'Ah, this is general artificial intelligence!'"

As Zuckerberg explained, the large language model Llama 2 released last year has attracted more attention from Meta to general artificial intelligence. Although the company believes that the ability of this model to generate code is not significant for ordinary users to use large language models in Meta applications, this function is critical for creating more intelligent AI. Therefore, Meta still decided to invest resources in development.

Zuckerberg explained: "At first, we may think that programming is not so important, because not many people will ask programming questions on WhatsApp. But in fact, programming is extremely important in structure, because it enables large language models to understand the rigor and hierarchy of knowledge, and gives them a stronger sense of intuitive logic."

He revealed that Meta is currently training Llama 3 and plans to add code generation function to it. Similar to Google's multimodal model Gemini, another focus of Llama 3 is to improve reasoning and planning capabilities.

When talking about Llama 2, Zuckerberg said: "Although Llama 2 is not the industry leading model, it is the best open source model at present. With Llama 3 and more advanced models, our goal is to create the most advanced products and eventually become the industry leader."

It can be predicted that with the continuous investment and exploration of technology giants such as Meta in the AGI field, we will be closer to the mysterious and seductive AGI era. In this process, both the discussion on the definition of AGI and the exploration of its realization path will reveal more mysteries about the essence of intelligence and the future of mankind.

Open source and closed source

The topic of who will ultimately control general artificial intelligence has always been the focus of intense debate in the industry. The recent near collapse of OpenAI has made this discussion even more popular.

As the helmsman of Meta, Zuckerberg, by virtue of his absolute voting control over the company's shares, sat firmly at the center of the storm. If AGI really becomes a reality, its status will undoubtedly be amplified, and even may bring unknown risks. His response is that Meta always adheres to the Llama strategy. At least in most cases, it can be regarded as open source.

Zuckerberg said frankly: "I am convinced that a bigger challenge is that when you create something truly valuable, it may eventually become highly concentrated. But if we can make it more open, then we can solve a series of problems caused by inequality of opportunity and value. This is also an important part of our entire open source vision."

Although there is no direct roll call, Zuckerberg's words clearly reveal the contrast with OpenAI. OpenAI's original goal was to open source its models, but over time, the company is becoming increasingly opaque. Zuckerberg criticized: "These companies used to be open. They used to disclose all their work and talk about how they would be open. But now, they seem to realize the value of all this and start to choose not to share it anymore."

Zuckerberg believes that the security benefits brought by the more closed AI development mode supported by Sam Altman and others are more like a smart business game. At the same time, he also pointed out that the currently deployed model has not caused catastrophic consequences.

He further pointed out: "In many cases, those leading big companies are also the most popular. They advocate setting various restrictions on how others build artificial intelligence. I believe that some of them have reasonable concerns about security, but this is also closely related to their strategic interests."

Of course, Zuckerberg has his own considerations. His open vision of AI may eventually lead to another form of centralization of power. Meta has more users than almost any company on the earth, and its social media business is also profitable. The AI function can undoubtedly make its platform more attractive and more practical. If Meta can help effectively regulate the development of AI by publishing its models publicly, then its impact on the entire ecosystem will be more far-reaching.

However, an outstanding question is: if Meta implements general AI, will it really open its source code? This ultimately depends on Zuckerberg's decision. At present, he has not given a clear commitment.

He said cautiously: "As long as we think it is meaningful, safe and responsible, we tend to open source. But obviously, we don't want to be tied down because we said we would do something."

Meta focus has not shifted

In Meta's grand blueprint, Zuckerberg seems slightly embarrassed about the timing of promoting general artificial intelligence. It has only been two years since he changed his company name to Meta and made a full bet on the Metauniverse. Although the smart glasses jointly launched by Meta and Ray Ban have begun to shine, it seems that the truly mature augmented reality (AR) glasses are still far away. At the same time, the release of Apple Vision Pro undoubtedly confirmed the technology giant's firm bet on hybrid reality, although this field still seems to be too lofty.

Zuckerberg is obviously unwilling to regard his attention to AI as the turning point of the company's strategy. He firmly said: "We will continue to dig deep into the real laboratory and the universe." It is reported that Meta still spends more than 15 billion dollars on these two projects every year. Recently, Ray Ban smart glasses have introduced visual artificial intelligence assistant, which has complete functions such as object recognition, language translation, etc. Zuckerberg firmly believes that generative AI will play an important role in Meta's future hardware innovation.

In Zuckerberg's eyes, the future virtual world will be created by AI, full of lifelike AI characters. He revealed that a new platform will be launched this year to enable everyone to create their own AI roles and integrate them into Meta's social applications. He even imagined that these AI characters might be able to independently publish content on Facebook, Instagram, Thread and other platforms.

Today's Meta is not only a leader in the field of the metauniverse, but also the world's largest social media company. However, it does not stop here, but is moving towards a new journey of general artificial intelligence. Zuckerberg put all this closely around the core mission of "building a connected future".

So far, this connection is mainly limited to the interaction between people. But Zuckerberg foresees that people will have more frequent conversations with AI in the future. Obviously, he is convinced that this future is not only inevitable, but also full of expectations, whether we are ready for its arrival or not. (small)