Energy consumption black hole in the era of artificial intelligence
It is estimated that by 2027, the AI industry will consume 85~134 TWh of electricity every year, equivalent to the total electricity consumption of Sweden or the Netherlands in a year.
Musk judged that the power gap may occur as early as 2025, "next year you will see that we do not have enough power to run all the chips".
Huang Renxun also worried about power supply, but gave a more optimistic outlook: in the past 10 years, computing and artificial intelligence have increased by 1 million times, while the cost, space or energy consumed by it has not increased by 1 million times.
Author: Wei Linhua
Source: Xuebao Finance Agency
At the end of March, Holtec Palisades, a closed nuclear power plant near Lake Michigan in the United States, received a loan guarantee of US $1.5 billion from the US Department of Energy and entered the restart phase. If the supervision link is successfully passed, it will become the first restarted nuclear power plant in American history.
The reason why the US Department of Energy restarted nuclear power generation was due to concerns about the imbalance of power demand. While the power consumption demand of manufacturing industry, electric vehicles and other industries is rising, the rapid development of artificial intelligence industry has accelerated the arrival of the power crisis in the United States.
"AI itself is not a problem, because it can help solve the problem." Jennifer Granholm, the US Secretary of Energy, said in an interview with Axiro. However, the growing demand for power from AI and data centers has become a real new problem.
Small app, electricity eating monster
How much power is consumed by AI applications?
In his paper, Dutch scientist Alex DeVries calculated such an account for the dialogue robot ChatGPT:
Every time ChatGPT tries to respond to a problem, it consumes 2.9 watt hours of power. What is this concept? Responding to 10 times of power, it can support a 15W LED bulb to work for 2 hours; After 100 responses, the smartphone can be charged about 13 times.
In one day, ChatGPT needs to process about 200 million conversation requests from users on average, which means that it consumes more than 564 megawatt hours of electricity in a single day (1 megawatt hour=1000 kilowatt hours, 564 megawatt hours is equivalent to 564000 kilowatt hours of electricity). Based on the average daily power consumption of each household in the United States, ChatGPT needs to consume 17000 American households a day.
Due to differences in model parameters, energy consumption treatment and other factors, the power consumption of different AI models cannot be accurately estimated. Therefore, Alex uses the A100 server launched by Nvidia as the measurement object to estimate the possible power consumption of the entire AI industry.
Based on his assumption, NVIDIA may launch 1.5 million A100 servers by 2027, 95% of which will be used in the AI industry. Each DGX A100 server is equipped with 8 A100 chips. Based on the power consumption of 11.4 million A100 chips, the annual power consumption of the entire AI industry will reach 85~134 TWh (1 TWh=1 × 10 ⁶ kWh) in 2027.
That is to say, by 2027, the power consumption of AI may approach the total power consumption of Sweden with a population of more than 10 million or the Netherlands with a population of 17 million, equivalent to 0.5% of the current global power consumption.
According to this estimate, the power consumption of AI may be comparable to that of Bitcoin mining. According to the calculation of Cambridge University, the annual power consumption of Bitcoin mining is about 165.99 TWh, which is close to the annual power consumption of Egypt with a population of 100 million.
The power consumption of Bitcoin mining is determined by its working mode. In the design of Nakamoto Satoshi, the father of Bitcoin, the Bitcoin system enables miners to race to calculate a hash value (a string composed of numbers and letters) that is difficult enough to create new blocks and obtain rewards through Proof of Work. This competitive computing process consumes a lot of power and computing power.
The reason why AI can eat electricity so much is that the training and reasoning process of the large model need to consume a lot of electricity.
The key to the quality of the large model lies in data, computing power and top talents. Behind the high computing power is the continuous operation of tens of thousands of chips around the clock.
GPU (graphics processor) is proved to be more suitable for AI training hardware than CPU (central processing unit) usually installed on notebook computers. If the CPU is regarded as a single task processing component, the advantage of GPU is to process multiple concurrent tasks at the same time. Although the GPU was not born to deal with AI needs at first, it also handles the characteristics of multitasking, making it a ticket to enter the AI big model training ground.
Compared with CPU, GPU can handle multiple parallel tasks
Source: Nvidia official website
The price of fast is higher energy consumption. It is estimated that a GPU consumes 10-15 times more energy than a CPU. In the process of large model training, multiple GPUs are required to operate continuously. The larger the model parameters and data, the greater the power consumption of training.
Taking GPT-3 training as an example, the Artificial Intelligence Index Report 2023 released by Stanford Institute of Artificial Intelligence shows that the power consumption of GPT-3 with 175 billion parameters in the training phase is up to 1287 MWh. Liu Jie, the president of the Artificial Intelligence Research Institute of Harbin Institute of Technology, made an analogy, which is equivalent to driving from the earth to the moon.
After completing the training, the power consumption of AI in reasoning far exceeds that of training.
Each time a request is responded to, the big model needs to complete the reasoning process to find the closest solution to the problem. According to the above data, the power consumed by GPT-3 in the training phase can not even support the operation of ChatGPT for three days.
With the development of multimodal large models into the mainstream, the power consumption will be further improved in the reasoning process of AI response requirements. According to the research of the artificial intelligence company Hugging Face, not only do multimodal large models consume far more power than general models, but also models involving image processing consume more power than pure text processing.
Specific to different tasks, simple tasks such as text classification, marking and question and answer are relatively low consumption, and thousand times of reasoning only needs 0.002~0.007 kilowatt hours. When responding to multimodal tasks, the maximum energy consumption for text to image generation is 2.9 kWh, which is equivalent to the power consumption of 100 ChatGPT responses.
The giant's AI dream has torn the power gap even bigger
From GPT-2 training with 1.5 billion parameters to GPT-3 training with 175 billion parameters, it only took one year for Open AI to leap from 1 billion parameters to 100 billion parameters.
When the big model is soaring, more and more large technology companies begin to put the integration of AI and the company's main business on the agenda.
Google tries to combine AI functions in search, but its energy consumption is amazing. In February last year, John Hennessy, chairman of Alphabet, Google's parent company, said that the cost of using AI in search would be 10 times that of ordinary search.
According to the aforementioned Report on Artificial Intelligence Index 2023 released by Stanford Institute of Artificial Intelligence, the power consumption of each AI search is about 8.9 watt hours. Compared with the power consumption of Google's single search of 0.3 watt hours, the power consumption of a single search with AI is almost 30 times that of a general search.
Microsoft, which cooperates closely with Open AI, also plans to "squeeze" AI into its main product lines, such as Office software, Windows operating system, Bing search engine, Azure cloud services, etc.
In order to provide more sufficient computing power to support the training and use of the AI large model, the construction of the data center, the infrastructure of the base, has been included in the next step of planning by technology enterprises.
In 2023, Google will spend more than 2.5 billion dollars to build data centers in Ohio, Iowa and Mesa, Arizona. Amazon, which is optimistic about AI development, plans to invest 150 billion dollars in the next 15 years to build a data center.
When the expanding power demand cannot be met one by one, the power of some cities in the United States has sounded an urgent alarm.
The United States has the largest number of data centers in the world. By 2022, the United States has more than 2300 data centers, accounting for 1/3 of the global data centers.
Among them, cloud computing giants including Amazon, Microsoft, Google, etc. have a particularly large data center layout in the United States. According to Synergy Research Group, Amazon, Microsoft and Google together account for more than half of all major data centers among super large operators. Microsoft has 24 zones in the United States, and one zone is equipped with three or more data centers.
According to the prediction of the International Energy Agency (IEA), the power consumption of the US data center will grow rapidly in the next few years. IEA warns that in 2022, the power consumption of the US data center will account for more than 4% of the total power in the US. By 2026, its power consumption will rise to 6%, and will continue to expand in the next few years.
However, contrary to the rapid growth of AI power demand, the power generation in the United States shows no obvious signs of growth.
According to the US Energy Information Administration, in 2023, the US full caliber net power generation will be 4178.171 billion kWh, down 1.2% from the previous year. In fact, in the past decade, the annual net power generation of the United States has been hovering at the edge of 400 billion kilowatt hours.
Change in net electricity generation in the United States from 1950 to 2023 (unit: billion kWh)
Source: Statista
One of the culprits of the power shortage in the United States is its fragile power grid transmission facilities. The power grid infrastructure in the United States, such as transformers and transmission lines, was built from the 1960s to the 1980s, with obvious circuit aging problems. The White House pointed out in a document in 2022 that many transformers and transmission lines are approaching or exceeding their design life, and 70% of transmission lines in the country have been used for more than 25 years.
Under the aging grid infrastructure, the idea of the United States to transmit electricity from other regions and connect clean energy to expand the grid reserve energy cannot be realized. A report issued by the Department of Energy (DOE) pointed out that the transmission system built in the United States has been faced with full load in Texas, Alaska and other regions.
In order to strengthen the flexibility and reliability of the power grids in the US states, the US Department of Energy announced last year that it would invest US $3.46 billion in 58 projects in 44 states.
The power crisis is at hand. In the near future, it may also become a key factor restricting the development of AI.
In February 2024, at the World Economic Forum in Davos, Sam Altman, CEO of Open AI, mentioned the power crisis brought by AI. In his view, AI will consume far more power than people expected. "We have not fully realized the energy demand of AI. Without a major breakthrough, we cannot achieve this goal (to AGI)."
On the Bosch Internet Forum, Tesla CEO Mask also emphasized the development dilemma of AI. "The next shortage will be power." He judged that the power gap could occur as early as 2025. "Next year you will see that we do not have enough power to run all the chips."
Clamp and way out
The overburdened power grid has begun to restrict the business expansion of technology enterprises.
On social media X, Kyle Corbitt, the founder of OpenPipe, an open source community, shared his conversation with Microsoft engineers. They mentioned the transmission difficulties faced by GPU in different states during the training of GPT-6 by Open AI.
"It is impossible for us to put more than 100000 H100 chips into a state without damaging the power grid." The maximum power consumption of an H100 is 700 watts. According to the estimation of Microsoft engineers, based on the annual utilization rate of 61%, the power consumption of 100000 H100 will be as high as 42 MWh.
In order to meet the soaring demand for electricity, the goal of reducing carbon emissions was first sacrificed.
According to the Washington Post, the growth of electricity demand in many places in the United States exceeded expectations. Taking Georgia as an example, it is estimated that the new power consumption in the next decade will be 17 times of the recent one. Coal fired power plants in Kansas, Nebraska, Wisconsin and South Carolina have decided to postpone retirement.
In the face of power hungry miners, different countries have introduced different levels of regulatory policies. The US Department of Energy estimates that the annual power consumption of cryptocurrency mining may account for 0.6% to 2.3% of the US power consumption. For this reason, the United States considered levying up to 30% of the energy consumption tax on digital asset mining on cryptocurrency mining business. Canada has announced a ban on mining cryptocurrency in three provinces.
AI has also attracted the attention of regulators. Since the energy consumption of each AI enterprise is difficult to be estimated quantitatively. Overseas regulators began to promote legislation, requiring AI development enterprises to disclose the use of energy to reasonably estimate the impact of AI on energy consumption.
In March this year, the Artificial Intelligence Act approved by 27 EU member states required "high-risk artificial intelligence systems" to report their energy consumption and resource use.
A few years ago, the man at the helm of science and technology enterprises bet on new energy companies to support huge power demand with clean renewable energy.
In 2021, Altman, CEO of Open AI, will invest 375 million dollars in Helion Energy, a nuclear fusion startup. In May 2023, Microsoft signed a power purchase agreement with the company, which is expected to purchase 50 MW of power from it from 2028. The bad news is that it is not even enough to support 1/25 of the power consumption of GPT-3 training.
Through technical optimization of performance, energy consumption can also be significantly reduced.
At this year's GTC conference, Nvidia CEO Huang Renxun brought a new GPU product Blackwell. By using the new architecture, its energy consumption has been reduced by more than 70%: to train a GPT model with 1.8 trillion parameters, the traditional method may require 8000 GPUs and 15 MW, lasting 90 days. Blackwell only needs 2000 GPUs and consumes 4 MW.
Compared with Mask and Altman's warning words, Huang Renxun is also worried about the supply of electric energy, but he gave a more optimistic outlook: "In the past decade, we have increased computing and artificial intelligence by 1 million times... but the cost, space or energy consumed by it has not increased by 1 million times."
BlackWell released by Nvidia at 2024GTC
Source: Nvidia official website
Write at the end
More than a century ago, the energy revolution changed people's way of life. From the fire power of burning wheat straw to coal and oil, in the critical period of historical development, people's exploration of new energy has promoted the process of the industrial revolution.
"Every coal basket contains power and civilization," said Emerson, an American thinker and writer.
The scarcity of one kind of energy often becomes the driving force for mining a new kind of energy. In the book "Love and Hate of Black Stone: The Story of Coal", the author Barbara Fritz tells the story of the "timber crisis" that took place in England in the 16th century.
"Due to the continuous expansion of the city, the forests in nearby counties are gradually cut down, and people have to transport timber from more and more distant places... Brewers in London alone burn 20000 trucks of wood every year." When the price of wood rises faster than inflation and becomes a scarce resource, the domestic coal consumption in Britain increases dramatically.
The exploitation and use of energy has become the key to industrial development. Sufficient coal supports the development of the textile industry and steel industry in Britain, making it the center of the first industrial revolution, while the exploitation of oil drives the prosperity of automobile, aircraft and other industries.
Under the crisis of fossil energy depletion, the use of new energy can not only alleviate the energy crisis approaching the artificial intelligence industry, but also carry the "power and civilization" of human science and technology to continue to move forward.
reference material
[1] Granholm eyes talks with Big Tech on AI power needs. Axios
[2] Amid explosive demand, America is running out of power. The Washington Post
[3] Nvidia CEO Jensen Huang at World Government Summit.
[4] The AI Act Explorer.
[5] Bitcoin: A Peer-to-Peer Electronic Cash System. Satoshi Nakamoto
[6] A. I. Could Soon Need as Much Electricity as an Entire Country. The New York Times
[7] Cambridge Blockchain Network Sustainability Index: CBECI. CCAF
[8] The Biden-Harris Administration Advances Transmission Buildout to Deliver Affordable, Clean Electricity. The White House
[9] Microsoft, Amazon and Google Account for Over Half of Today’s 600 Hyperscale Data Centers. Synergy Research Group
(Statement: This article only represents the author's view, not Sina.com's position.)
