In a recently published commentary, Alex de Vries raises concerns about the future energy demands associated with powering AI tools, suggesting that these demands may surpass the energy consumption of certain small nations.
As we hurtle towards achieving one of mankind’s most remarkable technological feats, we must pause to reflect on the consequences of this advancement.
AI-driven systems not only guzzle vast amounts of data during their training processes but also exhibit an insatiable appetite for electricity when operational. A recent study scrutinized the energy consumption and carbon footprint of several prominent large language models.
Among them, ChatGPT, operating on a colossal array of 10,000 NVIDIA GPUs, was found to be devouring 1,287 megawatt hours of electricity – equivalent to what 121 American homes consume in a year. Google’s AI could soon consume as much energy as Ireland.
As we accelerate our journey towards materializing one of the most significant technological breakthroughs in history, Alex de Vries’ commentary, featured in the journal Joule, underscores the possibility that the energy needs of AI tools could soon eclipse those of some small countries.
“In 2021, Google’s total electricity consumption was 18.3 TWh, with AI accounting for 10%–15% of this total. The worst-case scenario suggests Google’s AI alone could consume as much electricity as a country such as Ireland (29.3 TWh per year), which is a significant increase compared to its historical AI-related energy consumption,” cautioned Vries.
A growing appetite for energy in the AI cector
The advent of generative AI, particularly since OpenAI introduced ChatGPT to the world in late 2022, has sparked a surge in demand for AI chips. NVIDIA, a leading provider of high-end chips, reported record revenue of $16 billion for the quarter ending in July 2023, highlighting the escalating need for AI chips.
Furthermore, an emerging trend shows more companies venturing into chip development to meet the rigorous demands of AI. Google and Amazon have already introduced their proprietary AI chips, and rumors suggest that Microsoft may unveil its own chip hardware soon.
Microsoft, with significant investments in OpenAI, appears to be on the path to either creating its own chips or acquiring a semiconductor company to cater to their AI requirements.
This points to a looming increase in the energy footprint of the AI industry, as Vries elaborates, “For example, companies such as Alphabet’s Google could substantially increase their power demand if generative AI is integrated into every Google search.”
The next energy consumption challenge
According to SemiAnalysis, a leading semiconductor and AI blog, integrating a ChatGPT-like chatbot into every Google search would necessitate 512,820 of NVIDIA’s A100 HGX servers, equivalent to over 4 million GPUs. With a power demand of 6.5 kW per server, this translates to daily electricity consumption of 80 GWh and an annual usage of 29.2 TWh.
Vries also emphasizes that AI tools undergo an initial training phase, followed by an inference phase. The training phase, which consumes the most energy, has been the primary focus of AI sustainability research to date. However, the inference phase, when these tools generate output based on their training data, appears to exhibit significantly higher energy demands.
Energy consumption during the inference phase of AI
“OpenAI required 3,617 of NVIDIA’s HGX A100 servers, with a total of 28,936 GPUs, to support ChatGPT, implying an energy demand of 564 MWh per day,” Vries highlighted.
And this is just to get the chatbot started before any consumers even begin using it. “Compared to the estimated 1,287 MWh used in GPT-3’s training phase, the inference phase’s energy demand appears considerably higher,” he added.
Finally, Vries cautions against overly optimistic expectations that improvements in hardware and software efficiencies will fully offset any long-term changes in AI-related electricity consumption.
Nevertheless, efforts are underway to address this issue. Recently, MIT researchers made headlines by reducing the energy consumption of an AI model by 12-15% by capping the power consumed by the GPUs driving it.
Vries also suggests that older, disused GPUs utilized in Ethereum cryptocurrency mining could find new purpose as part of the solution.