The explosive recent growth of AI tools to generate text, images, or audio relies on gargantuan amounts of information.
That information doesn’t come for free. It can exact high – and unequal – costs in terms of energy, water, and labor, though these costs are largely invisible to users.
In terms of energy, generative AI models typically depend on extremely large-scale cloud providers, which use chips with more transistors that require at least 10 times as much energy as traditional versions. Unsurprisingly, models that use more training data and contain more parameters tend to guzzle more energy.
One estimate is that just training OpenAI’s GPT-4 entailed roughly 300 tons of CO2. This is about as much as 300 round-trip transatlantic flights. And that doesn’t even account for the continuing emissions from using the product.
As for water needs, it’s been estimated that a simple ChatGPT conversation uses up the equivalent of a 500ml bottle of water. The water resources needed to cool down massive data centers are particularly concerning to already water-stressed African countries looking to build out their own AI systems.
Water conservation and energy savings don’t always go hand in hand. For instance, turning to solar energy during peak sunlight hours can use up extra water.
Overall, the environmental expenditure is one of the reasons that computer scientists are or should be working on reducing the size of the large language models currently being used to train these generative AI products.
Another reason is limited computing power. Demand for this currently outstrips supply by about 9 to 1, estimated Björn Ommer, a computer scientist at the Ludwig Maximilian University of Munich, while speaking at the Heidelberg Laureate Forum, a math and computer science conference, on September 29. Ommer worked on the first version of Stable Diffusion, a text-to-image generator trained using billions of images.
There’s still some debate about whether we’ve seen an end to Moore’s law, which suggests that the number of transistors on a microchip doubles every couple of years. Ommer is clearly in the camp that considers this law no longer valid. Chips currently can’t be manufactured fast enough to keep up with the demand.
But there’s also a less tangible reason for trying to do more with less. “We should get more out of smaller models,” Ommer said. In his view, true intelligence comes from working with smaller models.
Sébastian Bubeck, who heads the Machine Learning Foundations group at Microsoft Research and who spoke alongside Ommer at the Heidelberg Laureate Forum, agreed. While he warned that in general, nobody on the planet knows where AI development is going, not even people who work on AI, “One thing we do know is there’s a ton of room for improvement.”
Bubeck highlighted the scope for achieving the same quality with much less data in future versions of generative AI tools. “GPT-4 is a first cut,” he said.
Whether computer scientists are motivated by environmental efficiency or the goal of achieving more sophisticated AI, the good news is that there are ways to minimize the computational complexity of AI without necessarily losing reliability. Machine learning models can be run on low-power, low-bandwidth devices like microcontrollers, which require comparatively little power.
Likewise, a humongous model that massively increases energy requirements while only incrementally increasing accuracy may not be efficient or desirable. One option may be to move away from a single large language model to rule them all, and toward smaller models that are more tailored to specific needs. However, this has proven challenging so far.
There is a growing body of resources to show computer programmers the carbon footprints of their code, such as Green Algorithms and ML CO2 Impact. Environmental efficiency may not be taught in computer science ethics classes (in the rare cases when ethics training is even offered or incentivized), but it’s an important part of AI ethics.
Admittedly, this can bump up against other ethical considerations, like the need to actually expand models in order to incorporate more varied data that better represents the world’s diversity.
Ultimately, computer scientists working on generative AI have amply demonstrated that they have the ability to achieve impressive results from staggering amounts of data. A challenge for the future will be seeing whether the models can be refined to be less ravenous for resources.
This article was reported with the support of a journalist travel grant to attend the Heidelberg Laureate Forum.