Artificial Intelligence (AI)

Artificial Intelligence (AI) is revolutionizing industries across the globe, catalyzing unprecedented growth and propelling innovation. Yet, this extraordinary progress comes with a consequential price - an enormous thirst for electrical power. Central to this power-guzzling equation are Graphics Processing Units (GPUs), the workhorses designed to handle the intensive computational tasks at the heart of AI applications. This colossal energy consumption and its corresponding environmental footprint present a significant challenge.

Executing AI algorithms usually requires GPUs for processing enormous data sets to derive valuable insights and the scale of power consumption associated with this is formidable. For example GPT-3, the model that underpins ChatGPT, consumed 1,287 MWh when the model was being trained - equivalent to the annual power consumption of 121 homes. Inference, which NVIDIA estimates accounts for approximately 80-90% of utilization, can get even more expensive. For example, in a recent Medium post data scientist Kasper Ludvigsen estimated ChatGPT’s inference power consumption to be 9,345 MWh, per month. That’s a lot of energy for just one (albeit popular) model - and the problem gets even more complicated when you zoom out. As highlighted in an article in Semiconductor Engineering, AMD’s CTO Mark Papermaster suggested at a recent Design Automation Conference that total compute energy consumption is on pace to pass energy supply:

On average, GPUs operate at a surprisingly low utilization rate of just 10 to 15 percent- representing a substantial level of underutilization in the available supply of GPU compute power within the market. However, the underlying issue is even more concerning - GPUs operating at these low utilization rates still consume nearly as much energy as those running at 100% utilization. This power inefficiency poses a significant environmental challenge, irrespective of GPU underutilization.

Juice presents a promising solution to the power-guzzling challenge faced by the AI industry. Our GPU virtualization software allows for dynamic GPU sharing (unlike hard partitioning), allowing deployments to ramp more easily to full utilization of their fleet and thereby boost overall FLOPS/WATT ratio, dramatically reducing energy wastage and curtailing environmental impact.