Technology

Cool, now how much power was consumed before even a single prompt was ran in training that model, and how much power is consumed on an ongoing basis adding new data to those AI models even without user prompts. Also how much power was consumed with each query before AI was shoved down our throats, and how many prompts does an average user make per day?

In total, the median prompt—one that falls in the middle of the range of energy demand—consumes 0.24 watt-hours of electricity, the equivalent of running a standard microwave for about one second. The company also provided average estimates for the water consumption and carbon emissions associated with a text prompt to Gemini.

This feels like PR bullshit to make people feel like AI isn't all that bad. Assuming what they're releasing is even true. Not like cigarette, oil, or sugar companies ever lied or anything and put out false studies and misleading data.

However, there are still details that the company isn’t sharing in this report. One major question mark is the total number of queries that Gemini gets each day, which would allow estimates of the AI tool’s total energy demand.

Why wouldn't they release this. Even if each query uses minimal energy, but there are countless of them a day, it would mean a huge use of energy.

Which is probably what's happening and why they're not releasing that number.

The company has signed agreements to buy over 22 gigawatts of power from sources including solar, wind, geothermal, and advanced nuclear projects since 2010.

None of those advanced nuclear projects are yet actually delivering power, AFAIK. They're mostly in planning stages.

The above isn't all to run AI, of course. Nobody was thinking about datacenters just for AI training in 2010. But to be clear, there are 94 nuclear power plants in the US, and a rule of thumb is that they produce 1GW each. So Google is taking up the equivalent of roughly one quarter of the entire US nuclear power industry, but doing it with solar/wind/geothermal that could be used to drop our fossil fuel dependence elsewhere.

How much of that is used to run AI isn't clear here, but we know it has to be a lot.

The real question is why anyone would want to use more power than a regular search engine to get answers that might confidently lie to you.

There were people estimating 40w in earlier threads on lemmy which was ridiculous.

This seems more realistic.

median prompt size

Someone didn't pass statistics, but did pass their marketing data presention classes.

Wake me up when they release useful data.

Now do training centers, since it's obvious they are never going to settle on a final model as they pursue the Grail of AGI. I could do the exact same comparison with my local computer and claim that running a prompt only uses X amount of watts because the GPU heats up for a few seconds and is done. But if I were to do some fine tuning or other training, that fan will stay on for hours. A lot different.

Nice share! Mistral also shared data about one of its largest model (not the one that answer in LeChat, since that one is Medium, a smaller model, that I guess has smaller energetic requirements)

https://mistral.ai/news/our-contribution-to-a-global-environmental-standard-for-ai

Let’s see OpenAI’s numbers