Linux

There are some things "AI" isn't fit to answer, like questions with nuance. And a "prompt" in itself isn't a quantifiable unit. I can tell you that my gaming rig with 96 GB of RAM and an RTX 2080 could only run a quantized model -- the model weights will eat up my RAM and keep my system drinking electricity. My CPU could handle it like a champ but let's be real, all of that is to get a crappy-performance model doing work.

I could believe that, like, a full day of heavy agent work consumed that much power running the agentic workload on my device, but there's no way that number is accounting for the power consumed in hosted model inference, let alone training.