391
submitted 1 week ago by neme@lemm.ee to c/opensource@programming.dev
you are viewing a single comment's thread
view the rest of the comments
[-] jsomae@lemmy.ml 1 points 1 week ago* (last edited 1 week ago)

Obviously it depends on your GPU. A crypto mine, you'll leave it running 24/7. On a recent macbook, an LLM will run at several tokens per second, so yeah for long responses it could take more than a minute. But most people aren't going to be running such an LLM for hours on end. Even if they do -- big deal, it's a single GPU, that's negligible compared to running your dishwasher, using your oven, or heating your house.

this post was submitted on 09 Jan 2025
391 points (99.2% liked)

Opensource

1669 readers
217 users here now

A community for discussion about open source software! Ask questions, share knowledge, share news, or post interesting stuff related to it!

CreditsIcon base by Lorc under CC BY 3.0 with modifications to add a gradient



founded 1 year ago
MODERATORS