alt text
Crudely drawn drawing of a person telling a computer "say 'i am in pain'". The computer replies with "> I AM IN PAIN". The person then says "oh my god."
this post was submitted on 05 Oct 2026
784 points (97.2% liked)
Fuck AI
8415 readers
957 users here now
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
It's Llama swap and I barely know how to use it.
I use Pi (the frontend) with llama.cpp in router mode. You put it in router mode by passing
--models-dirinstead of-mI'll have to figure this out. In llama swap the model unloads after a time to live which I've set to 300 seconds. I don't know if the model consumes more power just sitting on memory or not but it makes sense to me to just let it stay loaded until I need to use my computer instead if it.
I haven't noticed models using power idle, just a ton of memory.
No harm in keeping a model in memory, unless you plan to use that memory for something else. Unused memory is wasted memory