12
Desktop app for ChatGPT?
(lemmy.world)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 16 Sep 2023
12 points (83.3% liked)
ChatGPT
8912 readers
1 users here now
Unofficial ChatGPT community to discuss anything ChatGPT
founded 1 year ago
MODERATORS
Why do you want a desktop app if I may ask? Doesn't the webapp work just as well?
The web app is great and I’ll definitely keep using it, but I was after something with decent UI/UX that runs locally / offline (which I foolishly didn’t mention in my OP).
"Runs locally" is a very different requirement and not one you'll likely be able to find anything for. There are smaller open source LLMs but if you are looking for GPT-4 level performance your device will not be able to handle it. Llama is probably your best bet, but unless you have more VRAM than any consumer gpu currently does , you'll have to go with lower size models which have lower quality output.
Thank you, I realised that once I installed GPT4All! I’ve got Llama going now, and will look at upgrading my RAM to accommodate a larger model like Falcon if I feel I need it. I’ve learned a lot this morning, it’s been great!
Do keep in mind that if you upgrade your regular RAM this will only benefit models running on the CPU, which are far slower than models on the GPU. So with more RAM you may be able to run bigger models, but when you run them they will also be more than a literal order of magnitude slower. If you want a response within seconds you would want to run that model on the GPU, where only VRAM counts.
Probably in the near future there will be models that perform much better at consumer device scale, but for now unfortunately it's still a pretty steep tradeoff, especially since large VRAM hasn't really been in high demand and is therefore much harder to come by.