this post was submitted on 21 Jul 2026
25 points (100.0% liked)
Technology
1469 readers
22 users here now
A tech news sub for communists
founded 4 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Depends on the model, but maybe it's around 16 gigs of vram, and a good gpu, but you'd need a lot more resources to make it perform as good as the one in the web interface
You need 128GB of VRAM for aggressive quantization of DeepSeek Flash. 16GB might be enough for aggressive quantization of smaller models. In theory you can run Qwen3.6 27B in 2-bit quantization (or something like https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf ) and have some space for context but it's a toy. You need at least 24GB of VRAM to use reasonable quant of Qwen3.6 27B (or 32GB of shared VRAM to handle Qwen3.6 35BA3B which is faster but takes more VRAM than 27B) in actual tasks.
I've had great luck running qwen2.5-coder:7b on my work laptop. It's great for a wide range of dev and operational tasks with opencode.
So basically needs more power, huh...
Based on the other comments a lot more, the whole ai crash isn't really about regular consumers but big companies that prefer to host their own ai instead of paying open ai and anthropic
Honestly, the development of AI (not it happening but HOW it happened) has been very interesting and somewhat hopeful to me.
This is because the companies that control this technology don't have full control and basically are fighting each other for it.
Really fascinating ngl
I'd say it's even more about medium sized and small companies (and also companies that aren't focused on IT primarily) being able to have an actual choice of a provider and providers being forced to compete for a price.