this post was submitted on 21 Jul 2026
25 points (100.0% liked)
Technology
1469 readers
35 users here now
A tech news sub for communists
founded 4 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
ChatGPT and Claude are programs that are not publicly available. They run on privately owned, powerful computers (servers), and your only access to them is through those servers. For better-performing models, you have to pay a subscription. They also cost a lot for the companies to run because they waste a lot of energy.
Chinese AIs like deepseek or qwen or glm, on the other hand, are very efficient while performing pretty well, which means they're cheaper to run. At the same time, they're publicly available, so anyone can download the models and run them on their own computer.
How much VRAM or RAM do you need?
you need terabytes of vram if you want to run a full foundational model.. otherwise you're running a quantized version that introduces loss of information and accuracy.. some models can be quantized low enough to run on 8gbof vram, and many smaller models can run fine on 16, 24, or 32gb of VRAM
noted
Depends on the model, but maybe it's around 16 gigs of vram, and a good gpu, but you'd need a lot more resources to make it perform as good as the one in the web interface
You need 128GB of VRAM for aggressive quantization of DeepSeek Flash. 16GB might be enough for aggressive quantization of smaller models. In theory you can run Qwen3.6 27B in 2-bit quantization (or something like https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf ) and have some space for context but it's a toy. You need at least 24GB of VRAM to use reasonable quant of Qwen3.6 27B (or 32GB of shared VRAM to handle Qwen3.6 35BA3B which is faster but takes more VRAM than 27B) in actual tasks.
I've had great luck running qwen2.5-coder:7b on my work laptop. It's great for a wide range of dev and operational tasks with opencode.
So basically needs more power, huh...
Based on the other comments a lot more, the whole ai crash isn't really about regular consumers but big companies that prefer to host their own ai instead of paying open ai and anthropic
Honestly, the development of AI (not it happening but HOW it happened) has been very interesting and somewhat hopeful to me.
This is because the companies that control this technology don't have full control and basically are fighting each other for it.
Really fascinating ngl
I'd say it's even more about medium sized and small companies (and also companies that aren't focused on IT primarily) being able to have an actual choice of a provider and providers being forced to compete for a price.