this post was submitted on 30 Apr 2026
11 points (86.7% liked)
Free Open-Source Artificial Intelligence
4674 readers
5 users here now
Welcome to Free Open-Source Artificial Intelligence!
We are a community dedicated to forwarding the availability and access to:
Free Open Source Artificial Intelligence (F.O.S.A.I.)
More AI Communities
LLM Leaderboards
Developer Resources
GitHub Projects
FOSAI Time Capsule
- The Internet is Healing
- General Resources
- FOSAI Welcome Message
- FOSAI Crash Course
- FOSAI Nexus Resource Hub
- FOSAI LLM Guide
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Thanks! I don’t think I can run an 8B yet. Need to invest in a better machine. I’m stuck on 4B Q4.
The uncensored Qwen that I’m using started throwing infinite ?’s at me one time. Had to restart it and has been fine since.
It is certainly inaccurate, but in my mind's picture of how the transformers work, reducing their quantization and also doing what abliteration does, there is a line where you've done a lot of "damage" to the original model and so there will be places where it just hangs or goes off on severe tangents. There are good places for even the 1bit models where they don't get pushed to hard, but there are limits for them all, including the big ones.
Hugging Face does have a few Q4_K_M versions. Maybe something will fit.
For sure. You would need a model that is not censored at training.