this post was submitted on 16 Dec 2025
19 points (82.8% liked)
Technology
1406 readers
23 users here now
A tech news sub for communists
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
mixture-of-experts but the experts are the five heads.
this is technically possible already, just that it takes so much resources that not many people could make this model and the cost would be prohibitive for what is a hobby project. This is why China is developing forces of production first.
It might be cheap enough to make a LoRA actually. It would still probably take a week or so, but doable. :)
we have to do this actually