Technology

1420 readers

15 users here now

A tech news sub for communists

founded 3 years ago

MODERATORS

muad_dibber@lemmygrad.ml

burlemarx@lemmygrad.ml

egs81t@lemmygrad.ml

Granite 4.1: IBM's 8B Model Is Competing With Models Four Times Its Size - Firethering (firethering.com)

submitted 2 days ago by yogthos@lemmygrad.ml to c/technology@lemmygrad.ml

10 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] PoY@lemmygrad.ml 2 points 2 days ago (1 children)

i saw a comparison of the 8b model vs the dense 30b (iirc) dense model and it was almost the same... the 30b was slightly better on most tests but only barely

[–] yogthos@lemmygrad.ml 4 points 2 days ago (1 children)

It's honestly incredible to see because 8b is getting to the point where it will run well on a lot of consumer hardware. If we can get current frontier performance at that size, then you really would be able to solve most tasks locally.

[–] CriticalResist8@lemmygrad.ml 5 points 2 days ago (1 children)

The 4-bit quantized GGUF for granite 4.1 is sub 5GB, so it's probably going to run on any modern machine even if it's not particularly built for Vram... 6 gigs is what I had on my old 1080 gpu.

https://huggingface.co/unsloth/granite-4.1-8b-GGUF/tree/main

[–] yogthos@lemmygrad.ml 2 points 2 days ago

🎉