LocalLLaMA

2747 readers

16 users here now

Welcome to LocalLLama! This is a community to discuss local large language models such as LLama, Deepseek, Mistral, and Qwen.

Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.

As ambassadors of the self-hosting machine learning community, we strive to support eachother and share our enthusiasm in a positive constructive way.

founded 2 years ago

MODERATORS

SkySyrup@sh.itjust.works

pax@sh.itjust.works

noneabove1182@sh.itjust.works

Smokeydope@lemmy.world

MonsterBug@sh.itjust.works

New release: Gemma 3 family of models (huggingface.co)

submitted 1 week ago* (last edited 1 week ago) by Lantier@jlai.lu to c/localllama@sh.itjust.works

3 comments fedilink hide all child comments

GGUF quants are already up and llama.cpp was updated today to support it.

you are viewing a single comment's thread
view the rest of the comments

[–] brucethemoose@lemmy.world 1 points 5 days ago

I tested these out and found they are really bad at longer context... at least in settings that can sanely fit on most GPUs.

Seems the Gemma family is mostly for short-context work, still.