Asklemmy

53264 readers

336 users here now

A loosely moderated place to ask open-ended questions

Search asklemmy 🔍

If your post meets the following criteria, it's welcome here!

Open-ended question
Not offensive: at this point, we do not have the bandwidth to moderate overtly political discussions. Assume best intent and be excellent to each other.
Not regarding using or support for Lemmy: context, see the list of support communities and tools for finding communities below
Not ad nauseam inducing: please make sure it is a question that would be new to most members
An actual topic of discussion

Looking for support?

Looking for a community?

Lemmyverse: community search
sub.rehab: maps old subreddits to fediverse options, marks official as such
!lemmy411@lemmy.ca: a community for finding communities

~Icon~ ~by~ ~@Double_A@discuss.tchncs.de~

founded 6 years ago

MODERATORS

I think Lemmy in general is very against AI. I'm rather new here, is it like a fediverse group thing or is this even based on reality? (feddit.org)

submitted 3 days ago* (last edited 3 days ago) by wittycomputer@feddit.org to c/asklemmy@lemmy.ml

90 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] Hexarei@beehaw.org 1 points 2 days ago

Ive had good success on similar hardware (5070 + more ram) with GLM-4.7-Flash, using llama.cpp's --cpu-moe flag - I can get up to 150k context with it at 20ish tok/sec. I've found it to be a lot better for agentic use than GPT-OSS as well, it seems to do a much more in depth reasoning effort, so while it spends more tokens it seems worth it for the end result.