overview for bazinga

Hi, I am on Grapheneos and don't want to use Google play. Is there any way to make the map work in android? I have seen "manage custom tile sources" but do you know how it works?

Authoritarianism Is the Only Way Forward. Now We Just Have to Choose What Kind in c/communism@lemmy.ml

World's oldest known tortoise, Jonathan, still alive despite reports of death in c/goodnewseveryone@piefed.social

[–] bazinga@discuss.tchncs.de 7 points 2 days ago

Goodnewseveryone: BBC uses suspicious sources for journalism but figured it out eventually. Probably, haven't learned anything for the future and will continue using X as source

World's oldest known tortoise, Jonathan, still alive despite reports of death in c/goodnewseveryone@piefed.social

Which Llama Server Hardware do you use? in c/selfhosted@lemmy.world

[–] bazinga@discuss.tchncs.de 3 points 4 days ago* (last edited 4 days ago)

I agree. I also think that there is nothing good in for-profit AI corporations. I can recommend the book "the empire of AI" However, I personally think self hosting and having full control of the use is a bit different.

Which Llama Server Hardware do you use? in c/selfhosted@lemmy.world

[–] bazinga@discuss.tchncs.de 2 points 4 days ago (1 children)

Thanks, will also ask in the other group you mentioned. I am still having a gaming rig here with rx6900xt as well but way too big to get it wife approved into the living room and have no man cave to run it 24/7. ;) But maybe good for testing what I actually need in model size, I think it is just 1 generation before all the ai hype took off but going to try now right away.

Which Llama Server Hardware do you use? in c/selfhosted@lemmy.world

[–] bazinga@discuss.tchncs.de 0 points 4 days ago (3 children)

Thank you for the detailed writeup. Are you aware of anything small footprint? I am thinking like dgx spark size maybe a bit bigger?

21

Which Llama Server Hardware do you use? (discuss.tchncs.de)

submitted 4 days ago* (last edited 4 days ago) by bazinga@discuss.tchncs.de to c/selfhosted@lemmy.world

18 comments fedilink

I realize, I need to upgrade my little NUC to something bigger for higher inference of bigger llama models. I want something that you still can have on your living room's tv bench, so no monster rack please, but that has also the necessary muscle when needed for llama. Budget doesn't matter right now, want to understand what's good and what's out there. Thanks

EDIT: Wow, thanks for the inspiration, guess I need to look at bit for "how to stuff a huge graphics card into a mini box". To clarify a bit more what I want with it: I want to build a responsive personal assistant. I am dreaming of models bigger than 8B, good tool calling for things like memory, websearch etc., no coding, no image generation, no video generation required. Image recognition would be good but not a must. Regarding footprint, the no monster ;) Something that you can have in your livingroom, and could be wife approved - so no big gaming rig with exhaust pipes and stuff, needs to be good looking ;)