Self-hosting

4564 readers

2 users here now

Hosting your own services. Preferably at home and on low-power or shared hardware.

Please don't share vibe coded projects here.

Also check out:

founded 4 years ago

MODERATORS

poVoq@slrpnk.net

sam_uk@slrpnk.net

-3

FitMyLLM — Independent benchmarks for self-hosted AI (www.fitmyllm.com)

submitted 2 months ago by anzo@programming.dev to c/selfhosting@slrpnk.net

1 comments fedilink hide all child comments

cross-posted from: https://programming.dev/post/51407459

Check what can you use and at what rate of token per seconds would it be... It has examples of many models and quantization levels. Huge resource!

you are viewing a single comment's thread
view the rest of the comments

[–] Doctorbllk@slrpnk.net 3 points 2 months ago

I know I will invite ire with this, but I think a self hosted model is relatively acceptable. Get rid of the generative art and stick to things like code and evaluation via a model not being sourced by a massive data center (plus the capability to train a model in a way you may find even more acceptable than a default) and most if not all of the questionable aspects of LLMs fade away.