Selfhosted

Word of advice - don't scavenge old /server/ hardware if you plan to put GPUs in - unless you really like heat and noise. Those machines from the likes of HP will take one look at a consumer graphics card in a PCI slot and decide they need to run all the fans at 100% to ward off evil spirits. Not to mention just the general ballache of proprietary PCI riser cards, PCI power cables, etc. etc.

You're definitely better off with taking someone's old gaming rig off their hands.

In terms of specs - value VRAM above everything else. A slow, old 3000 series card with 24GB of VRAM is much more useful than a brand new 5000 with 16GB. If you can find old RTX3090 24GB, they're kinda ideal.

The one thing I will say for modern cards though is that they're much better for power efficiency - and in particular idle power (which is important if you're running the thing always on.) For my main LLM machine I have two RTX5060Ti (32GB total), which at the time was the sweet spot for price/performance/power, and it's very nice that they idle around 3 or 4 watts. I bought them before the world went crazy and prices went mad though, so they may not be the sweet spot any more.

One you're in 32GB VRAM type territory, you can run really really good dense models like Qwen-3.6-27b at a decent quant, decent context size, and good performance for things like coding, or bigger MoE models for more general use (particularly then if you have good CPU and regular RAM for offloading to CPU. For use as an assistant (i.e. not an OpenClaw fully automated slop machine,) I use 3.6-27b as a daily driver in Claude Code, and basically never use Sonnet.