I consolidated my setup a bit. This is my local LLM hosting server. I took a gamble on a Chinese NVLink SXM2 mezzanine board from ebay, it was surprisingly plug and play for my dual 16GB v100s ๐. I'm also running a Tesla P40 that I repasted with liquid metal that sits at 24C idle ๐ฅถ.
I had a similar goal with Hermes, but ended up splitting the context between separate profiles instead of teaching one agent everything. They share a Kanban board, while important decisions still come back to me.
It may be overkill for a homelab, but the profile isolation has worked better than I expected. I documented the setup here: https://github.com/nohuman-labs/agent-company