I consolidated my setup a bit. This is my local LLM hosting server. I took a gamble on a Chinese NVLink SXM2 mezzanine board from ebay, it was surprisingly plug and play for my dual 16GB v100s 😂. I'm also running a Tesla P40 that I repasted with liquid metal that sits at 24C idle 🥶.
What kind of model and space limitations are you under with v100s?
Some of the most interesting computer music I’ve heard in years was composed on them but idk if it’s worth getting into a whole new generation of hardware if it can only really do that.