First of all I would recommend you use Piefed instead. Easier to setup and maintain.
But I am not sure exactly what you want that Lemmy/Piefed instance for? As an internal forum of sorts? That can work, but is not really what it was developed for and there are better (non-federated) options.
If you want it to be an actually federated instance then the Rasberry will not cut it. The desktop might, if it has some good SSD storage for the database.
For in game voice-chat the simplest option is a Mumble server. Very low resource use and runs great on a Rasberry like yours. Otherwise you could also try setting up a Movim instance. It has text chat and voice/video calls that should reasonably work as a Discord substitute for small groups. It is also quite low resource and should run fine on that Rasberry.