I know you specifically want a hardware solution, but if you use pipewire I have something for you that took forever to figure out from the docs and does just what you want with a single static config file: https://pastebin.com/XigrzvfD

Put this in ~/.config/pipewire/pipewire.conf.d/10-virtual-sinks.conf and restart pipewire once. It is safe to try this. Once you remove the file and restart pipewire everything is back to normal.

This creates virtual output devices that you can assign applications to and control with e.g. pavucontrol. It's mapped to use my specific output devices, if you uncomment the lines and remove the node name expressions then all USB/PCIe devices are used.

It creates a setup of: virtual:[Games, Media, Comms] -> virtual:Main -> virtual:All Physical Outputs -> [output devices]

I wanted all audio to always play on all devices. You can of course adapt it to your use case. In my case the virtual Main is my global mute for everything. I never touch volumes or mute of the actual output devices.

I have the volume of these output nodes mapped to physical knobs to control games/voice/media independently globally.

EDIT: To control the nodes I use this script: https://pastebin.com/pANNDvup

Mute toggle: volume.sh set-mute virtual:Games toggle

Volume: volume.sh set-volume virtual:Games %d

You will have to adapt that slightly as I use it with OpenDeck and a stream deck clone for control. OpenDeck outputs [-]10 but wpctl needs 10-/10+. There is currently no way to set an absolute volume with that script since I didn't have the need.