If you already have/can run a local server, then maybe storing the luks passphrase there and running a script on it which sshs into the remote server end enters the stored passphrase on command. Maybe a simple http server triggers it, which you could auth using forward auth of your reverse proxy, so you wouldnt need to implement auth in your script.
Of cause the passphrase is stored in plain text, but that will be the case in any case not using a tpm.
You could do notifications with a simple webhook of your favourite chat app, or by running a ntfy server (and app) and also sending the notification with a curl from a initrd script.
The performance drop from virtualizing nics shouldn't be nearly as big. How are you passing the vlans to the VM? are you passing all over one virtio nic or one virtio nic for each.
The setup I ran for multiple years was basicly a bridge interface on the host for each vlan and a seperate virtio nic to the opnsense VM for each, I got almost 10 gbit/s like that with 8gigs of ram for opnsense and 4 or 8 cores (I cant remember) with hyperthreading of a 2nd gen epyc. I didn't do any optimisations for virtio.