It sounds like what you really need is a mesh VPN not really KVM.
Install Tailscale on all the machines and you're set.
Alternatively Hamachi.
Edit: You could also install tail scale on a single dedicated device on your destination network and configure it as a Tailscale router. This would enable you to access any IP-based device on the network without that device having Tailscale installed on it.