...and more importantly, where is that setting stored so I can turn it on for all of them?

I realize this varies according to OS, so I'm specifically asking about my Turnkey Nextcloud and Turnkey Mediaserver containers, which are based on Debian Linux. It's perhaps worth noting that my Turnkey Syncthing container, which uses the same base OS, does register its hostname with my DHCP server. I've gone digging around in /etc/ etc. in each of the containers, but so far I haven't found any configuration differences that would explain the difference in behavior. (Also, if it matters, my DHCP server is the one included with OpenWRT and running on my router.)

By the way, I'm aware that the best answer might be "you have an X/Y problem and you really ought to use static IPs and/or setup a reverse proxy," but I haven't gotten to that point yet. Besides, I still want to satisfy my curiosity about DHCP and hostnames regardless.

EDIT: to be clear, this post is about LXC containers running directly in Proxmox, NOT Docker.