Essentially it IS a tunnel, just with cloudflare's infrastructure in the middle handling auth and obscuring each end from the other.

Auth is handled by cloudflare. That doesn't mean cloudflare necessarily is the auth provider, though. Not likely in selfhosted, but one could set up some other auth provider, like azure, and cloudflare could give tunnel access to authorized users who actually provided credentials via azure.

The service, port, whatever being accessed via the tunnel may also require auth, and cloudflare generally doesn't handle that. For example, your cloudflare tunnel to your local sonarr instance requires auth at cloudflare first, to access the tunnel, then again at sonarr because your sonarr instance requires authentication.

In a docker environment, you would either tunnel to the docker host or to individual Dockers. The latter is more sensible and generally a bit more secure, if only because least access = better. There's probably some cloudflare tunnels docker out there that does half the setup for you, then you just stick it and the Dockers you want exposed through the tunnel all on the same docker network interface (which you create), but that's just speculation.

As far as setting tunnels up goes, the docs are really good at the step by step. Easiest way to learn it is to set up a VM similar to what you want and bang away at the steps until it does what you want. Some things are easy, like RDP. Other things are trickier.

The basics of setup are that you use the cloudflared application at both ends: one server-side to expose what you want and one client-side to access the tunnel via cloudflare.

Tailscale is the same kinda thing. I think it is way easier for a lot of people. There's a lot less setup involved. Just install the apps and make a few choices.

For personal use, I use wireguard to access my home server. Professionally I use cloudflare tunnels for a couple of things, but mostly an enterprise vpn.