Selfhosted

"In QNAP QTS, Port 443 is the default secure port for,..."

If it says it's already in use it's because it's already in use. Try changing the QNAP management UI port.

Is your Caddy actually reachable on port 443 when you use the Container Station App?

My first thought was that your NAS might use port 443 for its own web ui?

When I start Caddy with a docker compose file,

So, I've never owned a QNAP product running QTS nor have I run Caddy in a Docker container before, but I am assuming it looks something similar to this:

networks:
  proxy-network:
    external: true

services:
  caddy:
    image: caddy
    container_name: caddy
    restart: unless-stopped
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./data:/data
      - ./config:/config
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
    networks:
      - proxy-network

Have you tried changing the port #:

    ports:
      - 80:80
      - 4443:443

Hey, I'm just guessing here because I haven't used Caddy in Docker directly, but I do manage a project that uses FrankenPHP, which is essentially a wrapper for Caddy as I understand it, and I've had a problem that looks similar to this.

Caddy will attempt to generate an SSL certificate, and if you are using a reverse proxy, depending on how it's configured, it will internally attempt to fulfil that certificate generation over port 443, which will fail, because it doesn't have a configured SSL certificate. It's the old catch-22!

The solution I have found is to temporarily internally serve your environment over port 80 for external SSL connections. This will allow Caddy to retrieve an SSL certificate and put it in place. After this, you should then be able to switch back over to port 443 for SSL connections internally, and it'll use the certificate.

Once again -- this is just a guess, and I don't know the exact criteria in getting this going in Caddy, but it might be worth a try. In my FrankenPHP project at least, it was a matter of setting the SERVER_NAME variable to http://${DOMAIN}:80

NASes are so annoying. I had so many issues with my Synology NAS with plex and jellyfin. I ended up spinning up a debian server on a small Dell optiplex micro and then mounted my Nas as storage through debian. Installed Jellyfin as a normal debian package. No docker or anything like that. I've never used (or even heard of) QNAP, but are you able to reach it through SMB? If you can, then you can do it like I did

This probably doesn’t help you much right now but I have a QNAP as well. And I too despise the QTS software. But I found out that TrueNAS can run on it pretty easily. I have an NVME drive on a usb-c enclosure that I installed trueNAS on and it boots fine into it. If I ever wanna go back it’s just a remove the boot drive and reformat (the ZFS pools are compatible unfortunately).

You need root to bind to the first thousand ports, is it possible your container station is running as root or something to that effect? Does it work correctly when the port number is some other large number like 8921?