I figured swap would only be used if I ran out of available memory. I'm not an expert at all, so I could be totally wrong about that.

Yeah this is a fundamental misunderstanding of how swap works on modern systems. Unfortunately a common one as well. Lots of people think using swap is bad and that they should eliminate it or that they need more RAM when they really don't.

Is it possible that the container is allocating the space preemptively but not actively using it? So proxmox would see it as being used, but plex is simply reserving it. I think I've seen that occur on other containers like home assistant (but this is a VM not LXC). Proxmox says it's using almost all the memory available to it, but going into home assistant itself says it's only using a fraction of that amount.

Just as likely to be disk caching. Sometimes it's counted as using memory and sometimes not.