At the moment I'm using Audiobookshelf (completely separate from my Plex Media Server, but running on the same computer) in a Docker container, but since I don't yet have a reverse proxy set up it is LAN or downloads only. So, I would be very interested in an answer to this as well.

Edit: I just noticed your post in c/Plex; you may want to include a link to the other post in the body so that people answering can check the other to see if their answer has already been offered.

If you want a proper audiobook server, and don't mind installing one alongside Plex, Audiobookshelf is fantastic.

For my Jellyfin server I use Symfonium to play audiobooks. It's not FOSS, but it was well worth the $5 I paid for it due to all the audio settings available.

Just checked and it does support Plex, Emby, and Subsonic as well as Jellyfin.

Plexamp is free now. Just use that.