I'd say it's definitely worth it.
Our brains shouldn't have to work harder to listen to lossy music, which is what happens even if you can't reliably perceive it.
Listening to music on acid (a lot) has really shaped my views of it and how even the most minor things can have a major impact on the final experience.
I'm not an audiophile though and can enjoy music in a wide range of formats and quality; I just prefer FLAC almost anywhere.*
*some songs sound 'better,' or at least more iconic in a lower quality
You get to notice things you didn't notice before. It's a lot easier for our brains to 'zoom in' and process minute details that we don't perceive normally. Since lossless and lossy music is not the exact same audio vibrating the air, our brains are not going to interpret them exactly the same. This difference doesn't matter to most and isn't always perceivable, but it's there.
One thing that stood out to me during an acid trip was how moving my phone affected the playback speed of my bluetooth speakers. Moving it further away caused the song to slow down slightly for a moment, moving it closer caused the song to speed up slightly. You can imagine that this is because of some kind of 'space invaders' effect, where my phone is sending out signals at a constant rate and adjusting the distance to the receiver causes those signals to be received faster or slower, temporarily.