You are actually not alone in this, it’s totally normal to not perceive the difference in sounds which are not phonemic in your native language. This can be overcome by training with minimal pairs. There’s a great bit about this in a book called Fluent Forever.

Edit: I just got out of a doctor’s appt and have time now so I’ll just paraphrase as I remember it. When we are babies our brains learn to ignore the difference in sounds that don’t cause a change in meaning. So a child growing up in an English speaking environment has no trouble differentiating the English language “R” and “L” sounds from each other. Japanese doesn’t distinguish these sounds, and has its own sound which is somewhere in between. So when Japanese speakers hear either sound, their brain just puts both into the same bucket.

If you play an audio recording for a Japanese listener and then ask them whether that said “rug” or “lug”, they literally can’t hear the difference. This is normal. But if you play audio recordings like this- “Rug” and “Lug” are minimal pairs, words that differ only by one phoneme - ask the listener which word they heard AND immediately give them feedback on whether their response was correct or incorrect, they learn to differentiate the sounds rather quickly.

Hindi has a lot of retroflex consonants that likely don’t exist phonemically in your language. It’s only natural that you cannot distinguish them. But by training your ear with minimal pairs, you will learn to hear the difference. You could try searching “Hindi minimal pairs training” or check out the book I mentioned, the author actually has compiled resources for various specific languages.