It does often signal a bit of "the problems on Earth don't matter and there's nothing and no one there that I care about" selfishness and a whole lot of juvenile escapism on top of that

That's my unironic take as well.

I think it's not ethical under capitalism to be throwing this much money into space while people's basic needs aren't met.

It would probably take a luxury space gay communism type thing for me to visit other planets, because even if a socialist society where people's needs are met hypothetically would want me to assist their space program, my skill set would very likely be more useful here on Earth.

Even to go to orbit, it would take a fairly advanced socialist society perhaps where orbital flight is a viable means of planetary transportation to the general public to be ethical IMHO

Addendum: This isn't to say that space programs themselves are 100% unethical even under capitalism — they've definitely helped life on Earth so far by necessity of new materials that withstand extreme conditions. Elon's bazinga tourism is 100% unethical though.