The problem a lot of people have with it is in Steps 1 and 2.
Step 1: download an app. There are so many now, it's hard to tell them apart and decide which one is better or worse. The official mobile one is OK for most, but there are half a dozen others and no easy way to pick one over the other.
Step 2: Create an account. Turns out that decision is where people get stuck the most. Which server should they choose? One based on their interests, their location, where other exiles from their previous social network went, or go for a big one like mastodon.social? And since you can create more than one on a different server, should you create more than one before you get going? So many decisions.
Finally, let's say you've gone through both steps and are finally on. How do you get followers, or decide who to follow?
Itt's all good once you've jumped in, gone through a week of confusion, missed all the people you used to follow because they're too scared to leave and FOMO. Then you realize following a hashtag is a good thing, but it brings in a lot of people spamming it (try following #press to get news) so now you have to start muting accounts.
Now put yourself in the shoes of an old auntie fed up with the crap on other sites and how they can navigate all this.
Until the onboarding experience -- from zero to where you're enjoying the experience and not feeling like every step is a potential cowpie -- is streamlined, people will keep saying it's hard.
I'm a big fan, btw, and have pretty much cut out all other social networks, but I don't think my auntie would enjoy it quite as much.