I also fit that description. I find it to be more true of Lemmy than Mastodon, but the same thing was true of Reddit's early userbase. If anything, it was more extreme; the first people to find Reddit were lisp programmers, which is a couple orders of magnitude more nerdy than Linux users.

Lemmy is used by tech nerds right now because that's who the early adopters are for any new tech that doesn't aggressively target mainstream users with a big marketing budget. Much like Reddit did, the way to attract mainstream users here is to grow communities relevant to their interests. If you're reading this and you have interests that aren't tech, you can help. Join or create a community about it, post original content there either exclusively or before anywhere else.

Of course there's some UX work to do on Lemmy itself. That's to be expected with a software version starting with 0. I don't think federation is inherently too hard for mainstream users to understand assuming they've seen email. An onboarding experience that picks a server for them from a list of defaults would probably help - some apps do that, but join-lemmy.org doesn't.