Mostly because you need to be able to resolve the TLD. The root DNS servers need to know about every TLD and it would quickly be a nightmare if they had to store hundreds of thousands records vs the handful of TLDs we have now. The root servers are hardcoded, they can't easily be scaled or moved or anything. Their job is solely to tell you where .com is, .net is, etc. You're supposed to query those once and then you hold to your cached reply for like 2+ days. Those servers have to serve the entire world, so you want as few queries to those as possible.

Hosting a TLD is a huge commitment and so requires a lot of capital and a proper legal company to contractually commit to its maintenance and compliance with regulations. Those get a ton of traffic, and users getting their own TLDs would shift the sum of all gTLD traffic to the root servers which would be way too much.

With the gTLDs and ccTLDs we have at least there's a decent amount of decentralization going, so .ca is managed by Canada for example, and only Canada has jurisdiction on that domain, just like only China can take away your .cn. If everyone got TLDs the namespace would be full already, all the good names would be squatted and waiting to sell it for as much as possible like already happens with the .com and .net TLDs.

There's been attempts at a replacement but so far they've all been crypto scams and the dotcom bubble all over again speculating on the cool names to sell to the highest bidder.

That said if you run your own DNS server and configure your devices to use it, you can use any domain as you want. The problem is gonna get the public Internet at large to recognize it as real.

You can use whatever top level domain you want, you just have to convince everyone in the world to use your Root servers instead of ICANN, which ain't gonna happen. Tor has the .onion TLD, etc. There are no restrictions here. They're more like...agreements.

My guess would be to force a hierarchy as to distribute load. DNS is distributed in a sens. There are the root name servers that know about all TLD and then each TLD has its own server (in practice there is multiple TLD a single entity controls and it allocates as many server as needed to answer all DNS request for those). And those "TLD servers" know about the second level. And either they also know about the lower levels or those are further delegated.

So fewer TLD means that the "root" DNS servers do not have to keep a huge "phonebook" (TLD to IP address of the DNS responsible for them) and can therefore be efficient, which means that fewer of them are required. And fewer root server means its easier to update them and keep them consistent. And if nearly everyone can only register second level domains, then the root name servers do not need to be updated nearly as often.

In short, because of how DNS servers work. Each TLD has to be resolvable in the root servers and as far as I know ICANN doesn't manage root DNS servers and ICANN would have to become a registrar and coordinate all those TLDs with all the root servers all over the world.