So I'm curious — we're all here because we at least hate the current state of AI with hallucinating facts, being used to undress women and children, and all the fuckery that goes along with it.
I grew up watching Star Trek: The Next Generation, which takes place on a ship with a perfect AI that does everything right and basically does nothing wrong. It never hallucinates information; it's always right. It has never been used to undress people against their will; however, the Holodeck is kind of an extension of that and was used for that on Deep Space Nine, when operated by a Ferengi (capitalist alien race in a world where humans are communist). But the Enterprise holodeck would never do that. The shipwide AI also does not traditionally carry on conversations. The one time it does, the human was hallucinating — sort of. The doctor was in a pocket universe, people were disappearing, and at one point the AI told her she was the only crew person on the Enterprise, and no, that did not make sense, but that that was still how it was. Because, in her pocket universe, it was true.
So the question is... would you want a perfect AI that was incapable of lying or harbouring anything untrue? Basically you could ask it anything and it would give you the correct answer.
The one fault I can find with that fictional AI is when Data (the android), dressed like Sherlock Holmes, asked the computer to "create an enemy which rivals my intelligence." He meant to say Sherlock Holmes's intelligence, who he was cosplaying, but the computer made a self-aware malicious AI that got out of the Holodeck and tried to destroy the ship... because it was told to do so. Other than that, though.
...I'm not trying to mislead anyone, so I will drop the other shoe, answer the begged question now. I've always felt that to get to that level of AI, we need to wade through the shit we're in now. So yeah, before you ask, that's kind of the point of the thought exercise. However, I will also say that I do not think we will get to Star Trek AI, I think we will get to Terminator AI, destroying the world rather than lifting people up. I think maybe in the Star Trek universe, AI didn't really take off until people realised that war wasn't the answer, after WW3/the Eugenics Wars, and so they were making AI to make things better, not worse. We are not in that timeline. I look at what is happening now, IRL, and the timeline in the Terminator franchise, and it's clear to me that that one is more realistic.
That said, I still wonder if anyone would want AI if it did not have any of the problems.
In Star Trek AI is used in many applications that a humanoid cannot reasonably do themselves entirely manually (not without serious augmentation, which is heavily frowned upon).
The amount of heavy lifting the ship's computer does for navigating the galaxy at warp speeds, or to process some incredibly advanced calculations almost instantly. This would go significantly slower if a humanoid had to do that all manually.
And yet...
The crew of the Enterprise D aren't merely prompting the ship's AI. Yes, there is that too, but they're typically doing that while also being very hands-on with the ship's systems too. The crews in Starfleet have the expertise of deep level computer programming, to the point of physically arranging computer systems if need be.
There is no "vibe coding" in Starfleet.
But then there is the Holodeck.
I'm OK with certain applications of the Holodeck, like spontaneously creating virtual activity or recreation areas. These things aren't considered works of art, and aren't considered worthy substitutes over the real thing either. You find them on ships and stations because they're the best available substitute, the alternative being crewman slowly going mad from seeing nothing but sterile corridors all day. I don't think I've heard of recreational Holodecks for regular individuals (unlikely due to some regulation, more likely that fulfilment is achieved in other ways if you're surrounded by a real planet).
However, Voyager sort of fucks with this idea with the crew "creating holonovels". This is essentially a vibe coder's dream; being able to create fully interactive narrative driven videogames with nothing but prompts. That said... Even in Voyager, using someone's likeness is heavily frowned upon. So there is still an expectation of originality, rather than merely rearranging a reference dataset. I can maybe forgive Voyager on the basis of its premise; they're stuck on the other side of the galaxy far from home. What else would they even do? But that's about as lenient as I can be.
On a societal level, however, real skill in a craft is still greatly appreciated, perhaps even preferred, over something computer generated. Despite there being replicators and no money, there are still bars, pubs, restaurants, wine makers, beer brewers, jewelers, you name it, there are still people going to these places to get a real cooked meal and have a real experience in a real location. Craft, skill, humanoid created instead of generated, these things are still valued enough for there to exist whole streets full of shopkeepers and hospitality providers on Federation worlds.
"AI" as it exists now is created to deliberately replace humans skill, to take from human skill without offering credit or appreciation, to make humans obsolete in the creation of immediate end results that can be sold as products or give instant gratification. The Federation as a society is shifted massively away from such a mentality.
To be fair, Voyager's holo-novelist is the Doctor, who is also an AI. So he could be writing very quickly since he lives in the computer and isn't constrained by human limits.
Or was Paris doing it as well? I don't quite remember.