Technology

My point is that for Agentic AI mistakes with catastrophic consequences are just as likelly as minor mistakes, which is not the case for people because humans can spot the "obviously stupid" or "obviously dangerous", plus they make more of an effort to avoid mistakes that can have very bad consequences, so they tend to make catastrophic mistakes will less than minor mistakes.

People giving psychological advice are incredibly unlikely to tell suicidal people to "kill yourself", those giving food recipes are incredibly unlikely to say that pizza should have glue on top or those deploying software in Production are incredibly unlikely to delete the whole fucking Production environment including backups.

So even if the total rate of mistakes of an an Agentic AI was less than a human, its rate of catastropic mistakes would still be much higher than a human.

This is however not obvious unless one actually analises the risk profile of using Agentic AI in a specific place in a specific process, a skill very few people have plus it requires information about and/or understanding of Agentic AI which itself very few people have and the AI vendors activelly do not want people to have.

So you end up with an e-mail fluffing and defluffing machine being used to summarize and store medical info about patients and then down the line somebody gets given something that kills them because the data on file had a critical mistake.

This is why I said that its "the worst possible consequence of a mistake done here" that limit Agentic AI suitability: because generally you're going to have way more catastrophic mistakes with an AI that you will even with even an human with no domain experience.

When human makes a mistake, they learn, they continue to enrich humanity, they make a blueprint how not to make the same mistake again, if not for humanity, but at least for themselves. It also fuels some creativity so one mistake might lead to something good later.
When a mistake generator makes a mistake, it's just another mistake in a pile of mistakes that only worsen our collective human experience.