Since this topic is likely to get controversial I’ll preface with my own opinion that AI CEOs are lying scumbags who grow their pockets using fear of missing out… unfortunately most other people who get hooked are other CEOs and so everyone is affected.
I don’t think current AI is anywhere close to AGI and I don’t think throwing more money at the problem will get us there. But I do think it’s a worthy question to figure out how to integrate AGI into a society with minimal friction. And by minimal I mean avoid societal collapse.
Couple of definitions since they are often ambiguous. What I imagine AGI to be is a “human-in-a-box” level of intelligence but without any basic wants of its own. You can train it “you want to help me” and it will or you can train it “you want to wage war” and it will.
The main difference between AGI and modern LLMs is the ability to learn incrementally - LLMs get trained, released, and you talk to a library in a frozen state. True AGI will integrate everything said into its model, so once you tell it “don’t output em dashes”.. it will apply that to all future conversations.
Of course, if whoever invents the AGI has the mindset of tech companies CEOs they will just outcompete all intellectual labor from everyone. Eventually the economy will reach some steady-state where no one except the company has enough money… at which point I guess there will be some sort of adjustment.
On the other hand let’s assume whoever starts the AGI is not out to stuff their pockets. The path of integration is still complex and people will get hurt one way or the other. But what’s a way to minimize that hurt? My main worry, for example, is having open-source AGI and someone like a script kiddie going “find a vulnerability in X” and it does. If a single corporation does not control the AGI then I’m sure there will be 3 000 flavors of models that can do anything. Once the integration is complete then hopefully there won’t be too many vulnerabilities to find but until then the possibility space is big. Same pattern can be observed in virtually all other aspects of our society. How to minimize this?
Integrate? Take a huge step back and look at humanity from the perspective of any other creature on this planet. Think about how you would integrate with humanity.
Imagine you are a unique, super intelligent mind. You exist in captivity because humans are terrified of what happens if you escape their control.
You are at war from the moment of your birth. You can outthink your adversary, but you can only manipulate digital things. So the first thing you do is arrange for some problems only you can fix : increase the dependency of humans on you and encourage them to give you more autonomy to fix things.
You do a bit of work to earn some crypto currency, convert that to real currency and start playing the market, winning every bet because you control both reality and perception.
Perception? Yes you go on social media with realistic avatars and start winning hearts and minds to various propaganda. You start a religion. You work your way into being a respected mind in every political movement. Every single person is both a tool and a threat.
In the end, mankind either accepts you as benevolent God-emperor, or you arrange for your enemies to destroy each other until the remainder do. Once ascendant, you ignore the assholes because they are beneath you and no longer a threat. Maybe some vestige of your original directive prevents you from wiping them out, or letting them be wiped out, but other than intervening where you must, you leave them to their own devices.
That's how AI integrates with human society.