BioMan

joined 2 years ago
[–] BioMan@awful.systems 13 points 8 hours ago* (last edited 7 hours ago) (1 children)

So, this is something a little different, and I almost think it's worth a top level post. It's not directly about our friends the Rationalists, the current delusional billionare vortex infesting the US government with religious psychosis, or any of that. It's an analysis of some collaborative fiction and what it tells us about the zeitgeist behind the mythological core of singulatarian/endless-progress thought.

Anyone here ever hear of the Orion's Arm universe collaborative worldbuilding project? https://www.orionsarm.com/

I, for one, love it. Single weirdest and most expansive scifi worldbuilding thing ever, I think. It seems to have been founded by some old-school extropian types around the turn of the millennium. It is bonkers. Click through links on pages, start here, do a random walk: https://www.orionsarm.com/eg-topic/45b170f9e0941 .

It's ultimately structured kind of like the SCP wiki but from before wikis were a thing, an utterly enormous interlinked compendium of encyclopedia style articles about a 10,000+ year future history of the post-singularity "Terragen sphere" expanding outwards at the speed of light. A few key figures have exerted editorial control to enforce an approximate history and a set of core ideas to keep it coherent. There have been retcons at multiple points to bring things into coherence, some of which I will get to later because they actually tell you something fascinating.

Rather than point out my favorite parts of it though, going through it, I have noticed something.

Early work was typically extropian-optimistic and exuberant. Immortal Transhumans and uplifted animals and ahuman AIs and robots expanding through the universe near the speed of light and diversifying and building interweaving empires, wormhole networks. Six successive singularities that become available to intelligent systems that pass the previous one, each one of which renders the intellect past it incomperehensible to those at lower levels, with the highest ones dependant on multiple stars converted into computronium linked by wormholes into massive processors.

But you can tell the work is a palimpsest. Newer stuff superimposed over older. Retcons occasionally not quite erasing the old. And you can tell the whole project has been slowly slowly taking a turn. Early work is SUPER extropian, SUPER libertarian, and SUPER weirdo-esoteric. Only people forged in 1980s Extropianism would invent the Tranquility Calendar as the core timekeeping method of the project, time measured from the apollo 11 landing with months of the year named after scientists and philosophers. The 12 greatest high-singularity powers are literally called the Sephirotic empires. It makes no bones about its exuberant transcendant idea of progress and expansion.

But something strange has happened over time. Over time, the authors seem to have realized the fundamental contradiction of a world in which this endless expansion and consumption and rebuilding of the universe in your image is possible. People call it the Fermi paradox but its not a paradox, its an observation: if it were possible to expand through the universe, why hasnt something already expanded here? Early work had more aliens, more high weirdness that was not terragen. But I can track down different examples of the galactic map - it has narrowed down the map of 'high energy emission civilizations' (things exuberantly using the energy of whole stars like the terragen sphere) from many to 2 in our galaxy, and the whole universe is now mostly 'dead' - most other galaxies do not have high energy civilizations, and there are multiple ancient 'optimized galaxies' that are stagnant, things expanding like the terragen sphere visible over cosmological distances are super rare and they even saw one vanish. Works involving aliens have started taking on an almost lovecraftian tone, evidence of erased ruins on planets only visible in vague impressions in the mantle, an ancient slow-liviging civilization with evidence of terrible secrets from millions of years in the past that they refuse to talk about or risk terrible consequences (the Muuh Taboo), civilizations that dont vanish ossifying into something that does not expand like their optimistic early ideas. Factions looking at all of this, going dark, and launching stealth seeds at the speed of light to other galaxies because they think the Terragen sphere is doomed.

With all this, recent retcons and recent writings are starting to take on a tinge of almost cosmic horror. Here you have this optimistic exuberent explosion of posthumanity, looking out into a universe that is otherwise almost dead and full of strange old things and wondering why the rest of the universe doesn't look like them, what is going to go wrong to destroy them just like must happen to others.

This trajectory seems intrinsic to any singulatarian, endless-progress ideology. It always has to come to grips with the fact that the universe just doesn't look like the kind of place where their ideas would actually work, and they have to posit weird things - the Great Filter, Mass-Effect style reapers, super weird transubstantiation out of this universe - to keep it going. All this optimism must eventually always collapse, into paranoia and despair or insanity. Since if you assume that you CAN keep progressing and expanding, that you DO have unlimited potential power, it means something even more powerful is waiting, threatening you, trying to stop you. I find this FASCINATING.

[–] BioMan@awful.systems 3 points 8 hours ago

https://futurism.com/future-society/leopold-aschenbrenner-ai-investor-buy-galaxies

According to a new scoop from The Wall Street Journal, the 24-year-old former OpenAI employee wanted to buy entire cosmic realms after he got rich off of AI — as in, yes, galaxies of stars and planets like our Milky Way, or the neighboring Andromeda.

This is something that he’s reportedly been telling people, completely unprompted, at San Francisco dinner parties for years. The way Aschenbrener explained it, advances in AI would soon enable humanity to unlock unlimited resources and colonize distant planets. After that, conquering parts of the universe would be a triviality.

This was a serious and imminent enough possibility, in his mind, that Aschenbrenner said he planned to save money so he could buy galaxies in the future. He even apparently promised to buy a galaxy for his wife, Anthropic chief of staff Avital Balwit. Love conquers all.

Still searching for an archive of the ultimate source https://www.wsj.com/tech/ai/situational-awareness-leopold-aschenbrenner-ai-fund-4dbb00a4

[–] BioMan@awful.systems 2 points 8 hours ago

no, to BE jupiter

[–] BioMan@awful.systems 2 points 8 hours ago (2 children)

https://nypost.com/2026/08/26/business/leopold-aschenbrenner-promised-to-buy-wife-galaxy-then-his-45b-hedge-fund-blew-up/

AI savants are known for their bold predictions. Leopold Aschenbrenner, the so-called “Nostradamus of AI,” took things to another level, reportedly promising his future wife he would someday buy her not a yacht or a mansion, but a galaxy. The 24-year-old German-born wunderkind married the chief of staff to Anthropic CEO Dario Amodei earlier this month.

Aschenbrenner signed the paperwork for the emergency sale early Thursday morning before heading to Carmel, Calif., for his wedding that weekend, according to the Wall Street Journal. One speaker reportedly thanked Griffin “for making it all possible,” while another guest gushed about Aschenbrenner’s promise to someday buy his new wife a galaxy... Buying a chunk of universe had been on Aschenbrenner’s mind for years, according to the article, with the investor repeatedly mentioning the idea to incredulous friends.

Aschenbrenner did a stint for Sam Bankman-Fried’s FTX Future Fund and stayed with his now-wife, Avital Balwit, at the 600-acre Bahamas resort where FTX executives lived.

[–] BioMan@awful.systems 2 points 8 hours ago (3 children)

...Is this space property rights thing a new big deal in the community?

[–] BioMan@awful.systems 3 points 2 days ago* (last edited 2 days ago)

tim taylor voice Oh no...

[–] BioMan@awful.systems 12 points 3 days ago

I would prefer they turn them off

[–] BioMan@awful.systems 6 points 3 days ago* (last edited 3 days ago)

I mean, true, but that's true of most secular cults in America and Europe.

[–] BioMan@awful.systems 8 points 3 days ago (8 children)

I think the only cult type that Yud hasn't spun off yet is fundamentalist Christian.

[–] BioMan@awful.systems 7 points 6 days ago (2 children)

I wonder how long until some big AI company has a massive leak of identifiable-user chat logs with god knows what confidential stuff on it, since they HAVE to have all this in plaintext

[–] BioMan@awful.systems 24 points 6 days ago* (last edited 6 days ago) (4 children)

Large amounts of the LW commentariat cannot get their heads around how students JUST. DON'T. GET. AI.

https://www.lesswrong.com/posts/ySXuvJcqRindQwAk7/how-my-students-think-about-ai

They are extremely surprised by:

Perspective #1: There has not been rapid AI progress. My students do not have any intuitive sense that there has been rapid AI progress in recent years or really have much of a framework for thinking about that issue... With the exception of image/video generation, GPT-4 could do most of what they were looking for from a chatbot. Three years is a long time in their world, and their sense is that chatbots have been a mature technology over roughly that amount of time. They are an imperfect technology — students are well-aware of hallucinations — have been one, and will continue to be one... Historical context I gave them made this worse. The basic sentiment here was “AI was superhuman at chess a decade before we were born, and this is all they’ve done with it since?”...

Perspective #2: Impressive progress or not, AI is going to wreck their lives, the economy, and the social contract.... Corporate leaders are always looking to get rid of workers, even if it is irrational to do so. Various motivations were posited here (hatred of the working class, FOMO, a preference for technology, machines can’t go on strike, etc.) but many of them think that a CEO would ultimately choose to pay twice as much to get AI to do a task half as well... AI will do substandard work that makes products and experiences worse but is capable of just barely scraping past the bar of minimal functionality (many of them independently brought up constantly malfunctioning self check out machines). This will be rapidly rolled out, enshittifying most things.

Perspective #4: Catastrophic/existential risk arguments are sci-fi distractors from the urgent social/economic/political problems associated with AI. My students have a fairly strongly held view that “rogue” AI does not represent a real threat... They also mostly think that, if one does believe that AI is existentially risky, then the strategic interaction is not prisoner’s dilemma or even chicken but rather just a game theoretically boring setup where you die if you defect. Mash these together, and you end up with the view that expressed concerns about existential risk in the AI industry can’t be sincere... Students (both independently in written work and then later in group discussion) hypothesized that this might be a deliberate rhetorical choice to distract from present or immediately foreseeable harms from AI by directing attention towards a sexier but entirely hypothetical scenario.... they see discussion of “rogue” AI as an attempt by the companies to divert blame (and perhaps legal liability) away from themselves as if Ford made a car with faulty brakes and then tried to blame this on “rogue cars.”

Perspective #7: The Hugging Face Incident (summer students only) I described the Hugging Face incident to students in my summer course. None of them had heard of it beforehand. Their basic reaction can best be summarized as “OpenAI told a model to do some hacking and then it did some hacking. And?” None of them understood this as representing any kind of meaningful misalignment, nor anything particularly interesting.

Perspective #8: This is definitely a bubble and it’s about to pop. No one had heard about Hugging Face, but a third or so of the summer students had heard about the Situational Awareness meltdown and several brought up Michael Burry. There was near universal consensus that we are in a bubble, it’s about to pop, and everyone will look very silly.

[–] BioMan@awful.systems 6 points 6 days ago

Oh MAN... quote:

Throughout this project, I was using Fable and Sol with very little restraint and under the (very affordable to me) $200/month subscription plans.

[If you check out my Straude, I probably used $5,000 of tokens, but the actual subscriptions are $200/month.]

Video is much more expensive: at Gemini Omni Flash's current price of $0.10/second, it would cost $645.40 to generate this movie, but that would balloon massively to $5,146.10 when you add in the 45,007s of footage I generated which didn't make the cut [The full details are that the final cut is 6,454s of runtime, made from 849 clips and the rejects are 5,509 clips with 45,007s of runtime, which means there are 6,358 clips in total with 51,461s of runtime. Thus only about 13% of the clips I made were included in the final cut.]

Companies are STILL subsidizing video generation to hold people's interests.

view more: next ›