just after end of manhattan project there was an idea coming from some of manhattan project scientists to dispose american nukes and ban development of nukes in any other country. that's why we live in era of lasting peace without nuclear weapons. /s

some EAs had similar idea wrt spicy autocomplete development, which comes with implied assumption that spicy autocomplete is dangerous or at least useful (as in nuclear power, civilian or military)

I'd be surprised if Eliezer hasn't mentioned it at some point, maybe more in the way that you're after. Can't find any examples though.

In his Times article the only place he mentions nukes is what we should do to countries that have too many GPUs: https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/

Edit: Not Mr. Yudkowski but see https://futureoflife.org/document/policymaking-in-the-pause/

“The time for saying that this is just pure research has long since passed. […] It’s in no country’s interest for any country to develop and release AI systems we cannot control. Insisting on sensible precautions is not anti-industry. Chernobyl destroyed lives, but it also decimated the global nuclear industry. I’m an AI researcher. I do not want my field of research destroyed. Humanity has much to gain from AI, but also everything to lose.”

“Let’s slow down. Let’s make sure that we develop better guardrails, let’s make sure that we discuss these questions internationally just like we’ve done for nuclear power and nuclear weapons. Let’s make sure we better understand these very large systems, that we improve on their robustness and the process by which we can audit them and verify that they are safe for the public.”

A notable article from our dear friend Nick Bostrom mentions the atmospheric auto-ignition story:

https://nickbostrom.com/papers/vulnerable.pdf

Type-0 (‘surprising strangelets’): In 1942, it occurred to Edward Teller, one of the Manhattan scientists, that a nuclear explosion would create a temperature unprecedented in Earth’s history, producing conditions similar to those in the center of the sun, and that this could conceivably trigger a self-sustaining thermonuclear reaction in the surrounding air or water (Rhodes, 1986).

(this goes on for a number of paragraphs)

This whole article has some wild stuff if you haven't seen it before BTW, so buckle up. He also mentions this story in https://nickbostrom.com/existential/risks and https://existential-risk.com/concept.pdf if you want older examples.