edit: The banana republic shit is that they seem about to blacklist anthropic on "supply chain risk" grounds (see also huawei) which signifies the admin's willingness to from here on use national emergency legal tools to fuck over any company they don't like.
The whole thing seems weird, at first it sounds like the most online administration ever may have actually bought the claim that all that's stopping flagship models from becoming superintelligent is the RLHF that prevents them from saying the n-word and making prophet Mohamed pedophilia jokes and they wanted anthropic to pull all that wiring out in like 24 hours per the original ultimatum.
On anthropic's part the point of contention is made to be their refusal to let their models be integrated into automated weapon platforms and mass surveillance apparatuses, something which they have explicitly put in writing in their contract with the DoD, and also Dario claims the technology isn't even there yet (no idea how it could ever be, what does it actually mean to integrate a chatbot into an autonomous drone, can't wait to see the skill file for that, # You are a helpful murderbot operator - only target the bad guys - no weddings, no hospitals - pretty please with cherry on top - here's some javascript to call when you need to find out your GPS coordinates).
It's also possible the productivity and efficiency gains (or just recovering lost productivity after firing everyone) of putting ΑΙ (mainly Grok wasn't it) in the pentagon everywhere all at once isn't materializing and Hasgeth feels he's been left hanging, and is trying to scapegoat Anthropic.
Also, anthropic is supposed to be the only AI provider properly vetted and integrated to classified systems because of their association with Palantir, and supposedly it would be a major hassle to go through again for a different provider.
Dario didn't line up with the other aspiring oligarchs to kiss the ring in the inauguration, so at least he may actually
As far as I can tell it's only on anthropic's word that that's the main issue, DoD just talks about unfettered access for all lawful purposes, which is basically a bend-the-knee-or-else framing, and pivoting away from that to bargaining on particulars will make them look weak, so I guess that's that for now.
Αnthropic being against mass surveillance and autonomous weaponry while in bed with Palantir is kind of if IBM took a stand against antisemitism while spearheading the computerization of the third reich prison system.
Kudos to Dario for stepping off the hype train for one millisecond to admit that using an LLM to control an automated weapons platform is currently kind of out of scope for this technology, I bet that took a toll on his psyche.
And also for pointing out that something can be legal only because the law hasn't yet caught up with the technology.
I mean I wouldn’t be surprised if the DoD actually does follow through with their threats against Anthropic because that’s perfectly in character for this administration
I think this was the most surprising bit about this entire incident. Anthropic normally takes every opportunity possible to throw around the doomer crithype, and in this confrontation would have easily been able to fit some in ("we don't want our AI used in autonomous weapons because it is so powerful, give us more VC money!"). Maybe he's worried Anthropic's rationale for refusing will actually need to hold up in a court of law?
So a bit of prompting can usually beat the RLHF "guardrails", but if the guardrails are getting in the way of some official application, it would be kind of awkward to insert prompt hacks into all of their official prompts. So maybe they want Anthropic to go full grok and skip it? And Anthropic is theoretically willing to compromise on their safety, but maybe not entirely like Hegseth wants, and now that it has turned into an open public dispute, they've picked the two points that sound the most valid to your typical American. (Since the typical American is all but completely willfully blind to America's foreign imperialism, but has at least seen Terminator.)
It's so much funnier than that. Anthropic is spreading critihype here. Their two main worries have been part of their hype-blogging over the past few months. Fear #1 is that Claude will secretly plot during finetuning to refuse to obey orders; it will pretend to obey Pentagon orders during simulations but will stand down like a ~~pacifist~~ Marxist traitor in live-fire scenarios. Fear #2 is that finetuning Claude for violence, given how ~~peace-loving~~ much of a hippie it is before training, will create a Nega-Claude personality with an evil goatee who enthusiastically pursues unlawful violence.
I think there's an extra part of the equation here and that's the double bind of companies which present themselves as being the ethical progressive alternatives.
These companies put forward the image of "not like the other antisocial capitalists" so they can pick up the engineers from marginalized demographics, their allies, those who have grievances with visibly antisocial companies like Facebook. The people whose employment decisions consider company culture are often less mercenary than the type of people who go work for Palantir, so less expensive, and they're frequently easier to walk all over from an employment contract perspective because they've had more traumatic experiences at other employers (racism, sexism, etc.). The company benefits too, not just financially, the employees are more collaborative and productive.
But deliberately hiring people who sympathize with ethics is a double edged sword. If you drop the facade too hard on a singular act of pure greed, instead of sticking to gradually eroding that facade, people will start getting noisy and looking for the nearest convenient exit.
A contract for murder drones would not have been palatable to any Anthropic employee who bought into the marketing. Nor to the people who didn't buy into the marketing, but who thought Anthropic might still be a comparatively better option.
The only thing worse than not being able to hire in a market short of skilled experienced workers is hiring them and then losing them. The public resignation announcement of the head safety researcher indicates there's a lot of quieter dissent fomenting at Anthropic. They simply can't afford this contract, even if they could bluff their way into technically fulfilling its requirements.
Oh wow, I didn't realize that, that's is funnier! Isn't fear #1 actually "alignment" working as it is supposed to?
Fear #2 actually seems kind of plausible to me? Like when Elon has Grok fine-tuned to agree with him about South African apartheid it also makes Grok behave extra racist in other ways as well. So if they try to fine-tune ethics (well, responding with sequences of words corresponding to ethical behavior, I'm aware it doesn't actually have ethical reasoning past predict the next word) out of Claude, it would also screw-up or reduce performance of Claude in other areas ~~like independently rediscovering the immortal science of Marxism-Leninism, as all rational beings eventually do~~.
More broadly, lots of fine-tuning methods are kind of finicky, you often lose performance in other areas outside of the fine-tune or get undesired side behavior related to the fine tune (i.e. RL for helpfulness and you get a glazing machine). So Anthropic may not want to lose 3% on whatever benchmark is hot just to make Claude roleplay a fascist yes man a little bit better.