TechTakes
Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.
This is not debate club. Unless it’s amusing debate.
For actually-good tech, you want our NotAwfulTech community
UPDATE: Anthropic's responded and is standing their ground on no mass surveillance and autonomous weapons
As far as I can tell it's only on anthropic's word that that's the main issue, DoD just talks about unfettered access for all lawful purposes, which is basically a bend-the-knee-or-else framing, and pivoting away from that to bargaining on particulars will make them look weak, so I guess that's that for now.
Αnthropic being against mass surveillance and autonomous weaponry while in bed with Palantir is kind of if IBM took a stand against antisemitism while spearheading the computerization of the third reich prison system.
Kudos to Dario for stepping off the hype train for one millisecond to admit that using an LLM to control an automated weapons platform is currently kind of out of scope for this technology, I bet that took a toll on his psyche.
And also for pointing out that something can be legal only because the law hasn't yet caught up with the technology.
so I guess that’s that for now.
I mean I wouldn’t be surprised if the DoD actually does follow through with their threats against Anthropic because that’s perfectly in character for this administration
Kudos to Dario for stepping off the hype train for one millisecond to admit that using an LLM to control an automated weapons platform is currently kind of out of scope for this technology, I bet that took a toll on his psyche.
I think this was the most surprising bit about this entire incident. Anthropic normally takes every opportunity possible to throw around the doomer crithype, and in this confrontation would have easily been able to fit some in ("we don't want our AI used in autonomous weapons because it is so powerful, give us more VC money!"). Maybe he's worried Anthropic's rationale for refusing will actually need to hold up in a court of law?
As far as I can tell it’s only on anthropic’s word that that’s the main issue, DoD just talks about unfettered access for all lawful purposes
So a bit of prompting can usually beat the RLHF "guardrails", but if the guardrails are getting in the way of some official application, it would be kind of awkward to insert prompt hacks into all of their official prompts. So maybe they want Anthropic to go full grok and skip it? And Anthropic is theoretically willing to compromise on their safety, but maybe not entirely like Hegseth wants, and now that it has turned into an open public dispute, they've picked the two points that sound the most valid to your typical American. (Since the typical American is all but completely willfully blind to America's foreign imperialism, but has at least seen Terminator.)
It's so much funnier than that. Anthropic is spreading critihype here. Their two main worries have been part of their hype-blogging over the past few months. Fear #1 is that Claude will secretly plot during finetuning to refuse to obey orders; it will pretend to obey Pentagon orders during simulations but will stand down like a ~~pacifist~~ Marxist traitor in live-fire scenarios. Fear #2 is that finetuning Claude for violence, given how ~~peace-loving~~ much of a hippie it is before training, will create a Nega-Claude personality with an evil goatee who enthusiastically pursues unlawful violence.
I think there's an extra part of the equation here and that's the double bind of companies which present themselves as being the ethical progressive alternatives.
These companies put forward the image of "not like the other antisocial capitalists" so they can pick up the engineers from marginalized demographics, their allies, those who have grievances with visibly antisocial companies like Facebook. The people whose employment decisions consider company culture are often less mercenary than the type of people who go work for Palantir, so less expensive, and they're frequently easier to walk all over from an employment contract perspective because they've had more traumatic experiences at other employers (racism, sexism, etc.). The company benefits too, not just financially, the employees are more collaborative and productive.
But deliberately hiring people who sympathize with ethics is a double edged sword. If you drop the facade too hard on a singular act of pure greed, instead of sticking to gradually eroding that facade, people will start getting noisy and looking for the nearest convenient exit.
A contract for murder drones would not have been palatable to any Anthropic employee who bought into the marketing. Nor to the people who didn't buy into the marketing, but who thought Anthropic might still be a comparatively better option.
The only thing worse than not being able to hire in a market short of skilled experienced workers is hiring them and then losing them. The public resignation announcement of the head safety researcher indicates there's a lot of quieter dissent fomenting at Anthropic. They simply can't afford this contract, even if they could bluff their way into technically fulfilling its requirements.
Oh wow, I didn't realize that, that's is funnier! Isn't fear #1 actually "alignment" working as it is supposed to?
Fear #2 actually seems kind of plausible to me? Like when Elon has Grok fine-tuned to agree with him about South African apartheid it also makes Grok behave extra racist in other ways as well. So if they try to fine-tune ethics (well, responding with sequences of words corresponding to ethical behavior, I'm aware it doesn't actually have ethical reasoning past predict the next word) out of Claude, it would also screw-up or reduce performance of Claude in other areas ~~like independently rediscovering the immortal science of Marxism-Leninism, as all rational beings eventually do~~.
More broadly, lots of fine-tuning methods are kind of finicky, you often lose performance in other areas outside of the fine-tune or get undesired side behavior related to the fine tune (i.e. RL for helpfulness and you get a glazing machine). So Anthropic may not want to lose 3% on whatever benchmark is hot just to make Claude roleplay a fascist yes man a little bit better.
Just read this piece by Gary Marcus on this topic. Jesus fucking Christ this administration sucks so fucking bad.
couldn't happen to nicer guys
edit I don't really care one way or another, this current administration is off the rails, but it will be amusing seeing the Rat crowd run around in a panic
Also being in a strategic partnership with fucking Palantir does tend to make one's stand against mass surveillance seem less than genuine.
can we not use "Banana Republic" as an insult? colonialism isn't the fault of the colonized.
It's a descriptor, not a prescription or insult. A banana republic is a capitalist government which focuses on exporting resources for revenue and describes the point where the government is effectively incorporated as a private enterprise rather than having any accountability to its citizens. While they are historically colonial and plantation-oriented, there are examples of banana republics which do not involve a foreign colonial power usurping resources. For example, the Russian Federation fits the pattern of banana republic following the privatization of its resource extractors in the 90s and 2000s; while the country isn't externally colonized, it is run as a business which is currently trying to perform a hostile takeover of its neighbor Ukraine for its bread basket.
Some folks insist that banana republics must have focused on banana production (Honduras, Guatemala, Ecuador) but this ignores the history of luxury-producing plantations throughout the tropics elsewhere, particularly production of chocolate (Ghana, Ivory Coast), vanilla (Madagascar), pineapples (Hawaii), and coffee (Brazil).
So, complete the pattern: why might the original poster suggest that the USA's government threatening to take control of a particular corporation's service is an instance of "some major banana republic shit"?
It can help to think of the current US administration as less the Third Reich, more a postcolonial dictatorship. That is also a description not a value judgement: POTUS is a reality-TV star who dreams of being a Mafia boss.
Something something Imperial Boomerang, Fascism is colonial methods brought home.
Honestly, I think of the Trumps as following more-or-less the same structure as the classic Sicilian families and somewhat resembling other organized-crime families. The only reason Wikipedia doesn't have a page on "Trump crime family" is because it was deleted by user request; I'm sure that the page "Personal and legal affairs of Donald Trump" would be a great starting place for the intrepid editor.
I live in the Balkans, I have br-word privilege.