this post was submitted on 08 Jun 2025
80 points (100.0% liked)

TechTakes

1933 readers
50 users here now

Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

founded 2 years ago
MODERATORS
top 17 comments
sorted by: hot top controversial new old
[–] YourNetworkIsHaunted@awful.systems 16 points 1 day ago (1 children)

As the bioware nerd I am it makes my heart glad to see the Towers of Hanoi doing their part in this fight. And it seems like the published paper undersells how significant this problem is for the promptfondlers' preferred narratives. Given how simple it is to scale the problem complexity for these scenarios, it seems likely that there isn't a viable scaling-based solution here. No matter how big you make the context windows and how many steps the system is able to process it's going to get out scaled by simply increasing some Ns in the puzzle itself.

Diz and others with a better understanding of what's actually under the hood have frequently referenced how bad Transformer models are at recursion and this seems like a pretty straightforward way to demonstrate that and one that I would expect to be pretty consistent.

[–] Soyweiser@awful.systems 3 points 6 hours ago (1 children)

Sorry what is the link between bioware and towers of hanoi? (I do know about the old "one final game before your execution" science fiction story).

So I don't know if it's a strong link, but I definitely learned to solve the Towers when playing through KotOR, then had it come up again in Mass Effect, and Jade Empire, both of which I played at around the same time. From a quick "am I making this up?" search, it's also used in a raid in SW:TOR, and gets referenced throughout the dragon age and mass effect franchises even if not actually deployed.

[–] diz@awful.systems 15 points 1 day ago* (last edited 1 day ago) (2 children)

Further support for the memorization claim: I posted examples of novel river crossing puzzles where LLMs completely fail (on this forum).

Note that Apple’s actors / agents river crossing is a well known “jealous husbands” variant, which you can ask a chatbot to explain to you. It gladly explains, even as it can’t follow its own explanation (since of course it isn’t its own explanation but a plagiarized one, even if changes words).

edit: https://awful.systems/post/4027490 and earlier https://awful.systems/post/1769506

I think what I need to do is to write up a bunch of puzzles, assign them randomly to 2 sets, and test & post one set, while holding back on the second set (not even testing it on any online chatbots). Then in a year or two see how much the set that's public improves, vs the one that's held back.

[–] Soyweiser@awful.systems 1 points 6 hours ago

Latter test fails if they write a specific bit of code to put out the 'llms fail the river crossing' fire btw. Still a good test.

[–] YourNetworkIsHaunted@awful.systems 7 points 1 day ago (1 children)

That would be the best way to actively catch the cheating happening here, given that the training datasets remain confidential. But I also don't know that it would be conclusive or convincing unless you could be certain that the problems in the private set were similar to the public set.

In any case either you're doubledipping for credit in multiple places or you absolutely should get more credit for the scoop here.

[–] diz@awful.systems 6 points 1 day ago

I’d just write the list then assign randomly. Or perhaps pseudorandomly like sort by hash and then split in two.

One problem is that it is hard to come up with 20 or more completely unrelated puzzles.

Although I don’t think we need a large number for statistical significance here, if it’s like 8/10 solved in the cheating set and 2/10 in the hold back set.

[–] scruiser@awful.systems 12 points 1 day ago (1 children)

Another thing that's been annoying me about responses to this paper... lots of promptfondlers are suddenly upset that we are judging LLMs by abitrary puzzle solving capabilities... as opposed to the arbitrary and artificial benchmarks they love to tout.

[–] diz@awful.systems 10 points 1 day ago* (last edited 1 day ago)

Yeah any time its regurgitating an IMO problem it’s a proof it’salmost superhuman, but any time it actually faces a puzzle with unknown answer, this is not what it is for.

[–] scruiser@awful.systems 28 points 2 days ago* (last edited 1 day ago) (3 children)

The promptfondlers on places like /r/singularity are trying so hard to spin this paper. "It's still doing reasoning, it just somehow mysteriously fails when you it's reasoning gets too long!" or "LRMs improved with an intermediate number of reasoning tokens" or some other excuse. They are missing the point that short and medium length "reasoning" traces are potentially the result of pattern memorization. If the LLMs are actually reasoning and aren't just pattern memorizing, then extending the number of reasoning tokens proportionately with the task length should let the LLMs maintain performance on the tasks instead of catastrophically failing. Because this isn't the case, apple's paper is evidence for what big names like Gary Marcus, Yann Lecun, and many pundits and analysts have been repeatedly saying: LLMs achieve their results through memorization, not generalization, especially not out-of-distribution generalization.

[–] paraphrand@lemmy.world 12 points 1 day ago* (last edited 1 day ago) (2 children)

prompfondlers

Holy shit, I love it.

[–] blakestacey@awful.systems 16 points 2 days ago (1 children)
[–] scruiser@awful.systems 6 points 1 day ago

Just one more training run bro. Just gotta make the model bigger, then it can do bigger puzzles, obviously!

[–] Architeuthis@awful.systems 8 points 1 day ago* (last edited 1 day ago)

Hey now, there's plenty of generalization going on with LLM networks, it's just that we've taken to calling it hallucinations these days.

[–] MadMadBunny@lemmy.ca 6 points 2 days ago

So, not intelligent, just artificial?