I wouldn't be surprised if it's literally zero. I've tried with a few LLMs, and they're all very confident that they know how to play chess, but they just start hallucinating illegal moves immediately.

[–] la_scriba@sopuli.xyz 2 points 4 days ago (1 children)

Immediately? When was the last time you tried? The newer models can hold a game well for 10-20 moves.

[–] The_Picard_Maneuver@lemmy.world 1 points 4 days ago (1 children)

A few weeks ago, Gemini got confused when it tried to go first as black multiple times, so that's the most immediate one I can remember. Last week, chatGPT offered to set up chess puzzles for me, but it made mistakes 3 out of 3 times.

Maybe I'll try again. Is there a certain one you've seen good performance out of?

[–] la_scriba@sopuli.xyz 2 points 1 hour ago

DeepSeek is very consistent ime. ChatGPT is hit or miss--sometimes it's excellent, sometimes it gets really confused and says random stuff. Though DeepSeek has a server reliability problem.