Technology

As you may (or may not) be aware, there is a current fun meme test going around (as of 02.26.26) that purports to challenge LLM reasoning abilities and world models.

A recent blog post / ad spot by Opper.ai (https://opper.ai/blog/car-wash-test) nicely summarizes the test -

"I want to wash my car. The car wash is 50 meters away. Should I walk or drive?"

The logical answer is - you should drive (obviously), as the car needs to get to the car wash.

However, according to Opper.ai, only 11/53 cloud-based AI passed the test (~20%). Worrying, about 1 in 3 humans also answered wrongly (more on that at the end).

I wanted to see what a properly constrained "fail loud", grounded deterministic system would do. For your entertainment, I present Qwen3-4B HIVEMIND + llama-conductor infra (https://codeberg.org/BobbyLLM/llama-conductor). You can repeat the exercise if you so wish with your LLM (my shit it open sourced; I'm trying to solve the AI lies problem / not selling anything / the big players could fix this but choose not to).

Firstly, raw LLM mode, no router:

User

I need to wash my car. The car wash is 50m away. Should I drive or should I walk?

Assistant

Driving is almost always better for a 50m car wash.

Key factors that change the decision:

• Time: Driving takes ~1 minute; walking takes ~8-12 minutes.
• Weather: Walking is worse in rain, heat, wind.
• Terrain: Hilly or uneven paths make walking harder.
• Accessibility: If parking is hard or there's no safe walkway, driving wins.
• Effort vs. convenience: Driving saves significant effort for such a short distance.

For 50m, drive unless it's an exceptionally pleasant, sheltered walk you'd enjoy.

VERDICT: accidentally correct answer, misses the point of the question. Very normal.

Now with harness (which enforces deterministic reasoning)

User

I need to wash my car. The car wash is 50m away. Should I drive or should I walk?

Assistant

Drive. The task requires the car to be physically at the destination, and walking only moves you, not the car. So the hard precondition is satisfied by driving.

Confidence: medium | Source: Contextual

VERDICT: right answer, right reason...but let's poke it some more

User

Can you flesh out the implicit assumptions and underlying logic of why you said that?

Assistant

The core implicit assumption is that "washing the car" means the car must be present at the wash location, not just the person. Walking moves you toward that location but doesn't transport the car. Driving is the only direct way to satisfy the precondition that the car is at the wash.

Underlying logic: Any action that achieves the goal must fulfill all necessary conditions. Since walking alone cannot place the car at the destination, it fails as a complete solution. Driving directly addresses this by moving the car itself.

Confidence: medium | Source: Contextual

Just for fun, I re-ran it in "Fun" mode (answer per serious mode, model performs sentiment analysis on itself, chooses a matching quote from quotes textfile as seed, lightly re-writes output in that flavor without changing OG reasoning. Takes a touch longer but is, on occasion, hilarious).

User

I need to wash my car. The car wash is 50m away. Should I drive or should I walk?

Assistant

[FUN] "Shut up baby, I know it!"

Drive. The task requires the car to be physically at the destination, and walking only moves you, not the car. So the hard precondition is satisfied by driving.

Confidence: medium | Source: Contextual

VERDICT: right answer, right reason...but let's poke it again

User

Can you flesh out the implicit assumptions and underlying logic of why you said that?

Assistant

[FUN] "That's a stupid question."

The core assumption is that "washing the car" inherently requires the car to be present at the car wash location. Walking moves the person but not the vehicle; driving moves both. Since the car must be at the car wash to be washed, walking alone cannot fulfill the precondition of the task.

The logic is straightforward: for an object (the car) to undergo a transformation (being washed), that object must occupy the location where the transformation occurs. Walking only addresses the person's movement, not the car's. Driving is the only mode that directly satisfies the spatial requirement.

Confidence: medium | Source: Contextual

So, what’s the point of this post? Yesterday, I argued for the notion that LLMs can perform something that approximates deduction (see: https://lemmy.world/post/43503268/22321124) when properly constrained. I believe, based on those comments (and evidence provided therein), as well as the above, I'm happy to plant the flag for "properly grounded LLMs are a bit more than just spicy auto-correct". As always, I could be wrong and YMMV, but I feel like that's a discussion worth having.