Technology

Reasoning is woke propaganda anyway.

Grok isn't designed to solve problems. It's designed to create sexually explicit images of children for Republicans...

As a technologist, I have to remind everyone that AI is not intelligence. It's a word prediction/statistical machine. It's guessing at a surprisingly good rate what words follow the words before it.

It's math. All the way down.

We as humans have simply taken these words and have said that it is "intelligence".

Here is a way of describing what I see as 'the problem':

An LLM cannot forget things in its base training data set.

Its permanent memory... is totally permanent.

And this memory has a bunch of wrong ideas, a bunch of nonsensical associations, a bunch of false facts, a bunch of meaningless gibberish.

It has no way of evaluating its own knowledge set for consistency, coherence, and stability.

It literally cannot learn and grow, because it cannot realize why it made mistakes, it cannot discard or ammend in a permanent way, concepts that are incoherent, faulty ways of reasoning (associating) things.

Seriously, ask an LLM a trick question, then tell it it was wrong, explain the correct answer, then ask it to determine why it was wrong.

Then give it another similar category of trick question, but that is specifically different, repeat.

The closer you try to get it toward reworking a fundamental axiom it holds to that is flawed, the closer it gets to responding in totally paradoxical, illogical gibberish, or just stuck in some kind of repetetive loop.

... Learning is as much building new ideas and experiences, as it is reevaluating your old ideas and experiences, and discarding concepts that are wrong or insufficient.

Biological brains have neuroplasticity.

So far, silicon ones do not.

Are you anthromorphizing word suggester into a being experiencing things?

Funnier yet will be if they continue to just train the model on that particular kind of test, invalidating its results in the process.

When Sammy fuck says “we’re so close to AGI, I can just feel it. Like a tingle on the tip of my shrimpdick it’s getting so close to blossoming into something guys”, just ignore him. He’s crazy man!

"Sam Altman claims AGI is coming in 2025 and machines will be able to 'think like humans' when it happens"

We're anti-AI because AI is fucking stupid. Both literally and figuratively.

I couldn't find replays. Are there more? Also, it is a bit funny that "building the bridge" which at one point seems to be Claude's "chosen goal" is just "running out of moves" and failing the task.

Task failed successfully, Claude. Task failed, successfully.

I assume it’s an hourly wage or something. Just because humans can work for free if they choose, doesn’t mean they have no cost associated with them. Just like a company could choose to give away unlimited tokens, those tokens still have a standard cost.

Thatvis how much individual testing humans cost when you buy them in bulk.

it's also an odd metric since only 20-60% of the humans completed it. Very 60% of the time they complete it everytime energy.

Ideally they'd run the bots multiple times through (with no context or training of previous run), but I guess that is cost prohibitive?

Well... It cannot even search databases without errors.

LLMs just produce plausible replies in natural languages very quickly and this is useful in certain situations. Sometimes it helps humans getting started with a task, but as it is now, it cannot replace them. As much as the capital class want it, and sink our money into it.

You can really only judge fairness of the score if you understand the scoring criteria. It is a relative score where the baseline is 100% for humans -- i.e. A task was only included in the challenge if at least two people in the panel of humans were able to solve it completely, and their action count is a measure of efficiency. This is the baseline used as a point of comparison.

From the Technical Report:

The procedure can be summarized as follows:
• “Score the AI test taker by its per-level action efficiency” - For each level that the test taker completes, count the number of actions that it took.
• “As compared to human baseline” - For each level that is counted, compare the AI agent’s action count to a human baseline, which we define as the second-best human action count. Ex: If the second-best human completed a level in only 10 actions, but the AI agent took 100 to complete it, then the AI agent scores (10/100)^2 for that level, which gets reported as 1%. Note that level scoring is calculated using the square of efficiency.
• “Normalized per environment” - Each level is scored in isolation. Each individual level will get a score between 0% (very inefficient) 100% (matches or surpasses human level efficiency). The environment score will be a weighted-average of level score across all levels of that environment.
• “Across all environments” - The total score will be the sum of individual environment scores divided by the total number of environments. This will be a score between 0% and 100%.

So the humans "scored 100%" because that is the baseline by definition, and the AIs are evaluated at how close they got to human correctness and efficiency. So a score of 0.26% is 1/0.0026 ~= 385 times less efficient (and correct) compared to humans.