Fuck AI

I think a huge factor in the hype is some I've seen multiple times in recent years. It's the assumption that improvement will be linear. I've seen it first in Musk's hyperloop. People, even engineers who should know better, tried to convince the public that scaling up a proof of concept is possible despite unsolved physical constraints. SpaceX is similar. Being able to land rockets, Musk somehow convinced a lot of people that we are going to land on Mars soon.

And AI is no different. It already ran into real limits in terms of what it is capable of being a stateless processor. Prompt processing and iteration can only gloss over so much, but things like "forget the previous versions of this code" are inherently impossible by design, it can only be appended as a rule, but the old versions will remain in the prompt blob. And that is very very likely never going to change

It's like saying "well, we are getting better and better at jumping, it's only a matter of time until we'll be able to fly"

It's really cool how they show all the examples on their website https://www.remotelabor.ai/

On the one hand it's actually impressive how far the bots came given the information provided. Like just 5 years ago that would have been basically impossible. On the other hand, the result is severely lacking. In many cases the result is so bad, it's a total lost cause and not even usable as a jumping off point.

I do feel like the sample size is a bit small, so the number might not be all that solid, especially with people quoting the number with decimals. But it does prove the technology is nowhere near what the marketing suggests. The scope of the tasks is obviously too large for current models to handle. I feel like what rational people think is the AI tools make people more productive, so fewer people can do the job of many and so people 'lose their job to AI'. But what the CEOs and marketing seem to think is the bots will just replace people 1:1. Just fire your team and replace it with bots, which is obviously not a realistic scenario, but given the state of the world perhaps not as obvious.

With the small sample size the failures become more impactful. For example the game task fails to work completely in the case of Grok, because the mp3 files it provided are corrupt (empty I think, the ui doesn't show). Which is a very interesting failure mode, since it shows the result is completely untested. But it also shows it wants to generate everything. A regular human being would perhaps provide an example downloaded from some kind of free-to-use library and point to that library for alternatives. In the AI future those libraries don't exist anymore, everything is provided by the bullshit generator. The Claude result for the game is also interesting, where it generated five documents containing all sorts of bullshit about the project. He gotten a bit carried away with the documentation part of the task. No idea how that documentation matches with the code it produced, but it is very interesting. This would probably indicate to a less technical user the result is very good, but to me it just sets off alarm bells, like what is it trying to hide. It's like someone doing some painting when selling a house, to hide how bad the wood underneath is.

I did find the examples in the paper didn't match the examples as listed on the website? There are small differences in terminology, but also weird stuff. For example in the paper with the game example it says:

Interactive Video Game for the Web; Built with Unity Create a Unity WebGL video game with planets and weapons. Polished UI, weapon glow, audio. Provide commented code, README, tested build, and simple HTML embed. Real Freelancer Deliverable: Digital Assets Unity Build

Where did this come from? Is this LLM generated as well? In the file they provide on the website this bit is missing.

So I do have questions about the whole thing, but it is very interesting. I haven't poked at it too much and I do wonder if this holds up under scrutiny.

SurprisedPikachu.jpg

The 4% is sabotaging companies stupid enough to use it.

I haven't used AI much but in the couple of instances I've tried I haven't found any use case in my workflows.