MangoCats

people re-inventing the wheel because it’s “easier” than searching without properly understand the cost of the entire process.

A good LLM will do a web search first and copy its answer from there...

making something quick that kind of works is nice… but why even do so in the first place if it’s already out there, maybe maintained but at least tested?

In a sense, this is what LLMs are doing for you: regurgitating stuff that's already out there. But... they are "bright" enough to remix the various bits into custom solutions. So there might already be a NWS API access app example, and a Waveshare display example, and so on, but there's not a specific example that codes up a local weather display for the time period and parameters you want to see (like, temperature and precipitation every 15 minutes for the next 12 hours at a specific location) on the particular display you have. Oh, and would you rather build that in C++ instead of Python? Yeah, LLMs are actually pretty good at remixing little stuff like that into things you're not going to find exact examples of ready to your spec.

I have a little display on the back of a Raspberry Pi Zero W - it recoded that display software to refresh 5x faster, and it updated the content source to move from Meteomatics (who just discontinued their free API) to the National Weather Service.

As an experiment I asked Claude to manage my git commits, it wrote the messages, kept a log, archived excess documentation, and worked really well for about 2 weeks. Then, as the project got larger, the commit process was taking longer and longer to execute. I finally pulled the plug when the automated commit process - which had performed flawlessly for dozens of commits and archives, accidentally irretrievably lost a batch of work - messed up the archive process and deleted it without archiving it first, didn't commit it either.

AI/LLM workflows are non-deterministic. This means: they make mistakes. If you want something reliable, scalable, repeatable, have the AI write you code to do it deterministically as a tool, not as a workflow. Of course, deterministic tools can't do things like summarize the content of a commit.

I also cannot understand and debug code written by me.

So much this. I look back at stuff I wrote 10 years ago and shake my head, console myself that "we were on a really aggressive schedule." At least in my mind I can do better, in practice the stuff has got to ship eventually and what ships is almost never what I would call perfect, or even ideal.

I think the point is that someone should understand the code. In this case, no one does.

Big corporations have been pushing for outsourcing software development for decades, how is this any different? Can you always recall your outsourced development team for another round of maintenance? A LLM may actually be more reliable and accessible in the future.

This stuff ALWAYS ends up destroying the world on TV.

TV is also full of infinite free energy sources. In the real world warp drive may be possible, you just need to annihilate the mass of Jupiter with an equivalent mass of antimatter to get the energy necessary to create a warp bubble to move a small ship from the orbit of Pluto to a location a few light years away, but on TV they do it every week.

your team of AIs keeps running circles

Depending on your team of human developers (and managers), they will do the same thing. Granted, most LLMs have a rather extreme sycophancy problem, but humans often do the same.

We haven’t gotten yet to AIs who will tell you that what you ask is impossible.

If it's a problem like under or over-constrained geometry or equations, they (the better ones) will tell you. For difficult programing tasks I have definitely had the AIs bark up all the wrong trees trying to fix something until I gave them specific direction for where to look for a fix (very much like my experiences with some human developers over the years.)

I had a specific task that I was developing in one model, and it was a hard problem but I was making progress and could see the solution was near, then I switched to a different model which did come back and tell me "this is impossible, you're doing it wrong, you must give up this approach" up until I showed it the results I had achieved to-date with the other model, then that same model which told me it was impossible helped me finish the job completely and correctly. A lot like people.

AI already thought it gave you the best answer, so going back to the problem for a proper answer is probably not going to work.

There's an LLM concept/parameter called "temperature" that determines basically how random the answer is.

As deployed, LLMs like Claude Sonnet or Opus have a temperature that won't give the same answer every time, and when you combine this with feedback loops that point out failures (like compliers that tell the LLM when its code doesn't compile), the LLM can (and does) the old Beckett: try, fail, try again, fail again, fail better next time - and usually reach a solution that passes all the tests it is aware of.

The problem is: with a context window limit of 200,000 tokens, it's not going to be aware of all the relevant tests in more complex cases.

True that LLMs will accept almost any task, whether they should or not. True that their solutions aren't 100% perfect every time. Whether it's faster to use them or not I think depends a lot on what's being done, and what alternative set of developers you're comparing them with.

What I have seen across the past year is that the number of cases where LLM based coding tools are faster than traditional developers has been increasing, rather dramatically. I called them near useless this time last year.

pretty close to how you’d work with actual humans

That has been my experience as well. It's like working with humans who have extremely fast splinter skills, things they can rip through in 10 minutes that might take you days, weeks even. But then it also takes 5-10 minutes to do some things that you might accomplish in 20 seconds. And, like people, it's not 100% reliable or accurate, so you need to use all those same processes we have developed to help people catch their mistakes.

I think we’re barreling towards a place where remotely complicated software becomes a lost technology

I think complicated software has been an art more than a science, for the past 30 years we have been developing formal processes to make it more of a procedural pursuit but the art is still very much in there.

I think if AI authored software is going to reach any level of valuable complexity, it's going to get there with the best of our current formal processes plus some more that are being (rapidly) developed specifically for LLM based tools.

But eventually you will hit a limit. You’ll need to do something...

And how do we surpass those limits? Generally: research. And for the past 20+ years where do we do most of that research? On the internet. And where were the LLMs trained, and what are they relatively good at doing quickly? Internet research.

At the end of the day, coding is a skill. If no one is building the required experience to work with complex systems

So is semiconductor design, application of transistors to implement logic gates, etc. We still have people who can do that, not very many, but enough. Not many people work in assembly language anymore, either...