this post was submitted on 16 Sep 2024

94 points (100.0% liked)

TechTakes

1483 readers

108 users here now

Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

founded 1 year ago

MODERATORS

dgerard@awful.systems

AI-Generated Code is Causing Outages and Security Issues in Businesses (www.techrepublic.com)

submitted 3 months ago by jaschop@awful.systems to c/techtakes@awful.systems

35 comments fedilink hide all child comments

cross-posted from: https://sh.itjust.works/post/25239919

all 36 comments

sorted by: hot top controversial new old

[–] swlabr@awful.systems 35 points 3 months ago (3 children)

have they tried writing better prompts? my lived experience says that because it works for me, it should work as long as you write good prompts. prompts prompts prompts. I am very smart. /s

[–] luciole@beehaw.org 27 points 3 months ago* (last edited 3 months ago) (3 children)

Oh wow. The article says basically that but without the /s and then it gets even better. This is according to Mister AI Professor Ethan Mollick From The University Of Warthon and the link goes to a tweet (the highest form of academia) saying:

The problem with calling “prompt engineering” a form of programming is that it isn’t like what we call coding

In fact, coders are often bad at prompting because AI doesn’t do things consistently or work like code. The best prompters I know can’t code at all. They “teach” the AI.

Which is just great considering the next excuse in the text is:

this is due to insufficient reviews, either because the company has not implemented robust code quality and code-review practices, or because developers are scrutinising AI-written code less than they would scrutinise their own code

So who the fuck even reviews the prompt engineers’ code sludge, Mister AI Professor Of Twitter?

Whole text is such a sad cope.

[–] V0ldek@awful.systems 15 points 3 months ago (1 children)

developers are scrutinising AI-written code less than they would scrutinise their own code

Wait, is this how Those People claim that Copilot actually "improved their productivity"? They just don't fucking read what the machine output?

I was always like "how can Copilot make me code faster if all it does is give me bad code to review which takes more than just writing it" and the answer is "what do you mean review"????

[–] arbitraryidentifier@awful.systems 13 points 3 months ago

Wait, is this how Those People claim that Copilot actually “improved their productivity”? They just don’t fucking read what the machine output?

Yes, that's exactly what it is. That and boilerplate, but it probably makes all kinds of errors that they don't noticed, because the build didn't fail.

[–] gerikson@awful.systems 15 points 3 months ago (1 children)

Programmers hate programming and love code reviewing, right? Right?

[–] Soyweiser@awful.systems 11 points 3 months ago (1 children)

Soon they will try to fix this problem by having 2 forms of LLM do team coding. The surprised Pikachu faces will be something

[–] arbitraryidentifier@awful.systems 11 points 3 months ago

Looking forward to the LLM vs LLM PRs with hundreds of back and forth commit-request changes-commit cycles. Most of it just flipping a field between final and not final.

[–] Soyweiser@awful.systems 8 points 3 months ago* (last edited 3 months ago)

I didn't even read the article. Still believe in the prompts.

[–] pikesley@mastodon.me.uk 16 points 3 months ago (1 children)

@swlabr @jaschop

I fixed the quote from the article "programmers are not known for being great at writing prompts because many of us find the whole idea offensive and stupid"

[–] YourNetworkIsHaunted@awful.systems 18 points 3 months ago

I'm reminded of the guy in a previous thread who claimed LLMs helped him as a rubber duck partner. You know - the troubleshooting technique named for its efficacy when working with a bath toy.

[–] regrub@lemmy.world 8 points 3 months ago

Prompt engineering is the same as software engineering, right?

[–] sailor_sega_saturn@awful.systems 20 points 3 months ago (2 children)

Someone will have the "brilliant" idea to fix this by having chatbots review code in 5... 4... 3...

[–] swlabr@awful.systems 13 points 3 months ago

Welcome to my new startup where we train LLMs on compiled binaries. Now you can just prompt and get a complete executable, no coding knowledge needed. We value our company at $5b, product launch date indeterminate

[–] froztbyte@awful.systems 8 points 3 months ago (1 children)

I could swear I’ve seen a shartup with this pitch

will try check tomorrow, rn I’m enjoying the sounds of the first thunderstorm of the season

[–] sailor_sega_saturn@awful.systems 9 points 3 months ago* (last edited 3 months ago) (4 children)

Thanks now you've sent me down the rabbit hole since I searched for this and clicked on the first ad: coderabbit.ai

One of the code reviews they feature on their homepage involves poor CodeRabbit misspelling a variable name, and then suggesting the exact opposite code of what would be correct for a "null check" (Suggesting if (object.field) return; when it should have suggested if (!object.field) return; or something like that).

You'd think AI companies would have wised up by this point and gone through all their pre-recorded demos with a fine comb so that ~~marks~~ users at least make it past the homepage, but I guess not.

Aside: It's not really accurate to describe if (object.field) as a null check in JS since other things like empty strings will fail the check, but maybe CodeRabbit is just an adorable baby JS reviewer!

Aside: the example was in a .jsx file. Does that stand for JavaScript XML? because oh lord that sounds cursed

[–] Architeuthis@awful.systems 7 points 3 months ago

You’d think AI companies would have wised up by this point and gone through all their pre-recorded demos with a fine comb so that ~~marks~~ users at least make it past the homepage, but I guess not.

The target group for their pitch probably isn't people who have a solid grasp of coding, I'd bet quite the opposite.

[–] kuberoot@discuss.tchncs.de 6 points 3 months ago (2 children)

JSX is JavaScript, but you can also just put HTML in it (with bonus syntax for embedding more JS expressions inside) and it can get transpiled into function calls, which means it'll result in an object structure representing the HTML you wrote. It's used so that you can write a component as a function that returns HTML with properties already computed in and any special properties, like event listeners, passed as function references contained in the structure.

[–] froztbyte@awful.systems 7 points 3 months ago

that's a hell of a lot of words for "is a giant pile of mistakes"

[–] pikesley@mastodon.me.uk 4 points 3 months ago

@kuberoot @sailor_sega_saturn this is very upsetting

[–] froztbyte@awful.systems 5 points 3 months ago

sorry, the reality is worse

[–] sc_griffith@awful.systems 3 points 3 months ago (1 children)

why do so many awful tech companies have rabbit in their names

[–] sailor_sega_saturn@awful.systems 4 points 3 months ago

Because rabbits are cute and fluffy and good and it is the solemn mission of all terrible tech companies to take the things you love and make you associate them with useless AI products.

[–] Architeuthis@awful.systems 20 points 3 months ago

"When asked about buggy AI [code], a common refrain is ‘it is not my code,’ meaning they feel less accountable because they didn’t write it.”

Strong they cut all my deadlines in half and gave me an OpenAI API key, so fuck it energy.

He stressed that this is not from want of care on the developer’s part but rather a lack of interest in “copy-editing code” on top of quality control processes being unprepared for the speed of AI adoption.

You don't say.

[–] zbyte64@awful.systems 11 points 3 months ago

For some reason when I read this I am reminded of our "highly efficient rail" which often derails

[–] NigelFrobisher@aussie.zone 8 points 3 months ago

LLMs will save us from having to work on features now that we nearly ironed out all the issues introduced by Kubernetes.

[–] pyre@lemmy.world 7 points 3 months ago

"i don't know what happened, the truck was cruising just fine when we put the toddler on the wheel"

[–] n3m37h@sh.itjust.works 7 points 3 months ago (1 children)

Buahahahaha, lazy fucks just do the work

[–] froztbyte@awful.systems 8 points 3 months ago

as I have said here some time ago, these chucklefucks are a goldmine waiting to happen. just not the kind of gold they think.

[+] celsiustimeline@lemmy.dbzer0.com 3 points 3 months ago* (last edited 2 months ago) (1 children)

[deleted]

[–] towerful@programming.dev 8 points 3 months ago (2 children)

Reminds me of the story of the old engineer asked to come in and fix some machine in a factory.

The engineer inspects the machine, marks it with some chalk, then strikes the chalk mark with a hammer.
The machine works again.
The company asks for an itemised invoice after seeing the initial invoice for $10k.
To which they received:

hitting chalk mark with hammer: $1.

knowing where to place the chalk mark: $9,999

GPT suffers from garbage-in garbage-out just as much as a search engine does.
Knowing how to find search results to fix your specific situation is a skill.
Utilising GPT for such a task is equally a skill. With the added bonus of GPT randomly pulling the perfect API/Library out of its ass

[–] froztbyte@awful.systems 8 points 3 months ago

on a slight tangent, I often think about this piece of writing. in general, but I've also started wondering what that picture's going to look like after the tsunami of LLMs suddenly finds it's actually made of air and not water

[–] zalgotext@sh.itjust.works -3 points 3 months ago (3 children)

Yeah I feel like once people realize AI chatbots like ChatGPT are largely just search engines with AutoTldrBot built in, they'll be better at using them. ChatGPT is great for bouncing ideas off of or rubber-ducking through a solution. But just like with StackOverflow answers, you as the developer need to be able to recognize when ChatGPT is just spouting garbage, when it's getting you close to the answer, what adjustments you need to make to make its answers work for your situation, etc. In it's current state, it will never just magically hand you a fully developed, robust, well-integrated, complete solution though, as much as tech CEOs want it to.

[–] gerikson@awful.systems 9 points 3 months ago (1 children)

Sounds like a great solution people will be prepared to pay OpenAI $100B in the future for, and not at all like an incremental upgrade over StackOverflow with extra ecocide added.

[–] zalgotext@sh.itjust.works 8 points 3 months ago

Yeah after rereading my comment it's not super clear, but I'm not trying to endorse ChatGPT/OpenAI. I agree that AI is a pretty terrible solution for the use case of "search engine with a built in AutoTldrBot", because of all the reasons you mention. I was just trying to point out that it's being marketed as a replacement for actual software developers, and that's very very very far from reality at the moment.

[–] froztbyte@awful.systems 8 points 3 months ago

you as the developer need to be able to recognize when ChatGPT is just spouting garbag

easy: all the time

[–] towerful@programming.dev 3 points 3 months ago

GPT and the whole AI bs we have at the moment excels at being convincing. It's even prepared to back up what it says.
The problem is, that all of that is generated. Not necessarily fact.
It will generate API methods, entire libraries, sources, legal cases, and science publications.
And it will be absolutely convincing as it presents and backs up those claims.

For example, GPT gives some API function of some library that magically solves your issue. Maybe you aren't hugely familiar with the library, but you don't trust GPT - so you research this made up API method and find the actual way to do it. Except you have GPT saying this exists and it works the way you want it to. So you research more, dig deeper.
Eventually you end up reading the source code, have a deeper understanding of the API in general and how to actually find useful answers (IE how to search query for it), and end up using the method you found while trying to find the mythical perfect API method.
I mean, I guess that's a win? You learned some documentation, you solved the problem... Who cares?

Maybe I'm just bitter because that was how I first tried any of the new AI things. And I wasted 2-3 hours instead of actually solving the fucking problem by consulting the facts.