Pointless rant. Please ignore. I'm a software developer and we all know how AI has changed our industry. How we work or why we're fired and why we can't afford PCs.
Anyways, we're already all forced to use AI already and we're already atrophying the minds of our juniors. It's great.
New team meeting and one of our managers tells us that we're never going to write code anymore at all. The AI will read the JIRA ticket and create the pull request (change request to the codebase) on GitHub. Our job is to only review the code on GitHub and then rank how well AI did and then comment and then get AI to fix it. We have to do this so we can improve the AI process. Which is funny because none of the people who plan this AI shit are data scientists. The only way they can change things is by promoting, it's not like we're releasing our own coding models but anyways ... He's like, now you should be able to do much more work and just review PRs all day now and that we should never be doing only one thing. You can only tell AI through a GitHub comment to fix a mistake and then you can start reviewing the next thing.
We were like, if it's a simple fix why can't we just fix it?
"Because we need to improve the AI process"
But then, I have to context switch.
"Yes that's the point you can come back to it later"
Why come back to it later when we can solve it now? We can even use AI to solve it now.
"No, we want you just comment on the PR so the bot can handle it"
Context switching is free apparently... It's actually infuriating because apparently we're not using IDEs any more. I personally use the GitHub plugin to review PRs in my IDE but no one else seems to do it so I don't think they even took that into account.
These guys have auto merged AI code that's taken us weeks to unravel and which we still haven't fully been able to fix. They just merge shit all the time and a lot of it is fucking slip. AI merged hundreds of tests and no one cares when they break. They didn't configure prettier because AI doesn't use it so it breaks out formatting when humans do it.
I ranted to my own manager for 30 minutes about it today and he was just as upset because every developer is now asking what exactly are they doing. My manager asked me what I would do. I said the process sucks but what are we supposed to do as devs. If I review 20 PRs a day, how is the company going to ensure my skills are gonna be sharp? What are we doing about taking in ideas from regular devs? How do we ensure code ownership when we're just merging tickets we don't write and code we had no hand in shaping?
Sorry. I actually thought I had faith in my company with AI because they were coming up with thoughtful approaches but it seems like utter incompetence.
Most of the things that try to address this do so by adding information to the context window. Doesn't solve the fundamental issues, like LLMs still being solely dependent on word/token correlations and the assumption that if you throw enough conversations at it and do statistical analysis on the words used, you can encode the specific knowledge behind those words used into those statistics and then models can get back from those generalized statistics to discuss specific topics (which isn't really how statistics work).
Well that’s often counterproductive, too. Even the largest models degrade if you fill their context with stuff they don’t need, or push them outside the patterns they were finetuned for.
Not that RAG isn’t really important.
Back when it was just "add a note about the specific thing in the hidden prompt", it felt like an old-style chat bot was patched on top of the LLMs, where it had a series of specific responses to watch out for and fix. Now it feels more like there's a large database of this shit and we just hope that the LLM will look up the correct stuff when it needs it (and probably prompt it to do that when demoing it for others and creating impressive examples that it probably won't live up to very well in the real world unless each user knows how to direct it with follow up prompts when it inevitably goes in the wrong direction).