Technology
This is the official technology community of Lemmy.ml for all news related to creation and use of technology, and to facilitate civil, meaningful discussion around it.
Ask in DM before posting product reviews or ads. All such posts otherwise are subject to removal.
Rules:
1: All Lemmy rules apply
2: Do not post low effort posts
3: NEVER post naziped*gore stuff
4: Always post article URLs or their archived version URLs as sources, NOT screenshots. Help the blind users.
5: personal rants of Big Tech CEOs like Elon Musk are unwelcome (does not include posts about their companies affecting wide range of people)
6: no advertisement posts unless verified as legitimate and non-exploitative/non-consumerist
7: crypto related posts, unless essential, are disallowed
view the rest of the comments
There are ways to make it cheaper. Starting with maybe not encouraging token-maxing.
Generally, unless you're either a FOSS project or generating images/video, you have to be doing something very wrong to spend more on AI than on salaries.
Not really. LLMs are still completely unable to manage even medium scale architectures. At a corporate scale they’re literally just spending on trying to have the most context they can in the LLM. There’s no getting around it.
Yeah, the smarter way to use LLM-based agents is carefully defined tasks. Mozilla describes their vulnerability assessment processes in this blog post.
Mozilla describes the process they've used: building a harness that instructs a model to find a specific category of vulnerability on a specific interface, and then write up its findings. It's a narrow enough context that the model gets specific instructions, and a simple definition of success, and it sets up many such tasks that can be fed into the existing process for verifying and triaging bugs. Note that the output for this LLM pipeline basically feeds into the same interface for accepting bug reports from the public, or from their human contributors within the project.
There's a couple of takeaways here, too:
There are ways to use these tools, but none of it really seems like a truly revolutionary/disruptive change to how large projects are managed.
Generating images is not that expensive, it's surprisingly inexpensive.
Yeah, it's counterintuitive because it's a lot more work for a human to draw a picture (much less a photorealistic picture) than to write a few words, but human language grammar actually has a lot of strict rules that makes that stream of letters work as "valid" output, much less "decent" output that kinda matches the prompt/description. Transpose a pair of letters or even substitute a single letter (or token) and you've got an output that just doesn't work, in a way that generated images don't have to worry about.