this post was submitted on 17 Aug 2026
163 points (87.9% liked)
Technology
87389 readers
3359 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Never seen a non-deterministic compiler though
C/C++ compilers are non deterministic due to support of super macros that change run to run, non-deterministic optimisation strategies or ordering due to parallelism, and linkers often produce different outputs every time they are run where subtle bugs can cause crashes when addresses don't line up how you expect. And that's without mentioning projects that use a configuration step.
Can't most serious compilers produce reproducible builds these days given the same build environment. I know there has been a drive towards reproducible builds in general for security verification purposes.
Yeah, but it does take a lot of work to coerce them to do it.
GCC is honestly obnoxious, and you have to do a bunch of unintuitive things to get it. The compile stage needs the built-in RNG seeded, parent file paths stripped, you need to ensure that the date/time macros are not used anywhere, and you need to ensure that all command line flags are passed in the exact same order every time.
I just went through this with GCC16 on a new project.
Not by default usually but yes, you need to do a lot of work to set all necessary configurations and sometimes provide your own RNG seed for things which insist on random looking values.
Not only can they but certified toolchains exist in the safety critical space. Medical, aerospace, nuclear projects, and the like are often required to use and procure such toolchains as part of their validation.
I’m not very good with C/C++ so please correct me, isn’t that what’s called a “race condition”? Parallelism can cause non-determinism but not in the same sense LLMs generate non-deterministic output. Compilers are not statistical machines.
You don't need parallelism to have a race condition, just not handling an event with expected timing can cause one - like when two keys are pressed within one polling cycle and you depend on one being pressed before the other for some logic like up and right arrow for a diagonal but they register as right and up so the diagonal movement doesn't trigger
Compiler optimisation strategies sometimes use statistical machines and link time optimisation does use random number generators for producing output
You can have single threaded race conditions appear simply from inputs appearing in an unintended ordering. Or on a single core CPU, you can have one task meant to be done first take unusually long time, so the CPU gives time to another thread which finishes first but expected to finish last.
From my understanding, which is very limited, race conditions are more an issue with concurrent programming. Parallel computing uses separate processor cores for each task so there's less reliance on stack machines. But I guess each core still shares the memory, so maybe it still happens. Like I said, my understanding is limited. Just use rust.
Or one that pretends compilation was a success when it wasn't.
An LLM is not inherently non-deterministic though - if you don't randomly sample and instead have a fixed rule (which is what the recent fingerprint embedding approach does), if applied in all cases the output is deterministic, as the neural net at its core is deterministic function. A lot of the randomness beyond that is due to optimizations [source].
LLMs are however unreliable at 'compiling'. Whether or not it will be able to complete the requested task (translate human language into code) correctly it not guaranteed - at least nowhere near the compilers we use.
While they can be deterministic in runtime they can be compared to cryptographic hashes in that they come preloaded with pseudorandomness which will behave unpredictably
Technically, LLMs (and most ML models) are deterministic with the same input and same seed.
I get what you mean though.
So does ChatGPT intentionally change seed at every interaction so it always spits two different outputs given the same input?
In short: yes. You can tune these values when self hosting - it basically changes the b chance which tokens will be used under which circumstances.
Like Scipitie said, sort of.
In the ChatGPT app, there's tons of shuffling in the background, like context being injected, maybe sampling changed, agenic action, quantization... its "far" from the actual LLM, opaque by design, and as a result certainly not deterministic.
You can sometimes get deterministic output with the OpenAI API, but it's also dependent on nothing changing on their end. And their end changes a lot.
But self hosting or using a more consistent provider will give you deterministic output.
There is a parameter in llms called temperature. If you reduce it down to zero it will become deterministic. And probably even worse.
If you do that it will basically produce the most average possible output, given the context. I don't know if that's going to be useful in a compiler context..
Yeah but the output would be crap. Just use the same prng seed and you'll get reproducible output.
Exactly this. If you made an LLM that had deterministic code output then I’m all up for saying “AI is a compiler for human language”. But until then AI is most definitely not a compiler.
Setting the temperature parameter to 0.00 makes an LLM deterministic.
No. Floating point arithmetic and ordering of operations won’t make 0.0 deterministic.
Um... yes. Matrix multiplication is deterministic, there's nothing non-deterministic about an LLM, unless you artificially add randomness to them, like randomly selecting 1 of the top N ranked tokens.
The function is deterministic, agreed. The implementation is almost always not. Hardware floating point addition is not associative. So a GPU kernel that splits a reduction differently (ie interleaving it with anything else, like running your graphics, or sharing your work with other users on the same hardware) will produce different results over different runs even at temperature 0.0.
Determinism is almost always impossible when dealing with floating point on a multi-process/multi-user system.
There are attempts to create batch invariant language models (https://github.com/thinking-machines-lab/batch_invariant_ops) but all the major ones are not.
Yeah, that sounds like a disaster waiting to happen...