this post was submitted on 06 May 2025
85 points (100.0% liked)

technology

24027 readers
150 users here now

On the road to fully automated luxury gay space communism.

Spreading Linux propaganda since 2020

Rules:

founded 5 years ago
MODERATORS
all 42 comments
sorted by: hot top controversial new old
[–] KoboldKomrade@hexbear.net 71 points 4 months ago (1 children)

Who knew the self reinforcing markov chain would do this?

Literally anyone with a brain in compsci 5 years ago? Nah that's "hallucinations".

[–] Stolen_Stolen_Valor@hexbear.net 26 points 4 months ago (4 children)

I don’t know what a Markov chain is nor do I know dick about computer science so correct me if I’m wrong, but I’m assuming the ai is picking up its own shitty data and regurgitating it but worse? This just over and over again?

The more I think about this the ai is just going to destroy the internet completely huh?

[–] KoboldKomrade@hexbear.net 12 points 4 months ago (1 children)

Basically, you're correct. All the rest below is just me blabbin'.

I read a bit about markov chains a few times, and we talked about them in a few of my comp sci classes, but I'm not an expert or anything. But the basic explanation of them is that its a function that returns the probability which state should occur next. A fancy way of saying "What the odds of X, Y, or Z happening".

The classic example is you feed the chain something like a book, or all the books you have. Then you can ask what's likely to come after a word, and it will either spit out a single word or it gives you a list of the likelihood of all words it "knows". Texting apps, etc, word suggestion algorithms most likely used them.

You can train them on some sample data, then "customize" it so it "learns" from what the end user types. So my phone likes to say one person's name after I type "hi" because I've typed "hi x" a lot. You can also have them "look back" more words, classic chains only look at the most current input. They're much more complex and the time-space-energy required to compute them, so in general look back has only extended a few dozen words (in general).

All of that might sound familiar if you've heard much about how LLMs work. Because (in my opinion) its the same. I haven't seen a compelling argument that LLMs/Machine Learning aren't reducible to Markov Chains yet. (In the same way all computers are reducible to Turing Machines. Both machines suffer the same limits, one just hides some of the math so a normal human can use them.)

That isn't to say they're more powerful (LLMs can look back a few hundred words fairly ok), but they suffer the same limits Markovs inherently do. IE: The output is only as good as the inputs, deciding the output is subjective (because you can choose either the most common, or pick randomly, or ...), and they fundamentally don't "know" anything beyond their state machine. They can't verify. They can't research. As of right now, both systems can't even look at their original inputs. (And if they included those inputs, that'd take on a few terabytes of data.) All those books, text messages, reddit posts, all reduced to relations of words represented by probabilities.

I swear feeding a Markov's output to itself was discussed in one of my classes, and the professor was like, yeah it improves it a bit, but if you do it more then X amount then the whole thing falls apart. This was before 2020, in undergrad classes. People 100% said the same about LLMs before they started posting online, and now the silicon valley dweebs are hitting the same problem. I swear tech bros, besides being the most hiterite prone, love to recreate disasters. Same happened with "crypto" shit and basic financial schemes.

TLDR: Fuck "ai". My AI class sucked because it was in Java, but at least we were making a silly game AI (specific game withheld for privacy, but it was 20+ years old). If they're not talking about min-maxing and search trees and master agents, and instead are pushing this crap, comp sci is overcooked.

[–] Sabir_Hajjar@hexbear.net 7 points 4 months ago

It is not true that a pretrained transformer is reducible to a Markov chain in the same way that a computation is reducible to a turing machine. While any computation could be achieved by a turing machine (given arbitary time), not every pretrained transformer could be described by a (sufficiently complicated) Markov chain of arbitrary length.

One reason is the attention mechanism, which allows a transformer to weight some tokens in a sequence as differently than others.

But the biggest difference is that a Markov chain can only consider an entire state at once, and produce the next state sequentially, while transformers can consider many parts of a sequence as individual states. Transformers are also capable of autoregression (they're basically just RNNs+attention heads), while Markov chains are not - new states transform old states, and do not simply append.

Now, if you take the attention mechanism out of the transformer, you're basically just left with an RNN, and if you then take the autoregression out of the RNN, you've got a Markov chain in the limit, probably. So you could argue a Markov chain is in the limit of an LLM (kind of obvious, since you should expect an LLM trained on a Markov chain to predict it totally), but you could never argue an LLM can be produced in the limit of a Markov chain (you can train any arbitrarily large Markov chain and it will never get anywhere close to an LLM).

[–] Lamprey@hexbear.net 11 points 4 months ago* (last edited 4 months ago)

It happened way fucking quicker than I expected

[–] CriticalResist8@lemmygrad.ml 10 points 4 months ago

a Markov chain predicts the next state based on the current state. If today is sunny, how likely is it that tomorrow will be rainy? Mathematically, this can be reduced to a Markov chain (so we don't have to take into account the season, weather patterns or anything like that for this example).

But a Markov chain isn't just saying how likely it is to be rainy on a given day, but how likely it is to be rainy tomorrow based on today. If today is sunny, there's a let's say 70% chance that tomorrow will be rainy. If today is rainy, there's a 40% chance that tomorrow will be rainy (and conversely a 60% chance that tomorrow will be sunny because possible states must always equal 100%).

Autocorrect works similarly. It predicts the next word based on the current word you've typed out. LLMs are kinda glorified markov chains because they also predict words (called tokens, which are about 3 to 4 characters) but they do it over a much larger "current state", which is the chat history, custom instructions if you gave any on chatgpt, etc. The context that is passed on with your prompt consists of several tokens and the AI generates one token at a time until little by little it's formed a full response that it outputs.

In this way the markov chain of LLM is if I give it the sentence "Hello! My name is" for example, it will predict which token is the most likely to follow and should output it. We can assume this should be a name but truthfully we don't know the exact probabilities of the next state. If I give it "Hello, my name is" - changing just one character might also change the prediction weighting. I say "might" because AI is a black box and we don't really see what happens when the data passes through the neurons.

However if you send that sentence to chatGPT it will correctly tell you that your message got cut off and asks you to finish it. They do some post-production fine-tuning to get it to do that. Compare to deepseek without the reasoning model:

[–] Philosoraptor@hexbear.net 46 points 4 months ago (4 children)

According to the article, it's a bigger problem for the "reasoning models" than for the older-style LLMs. Since those explicitly break problems down into multiple smaller steps, I wonder if that's creating higher hallucination rates because each step introduces the potential for errors/fabrications. Even a very small amount of "cognitive drift" might have a very large impact on the final answer if it compounds across multiple steps.

[–] simontherockjohnson@lemmy.ml 20 points 4 months ago* (last edited 4 months ago) (3 children)

AI alchemists discovered that the statistics machine will be in a better ball park if you give it multiple examples and clarifications as part of your asks. This is called Chain of Thought prompting. Example:

Then the AI Alchemists said, hey we can automate this by having the model eat more of it's own shit. So a reasoning model will ask it self "What does the user want when they say < Your prompt>?" This will generate text that it adds to your query, to generate the final answer. All models with "chat memory" effectively eat their own shit, the tech works by reprocessing the whole chat history (sometimes there's a cache) every time you reply. Reasoning models because of the emulation of chain of thought eat more of their own shit than non-reasoning models do.

Some reasoning models are worse than others because some refeed the entire history of the reasoning, and others only refeed the current prompt's reasoning.

Essentially it's a form of compound error.

[–] Philosoraptor@hexbear.net 2 points 4 months ago (1 children)

Interesting. If that's right, it makes a lot of sense that models with this kind of recursive style would generate errors at a much higher rate. If you're taking everything in the session so far as an input and there's some chance for every input that the model produces an error, the errors will rapidly stack up with this kind of functionality. I've seen those time-lapses of how far generative AI can drift over 100 (or whatever) iterations of "reproduce this photo without making any changes" type prompts, with the output of each generation fed back in as input. This strikes me as the same kind of problem, but with text.

[–] simontherockjohnson@lemmy.ml 4 points 4 months ago* (last edited 4 months ago)

It happens faster to images because of the way LLMs work. LLM's work on "tokens", a token for text is typically a character, fragment of a word, a word, a fragment of a sentence. With language it's much easier to encode meaning and be more precise because that's what language already does. The reason NLP is/was difficult is because language is not algorithmically consistent, it evolves, and rules are constantly broken. For example Kai Cenat is credited with more contributions to the English language than the vast majority of people because children decided to talk like him. Point being is that language does the heavy lifting in terms of encoding a string of characters into something meaningful.

With images, it's a whole different ball game. Image tokenizers often work in several different ways, there are two types of token hard and soft. Hard tokens for example could be the regions of the image, part could be the colors, the alpha channels.

Hard tokens are also the visual encoders of meaning so a chair, table, or car could be a hard tokens based on their bounding boxes. These tokenization techniques are based in a lot of other types of machine learning.

Note that these tokens often overlap in practice and consume regions of other tokens, however as "hard" tokens they are considered distinct entities, and this is where the trouble starts esp. for image generation (that's roughly why a lot of AI did and still does things like draw extra fingers).

The next type of tokens are soft tokens, and they're a bit harder to explain, but basically the idea is that soft tokens are encoded by detecting continual statistical distributions within images. It's a very abstract way of reading an image. Here's where the trouble compounds.

So now when we're writing an image, what do we write the image with? You guessed it. Tokens. The reason that those AI drift time lapses exist is because LLMs are statistical and not "functional". They don't have the mathematical concept of "identity". Otherwise they'd try to recreate the same image by copying the data in the exact tokens (or just copy the image itself) , instead they try to regenerate the image by generating new tokens with the same attributes that it read from the image.

To illustrate this lets say an image contains a blue car and the AI can only tokenize it as blue car. Asking an LLM to run an identity function on that image will result in a different car because the resolution of the token is only like 2 dimensions "blue" and "car" which roughly means it will output the average "blue" "car" from its training data. Now with human made things it's actually a lot easier. There's a finite variation of cars. However there's an infinite variation of things that can happen to a car. So an AI theoretically can run an identity function off of a particular make/model/year of a vehicle but if the paint is scratched or the paint job is unique it will start to introduce drift there's also other sources of drift like camera angle etc. With natural objects this becomes a whole different ball game because of the level of variation, this complexity compounds with scenes.

So identity functions on text are extremely easy in comparison for example:

This works because the tokens are simpler and there is less of a loss of "resolution" from the text to the tokenize form. E.g. word "Poopy" is token "poopy". But once you get into interpreting an image, and re-encoding those interpretations onto a new canvas it becomes much more difficult. e.g. image of "Dwayne the Rock Johnson" is most likely a series of tokens like "buff man", "bad actor", etc.

This is a rough explanation because there's a lot of voodoo, and I'm more of a Software Engineer than I am a statistics/data guy so I approach the alchemy a little bit from an alchemical standpoint.

[–] yogthos@lemmygrad.ml 13 points 4 months ago

That's plausible. I suspect that the whole reinforcement learning step the models use only gets you so far. I think that neurosymbolic approach is actually more promising. The idea there is pretty clever. You use a deep neural network to parse noisy data from the outside world, and classify it. Then you use a symbolic logic engine to operate on classified data, and now you have actual reasoning happening within the system.

[–] Speaker@hexbear.net 5 points 4 months ago

The few times I've had a local one share the "reasoning", the machine mainly just ties itself in knots over trivial nonsense for thousands of words before ignoring all of that and going with the first answer it came up with. Machine God is a Redditor.

[–] muntedcrocodile@lemm.ee 41 points 4 months ago (2 children)

Its training on ai generated content

[–] queermunist@lemmy.ml 30 points 4 months ago (2 children)

You'd expect them to control for that.

But who the hell knows with these people.

[–] jack@hexbear.net 45 points 4 months ago (1 children)

How could they? The Internet is flooded with that garbage, and not just from their own model.

[–] RedWizard@hexbear.net 20 points 4 months ago (1 children)

Yeah, they would have to have a sure fire way to identify the content as 100% AI generated so they could ditch it. If they're training on Reddit Comments then they're fucked.

[–] TankieTanuki@hexbear.net 18 points 4 months ago

To be fair, they do thanks the gold for the stranger fuck CCP bacon.

[–] kristina@hexbear.net 9 points 4 months ago* (last edited 4 months ago) (2 children)

ChatGPT doesn't have access to verified not-bots. Google and Facebook does (they can read all of your messages). Expect them to become the privacy-invasive leads on this.

[–] ProletarianDictator@hexbear.net 4 points 4 months ago

Exactly it.

Meta will be able to have bots dominate their platforms while being able to distinguish and train on the real human users.

Human interaction with bots will also allow labelling some bot generated data as acceptible for reuse as training data when real human data isn't enough.

Social media platforms will have a massive advantage for LLMs in the long run. Glorified search engine platforms such as ChatGPT are a relic of this current era.

[–] queermunist@lemmy.ml 3 points 4 months ago

They could use human filters to discard obvious bots... but that would make this shit even more expensive and unprofitable so they have to cut corners and make their own project fail.

[–] Monk3brain3@hexbear.net 40 points 4 months ago* (last edited 4 months ago) (2 children)

I need a burning "AI" emoji. Quotation marks because it's not AI and AI isn't real.

AI and crypto have turned me into a goddam luddite.

[–] insurgentrat@hexbear.net 30 points 4 months ago

It's less that AI isn't real and more that it's a nebulous marketing term.

A videogame enemy is AI, a spam classifier is AI, a computer vision motion alarm is AI, an extremely convincing text emitter is AI.

I honestly can't understand why people use these things outside of applications for transforming text you created where you are checking the output. Their whole thing is learning how to output text which is an extremely convincing facsimile of human writing. Like their whole thing, literally just the thing they do and optimise for, is forging being a thinking human being. That's quite useful for say summarising a body of text you wrote, or helping you soften the tone of something, or draft rhymes or something. If you use them to learn anything though, or produce something you don't entirely understand inside that output will be things with only a limited relationship with reality and every single piece of it will look extremely convincing.

Like holy shit, even putting aside all other concerns you are exposing yourself to a specialised misleading machine. Why not just take a hammer to your own head while you're at it?

[–] buckykat@hexbear.net 34 points 4 months ago (1 children)

Because the Internet is quickly becoming an ouroboros of LLM shit

[–] 30_to_50_Feral_PAWGs@hexbear.net 4 points 4 months ago

Dead internet theory, now with necromancy!

[–] ZeroHora@lemmy.ml 30 points 4 months ago

Jesus Christ, 50% or more is hallucination.

[–] polpotkin@hexbear.net 26 points 4 months ago

The tech hasn't gotten much better since gpt 3.5 and everything we see is the result of fine tuning, which uses human input to bias the model (openai is 80% a proxy of human labor).

[–] footfaults@lemmygrad.ml 24 points 4 months ago

Habsburg AI confirmed

[–] marxisthayaca@hexbear.net 14 points 4 months ago

Synthetic data probably

[–] tamagotchicowboy@hexbear.net 1 points 4 months ago

Easy, because you can't feed a system shit it generated in a virtual ouroboros meets AI centipede. Dataraters are encouraged to do research, how you do that? Search engines, what's those search engines use for the top of its search results? AI generated poopoopeepee. The circle of filth continues. LM generates shit notices patterns, if all it notices is its own patterns with diminishing human input don't whine when its highly refined waste.