Technology

I'd rather go back to raspberry pi levels of compute with BBS style text only sites.

But... I've already got SoftRAM 95

It's that they are trying to use statistics to encode entire thought processes into hidden variables from conversation snippets. They want to use statistics to go from many individual interactions to a large model, and then use that model to predict individual interactions again.

Has it been shown that the human brain doesn't model the world in a similar way, though? A huge portion of human knowledge is both stored and transmitted in the form of language. Lots of human knowledge also follows the garbage in, garbage out theory, where you can have entire areas of knowledge that aren't actually true but might be internally consistent, at least within certain scopes: conspiracy theories, belief in the supernatural, entire academic disciplines built on a religion or theology that not everyone believes, etc. Or even world building in fiction, the words on a page can be enough to convey ideas such that it "tricks" human brains into filling in the gaps so that they internally see a rich, fleshed out world that is entirely fictional and where specific details might not find strong direct support in the underlying text.

it has no concept of correctness

But statistical weight on what is more or less likely to be correct still makes a difference to objective quality of the outputs. If the model weights are trained on the reality that high quality university texts describe something and reflect some sort of underlying model of what is described using language, then can't the model itself learn as much as a human could from those words on a page?

All models are wrong, but some can be useful. And different models have different quality in different domains. So although I don't believe LLMs will overtake the hump of getting ahead of human knowledge, I also don't believe that any given LLM can be evaluated on quality, and that Facebook's LLMs are significantly behind other LLMs we see.

And that maybe a huge part of it is its internal process of preparing the model to evaluate the quality of its inputs, such that the output it produces can also score high on quality.

I listened to a podcast with a couple smart mathematicians talking about AI recently and this rings true based off what I heard them discuss.

They hypothesized that only verifiable domains can really see advances due to AI. So mathematics, physics, a load of the other sciences, and medical research. Even programming, as long as you have a pre-designed solution.

But for problems where you can’t look at a solution and say “yeah, that’s an optimal solution or close to it”, ie basically any business problem; they are much less useful, a big reason being what you mentioned in your comment.

You are oversimplifying it so much it’s inaccurate.

You’re judging a future tech of a tiny tiny tiny piece.

And also that they’re a shitty data annotater.

You're right, but let's not forget that the facebook phone is really popular and the meta glasses definitely don't look like nerd emoji. 🤓

Time to roll some AR augmented dice, so those perv glasses of his have a use.

Like actually? Early image models were given open ended prompts so they would just generate images based on the frequency of topics and traffic on the internet. Turns out if aliens saw our culture they would assume that some of our most important things are Jesus and crustaceans. Zuck's dementia-ward demographic were mostly responding to any picture with a Jesus in it, though.

But that was before generative ai found out that humans go absolutely ape-shit for neonatal face features