this post was submitted on 28 Feb 2025
19 points (88.0% liked)

Open Source

35384 readers
149 users here now

All about open source! Feel free to ask questions, and share news, and interesting stuff!

Useful Links

Rules

Related Communities

Community icon from opensource.org, but we are not affiliated with them.

founded 5 years ago
MODERATORS
 

Explainer of Diffusion LLMs from Andrej Karpathy: "Most of the LLMs you've been seeing are ~clones as far as the core modeling approach goes. They're all trained "autoregressively", i.e. predicting tokens from left to right. Diffusion is different - it doesn't go left to right, but all at once. You start with noise and gradually denoise into a token stream."

top 1 comments
sorted by: hot top controversial new old
[–] mindbleach@sh.itjust.works 2 points 1 month ago

The premise is sort of hilarious. "Everybody's just blindly copying this one kind of network. We made the bold decision to copy the other one."