People Twitter

10028 readers

1545 users here now

People tweeting stuff. We allow tweets from anyone.

RULES:

Mark NSFW content.
No doxxing people.
Must be a pic of the tweet or similar. No direct links to the tweet.
No bullying or international politcs
Be excellent to each other.
Provide an archived link to the tweet (or similar) being shown if it's a major figure or a politician. Archive.is the best way.

founded 2 years ago

MODERATORS

SendMeYourTaTas@sh.itjust.works

pelespirit@sh.itjust.works

636

Managers (media.piefed.zip)

submitted 16 hours ago* (last edited 16 hours ago) by inari@piefed.zip to c/whitepeopletwitter@sh.itjust.works

139 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] GamingChairModel@lemmy.world 1 points 7 hours ago (1 children)

AI companies are bankrupting themselves with training costs that they need to recoup back by selling inference.

I think they hit a wall in actual returns on performance with pretraining, years ago. Then they started scaling up on post-training/reinforcement learning to continue improvement, but that might be hitting a plateau as well. More recently it looks like they're relying more heavily on scaling up on inference, which is a significant problem for their long term business models.

If they're not able to cheaply deliver inference (and charge at a premium), how will they be able to sustain their businesses?

It seems that the most recent, largest models are using a lot more tokens to accomplish the same tasks, so even as token cost drops the actual cost of using the latest models seems to be going up with time (even as performance improves).

[–] theunknownmuncher@lemmy.world 1 points 6 hours ago* (last edited 6 hours ago)

If they’re not able to cheaply deliver inference (and charge at a premium), how will they be able to sustain their businesses?

I definitely agree that they have a big problem on their hands, and are in deep deep trouble. They are in a position where they must sell a service that is very cheap in order to pay for up front costs that were very expensive.

This is also why the release of Deepseek was such a devastating blow to US AI companies. It proved that:

they don't really have a moat that would lock users into their service, or secret special knowledge that prevents other companies from training competitive models. They're in a race to the bottom
Deepseek was not only able to train a model of the same caliber, but they were able to do it at a tiny fraction of the cost that US AI companies spent on training US models. Because they spent so much less on training, it means that Deepseek is able to undercut the US companies and offer inference at a much lower price