GamingChairModel

joined 3 years ago
[–] GamingChairModel@lemmy.world 2 points 14 hours ago

up to 150kW at full load.

That's the last generation. They're moving from Blackwell to Rubin chips now, and the 72-GPU Rubin servers use up to 230 kW.

The typical residential connection in the U.S. has a 24 to 48 kW electrical connection. A block of houses might not have enough power infrastructure to power just one of these server racks.

[–] GamingChairModel@lemmy.world 1 points 23 hours ago (2 children)

It's that they are trying to use statistics to encode entire thought processes into hidden variables from conversation snippets. They want to use statistics to go from many individual interactions to a large model, and then use that model to predict individual interactions again.

Has it been shown that the human brain doesn't model the world in a similar way, though? A huge portion of human knowledge is both stored and transmitted in the form of language. Lots of human knowledge also follows the garbage in, garbage out theory, where you can have entire areas of knowledge that aren't actually true but might be internally consistent, at least within certain scopes: conspiracy theories, belief in the supernatural, entire academic disciplines built on a religion or theology that not everyone believes, etc. Or even world building in fiction, the words on a page can be enough to convey ideas such that it "tricks" human brains into filling in the gaps so that they internally see a rich, fleshed out world that is entirely fictional and where specific details might not find strong direct support in the underlying text.

it has no concept of correctness

But statistical weight on what is more or less likely to be correct still makes a difference to objective quality of the outputs. If the model weights are trained on the reality that high quality university texts describe something and reflect some sort of underlying model of what is described using language, then can't the model itself learn as much as a human could from those words on a page?

All models are wrong, but some can be useful. And different models have different quality in different domains. So although I don't believe LLMs will overtake the hump of getting ahead of human knowledge, I also don't believe that any given LLM can be evaluated on quality, and that Facebook's LLMs are significantly behind other LLMs we see.

And that maybe a huge part of it is its internal process of preparing the model to evaluate the quality of its inputs, such that the output it produces can also score high on quality.

Yeah, but if they spread the cost across many customers, the cost per customer is going to be much smaller, even if it doesn't last as long before needing a replacement.

If it costs $100,000 to build a fiber line to a single home for 30 years (360 months) that house will need to pay $278/month for 30 years to break even. Throw in interest rates/inflation, and it'll be more.

But if a satellite that costs $1.5 million to build and launch into orbit can serve even 200 customers for 5 years, that's only $125/month per customer.

As it stands right now, Starlink serves something like 12 million customers on 10,000 satellites. So that's an average of 1200 customers served by each satellite, which is what makes $50/month service feasible as a business.

[–] GamingChairModel@lemmy.world 2 points 2 days ago (2 children)

That just means the high up front costs of either trenching fiber or launching satellites need to serve a lot of people to recover that cost. That means the last mile for rural residents tends not to be cost effective for fiber, because there aren't enough connections served by any given segment.

But making it so any given satellite can serve lots of people in its footprint at any given moment might make it cost effective to serve rural residents.

One common strategy is to run fiber to a specific central location and run point to point microwave antennas to the individual houses/buildings served. That way the fiber itself can carry the traffic of hundreds of users, and each house just needs to have an antenna with line of sight to the place where the fiber is terminated. Rural WISPs have been doing this from before Starlink.

[–] GamingChairModel@lemmy.world 14 points 2 days ago (1 children)

Publication date April 2, based on reporting from Bloomberg on April 1. Anyone have an update 3 months later? I want to see what happened to the remaining half.

Intellectual property is as real as any other property is: it's all made up by people, but it has the force of social/economic/governmental institutions behind it so it can affect your life.

It's just that legal concepts tend to attract discussions by people who can't tell the difference between a descriptive or positive argument (how things are) versus a normative argument (how things should be).

CNBC ran this article earlier this week, too:

Employers who laid off workers citing AI are already starting to regret it

If CNBC of all places is running stories like this, that represents a huge vibe shift.

 

(Note: McSweeney's is a satirical publication)

It's like responding to your employee losing an arm, ripped off by your tiger, and saying "I'm never going to financially recover from this."

[–] GamingChairModel@lemmy.world 1 points 1 week ago (1 children)

Basically they'd need about as much in radiator fin surface area as they would have in solar panel area. The ISS has 8 solar array wings, 35m x 12m, that can produce about 30 kW each, or 240 kW total, in sunlight (which is only half the time). The ISS has a complex cooling system, but relies on 4 radiators about 3.1 m x 13.6 m to reject up to 14 kW of heat each (56 kW total) for cooling the solar arrays themselves. The main cooling system uses 6 radiators, each 23.3 m x 3.4 m, to reject 70 kW of heat (from this report it sounds like each radiator may be capable of rejecting more than 1/6 of the heat but that the system as a whole needs to be kept under 70 kW of heat rejection).

So that seems like about 650 square meters of radiators can provide about 120 kW of heat rejection.

Today, a 72-GPU Blackwell server is 130 kW in a single server rack. The next generation rolling out now has 72 Rubin GPUs in a 230 kW server, in a single rack. And that's not even a "data center." That's just a single (albeit very powerful) server. How many can you string together, with networking equipment beaming data connections back down to the ground, before the ratio of solar panels and radiators to the actual ship size becomes unworkable?

That said, it's technically possible, especially if you can radiate the heat at higher temperatures than the ISS does, as the Stefan-Boltzmann law shows that the hotter the radiator, the more heat it can reject. Just completely infeasible from an engineering and economical standpoint, for any data center that hopes to be relevant in an age of 100+ MW data centers.

[–] GamingChairModel@lemmy.world 7 points 1 week ago (2 children)

"I read your message but have nothing more to add"

ACK

If you're ok zooming out far enough you can serve a static image of the pale blue dot and a red arrow pointing to it.

 

I've read some of Ed Zitron's long posts on why the AI industry is a bubble that will never be profitable (and will bring down a lot of companies and investors), and one of the recurring themes is that the AI companies are trying to capture growing market share in an industry where their marginal profits are still negative, and that any increase in revenue necessarily increases their costs of providing their services.

But some of the comments in various HackerNews threads are dismissive, saying that each new generation of models makes the cost of inference lower, so that with sufficient customer volume, the companies running the models can make enough profit on inference to make up for the staggering up-front capital expenditures it took to build out the data centers, train their models, etc.

It's all pretty confusing to me. So for those of you who are familiar with the industry, I have several questions:

  1. Is the cost of running any given pretrained model going down, for specific models? Are there hardware and software improvements that make it cheaper to run those models, despite the model itself not changing?
  2. Is the cost of performing a particular task at a particular quality level going down, through releases of newer models of similar performance (i.e., a smaller model of the current generation performing similarly to a bigger model of the previous generation, such that the cost is now cheaper)?
  3. Is the cost of running the largest flagship frontier models going down for any given task? Or does running the cutting edge show-off tasks keep increasing in cost, but where the companies argue that the improvement in performance is worth the cost increase?

I suspect that the reason why the discussion around this is so muddled online is because the answers are different depending on which of the 3 questions is meant by "is running an AI model getting cheaper over time?" And the data isn't easy to synthesize because each model has different token prices and different number of tokens per query.

But I wanted to hear from people who are knowledgeable about these topics.

 

Curious what everyone else is doing with all the files that are generated by photography as a hobby/interest/profession. What's your working setup, how do you share with others, and how are you backing things up?

view more: next ›