Nvidia sells plenty of GPUs for actual money, they are good for it.
No, the real issue is the depreciation for the people owning GPUs. Your GPU will be usable for 4-6 years, and 2-4 of those years will be spent as ”the cheap old GPU. After that time, you need new GPUs. (And as the models are larger by then, you need moahr GPU)
How the actual fuck do these people expect to get any ROI on that scale with those timeframes? With training, maybe the trained model can be an asset (lol), but for inference there are basically no residual benefits.
Calculate the total power cost of running it at 100% load since 2014
Calculate Flops/Watt and compare with modern hardware
Calculate MTTF when running at 100% load. Remember that commercial support agreements are 4-5 years for a GPU, and if it dies after that, it stays dead.
In AI, consider the full failure domain (1 broken GPU = 7+ GPUs out of commission) for the above calculation.
You’ll probably end up with 4-6 years as the usable lifetime of your billion dollar investment. This entire industry is insane. (GTX 1080 here. Was considering an upgrade until the RAM prices hit.)
Nvidia sells plenty of GPUs for actual money, they are good for it.
No, the real issue is the depreciation for the people owning GPUs. Your GPU will be usable for 4-6 years, and 2-4 of those years will be spent as ”the cheap old GPU. After that time, you need new GPUs. (And as the models are larger by then, you need moahr GPU)
How the actual fuck do these people expect to get any ROI on that scale with those timeframes? With training, maybe the trained model can be an asset (lol), but for inference there are basically no residual benefits.
I'm still rocking a GTX970 from 2014
Do this:
You’ll probably end up with 4-6 years as the usable lifetime of your billion dollar investment. This entire industry is insane. (GTX 1080 here. Was considering an upgrade until the RAM prices hit.)