News Tiny Corp decides to make both AMD and Nvidia versions of its TinyBox AI server - GeForce version is 67% more expensive

[Admin]

Tiny Corp appears to have decided whether or not to follow its original plan of using AMD GPUs, or switch to Nvidia GPUs to power its Tiny Box server. The muddled answer is that it will make both.

Tiny Corp decides to make both AMD and Nvidia versions of its TinyBox AI server - GeForce version is 67% more expensive : Read more

 

[artk2219]

Looks like someone didn't complete a feasibility test before shoving their foot in their mouth.

 

[PEnns]

"...and at the time of our initial report had received 583 preorder"

As if we need further proof that "pre-ordering" is a very dumb idea.

People are worried this TinyBox thing will be sold out and they will suffer some kind of technological harm??

 

[Alvar "Miles" Udell]

If you like to tinker and feel pain, buy red. If you want it just works buy team green.

Basically describes my experience with Team Red (9600XT to Fury Nano) and Team Green (2070 Super). AMD needs to heavily invest in software QC so it "just works".

 

[peachpuff]

Looks like someone didn't complete a feasibility test before shoving their foot in their mouth.

Oops...

 

[Notton]

Hotz boxes that no one wants to touch, and you definitely don't want to be holding them when the music stops.

They are the Hotz Potato

 

[usertests]

Man freaks out publicly, gets more free press for his tiny company.

 

[digitalgriffin]

Even if they ironed out all the flaws, the man is obviously a disaster to work with. Won't touch his products with a 10 foot pole. A fix for one means a fix for all. And building a competing box is just as easy. And likely with less headaches.

 

[artk2219]

Basically describes my experience with Team Red (9600XT to Fury Nano) and Team Green (2070 Super). AMD needs to heavily invest in software QC so it "just works".

The man is trying to use the cards in a way that they weren't designed and tested for, and is surprised and angry that his consumer cards aren't running his professional workload as well as he would like, and he isn't getting bespoke professional level support on consumer level equipment. Almost like thats why professional products with better support exist, or heck, why AMD's entire semi-custom group exists.

 

[ivan_vy]

The man is trying to use the cards in a way that they weren't designed and tested for, and is surprised and angry that his consumer cards aren't running his professional workload as well as he would like, and he isn't getting bespoke professional level support on consumer level equipment. Almost like thats why professional products with better support exist, or heck, why AMD's entire semi-custom group exists.

you are on point. Mr. Hotz wanted tu put a race car motor in a tractor to plow the field, at the end of the day is just about horsepower, right? right? /s
and asked AMD to compete with themselves, MI-300 exists for a reason.

 

[bit_user]

If I were a Nvidia reseller, I'd be screaming for Nvidia to enforce their EULA and send Tiny a notice that they will not be receiving any shipments of RTX 4090 cards.

Nvidia updates GeForce EULA to prohibit data center use

Instead, it wants you to buy the significantly more expensive Tesla

www.datacenterdynamics.com

He should have to use workstation or datacenter cards, as far as I understand. That would probably push the price of his Nvidia boxes to well above twice that of his AMD model.

 

[edzieba]

If I were a Nvidia reseller, I'd be screaming for Nvidia to enforce their EULA and send Tiny a notice that they will not be receiving any shipments of RTX 4090 cards.

Nvidia updates GeForce EULA to prohibit data center use

Instead, it wants you to buy the significantly more expensive Tesla

www.datacenterdynamics.com

He should have to use workstation or datacenter cards, as far as I understand. That would probably push the price of his Nvidia boxes to well above twice that of his AMD model.

From that very same link:

A spokesperson added: “We recognize that researchers often adapt GeForce and Titan products for non-commercial uses or other research uses that do not operate at data center scale. Nvidia does not intend to prohibit such uses.”

The 'Tiny box' is a literal desktop cube. It doesn't even have rackmount ears!

 

[bit_user]

From that very same link:

The 'Tiny box' is a literal desktop cube. It doesn't even have rackmount ears!

That doesn't mean its customers are buying them for noncommercial use, and that's the bit of your quote that counts.

Lots of companies manage to do things like having academic discounts. Fine, let him have an exemption for such cases, but he should be able to account for every one of the units he sells to a non-commercial entity, if/when audited.

 

[Shockwave007]

Going to go out on a limb here and say that since they went cheap and decided not to use enterprise grade cards from AMD's professional line in the first place, is the reason they ran into these problems.

 

[edzieba]

That doesn't mean its customers are buying them for noncommercial use, and that's the bit of your quote that counts.

Go read the EULA and tell me where 'commercial use' is prohibited.
The prohibition in "2.1.3 Limitations." is on "Datacenter Deployment" only, not general commercial use.

 

[Pierce2623]

Basically describes my experience with Team Red (9600XT to Fury Nano) and Team Green (2070 Super). AMD needs to heavily invest in software QC so it "just works".

You’re also comparing 2014 drivers to current ones. Everyone kinda sucked until 2017. Pascal was the first GPU I ever had that just worked every time. Now since then I’ve owned a 3060, 3070, 6700xt, 7800xt, and a 6650m. All just work perfectly every time at stock settings and none ever crash without overclocking. That’s just modern hardware and modern drivers.

 

[bit_user]

Go read the EULA and tell me where 'commercial use' is prohibited.

Well, this clause:

"Customer may not ... provide commercial hosting services with the SOFTWARE."
​

Although, that's not saying you can't use it for commercial purposes, just that you can't charge others for time on the system.

The prohibition in "2.1.3 Limitations." is on "Datacenter Deployment" only,

Define a datacenter, though. This thing burns like 5 kW, so it won't work with normal household wiring. Furthermore, you're probably going to have to put it somewhere that's air conditioned, 24/7. That sounds like a machine room. I'll bet a good lawyer can make the case that a machine room fits the general definition of a datacenter.

BTW, gaming cards really aren't designed for 24/7 workloads. Just look at what mining does to them, and miners usually even go to the trouble of reducing the core clocks and undervolting them. So, good luck getting warranty claims fulfilled.

 

[edzieba]

Define a datacenter, though. This thing burns like 5 kW, so it won't work with normal household wiring.

That depends on your house. Anaemic US domestic 110V wiring maybe, elsewhere 5kW is not even outside the realm of a domestic supply. If you have a home machine shop, 5kW is not an unreasonable load to have in a garage.

Furthermore, you're probably going to have to put it somewhere that's air conditioned, 24/7. That sounds like a machine room.

Or just somewhere with an external vent, or a room you don't mind being a bit warm with the door closed. 5kW may be a lot for those used to desktop machines, but small in the realm of heating elements.

BTW, gaming cards really aren't designed for 24/7 workloads. Just look at what mining does to them, and miners usually even go to the trouble of reducing the core clocks and undervolting them. So, good luck getting warranty claims fulfilled.

"Mining used ooh scary scary" has not borne out in actual testing of hardware. Thermal cycling from bursty workloads is going to be much harder on the common failure point of consumer GPUs (the BGA solder balls) than constant high workloads that provide a stable temperature.
Datacentre SKUs have coolers designed to take advantage of server airflow paths (along the length of the card), but the actual cards themselves are not constructed any differently. If you peel the cooler off of a FE card, you'll even find the unpopulated pads for the end-mounted power connectors that would be present for the same PCBA used in a datacentre SKU, and often even NVLink card-edge fingers. It's easier to design one card and use it for both consumer and workstation and DC than it is to double or triple your dev budget for no good reason.


And on top of all that... the EULA binds the end user of the software (i.e. whoever has bought a TinyBox and is running the Nvidia drivers on it), not TinyCorp. This is why all the other companies building actual rackmount servers around consumer Nvidia GPUs - for example Comino's Grando boxes - continue to operate without issue.

 

[bit_user]

And on top of all that... the EULA binds the end user of the software

You don't think Nvidia has other agreements they make resellers and integrators sign? That seems unlikely.

Not only that, but if you want to bundle Nvidia's software with your machine, you need a redistribution license.

This is why all the other companies building actual rackmount servers around consumer Nvidia GPUs - for example Comino's Grando boxes - continue to operate without issue.

Sounds like playing with fire, to me. Wait 'till Nvidia sees its revenue decline for a couple quarters and then check if Nvidia still lets them get away with it.

 

[Pierce2623]

Well, this clause:

​

"Customer may not ... provide commercial hosting services with the SOFTWARE."​

​

Although, that's not saying you can't use it for commercial purposes, just that you can't charge others for time on the system.


Define a datacenter, though. This thing burns like 5 kW, so it won't work with normal household wiring. Furthermore, you're probably going to have to put it somewhere that's air conditioned, 24/7. That sounds like a machine room. I'll bet a good lawyer can make the case that a machine room fits the general definition of a datacenter.

BTW, gaming cards really aren't designed for 24/7 workloads. Just look at what mining does to them, and miners usually even go to the trouble of reducing the core clocks and undervolting them. So, good luck getting warranty claims fulfilled.

Six 7900xtx and a 64 core Epyc does NOT use 5kW. About 2300w with all at max tdp and that won’t be very often that all six GPUs and the CPU will all be under sustained maximum load. At least in my experience, it’s quite difficult to keep a workstation with 6 massive GPUs and a 64 core Epyc anywhere CLOSE to fully utilized.

 

[bit_user]

Six 7900xtx and a 64 core Epyc does NOT use 5kW. About 2300w with all at max tdp

I was talking about the RTX 4090 option, since we're talking about Nvidia's licensing constraints.

and that won’t be very often that all six GPUs and the CPU will all be under sustained maximum load. At least in my experience, it’s quite difficult to keep a workstation with 6 massive GPUs and a 64 core Epyc anywhere CLOSE to fully utilized.

Does your experience involve training neural networks? Because that's what this is for and I think that does involve sustained, intensive loads.