News Gigabyte AI TOP 100E SSD features incredible 219,000 TBW endurance rating — 183X more than the venerable Samsung 990 Pro

[CmdrShepard]

The naming doesn't matter, calm down.

It does because it is deceptive and done to game SEO.

 

[bit_user]

Why would a sane, normal person who knows what they are doing ever do such a stupid thing?!?

It's:

- Cheaper
- Faster
- More power-efficient
- Doesn't require special software

To just buy more RAM, or if you can't afford it just quantize the damn LLM down to a manageable size.

I disagree that it's cheaper. We don't know how much this costs, but a similar amount of RAM is almost certainly going to be way more expensive. Not to mention the scenario of trying to inference a model that's bigger than the amount of RAM a machine can support. Then, you have to upgrade to the next tier of CPU, board, etc. If we're talking about laptops, some people would never want to lug around a mobile-workstation class machine (myself included).

With theses new round of AI PCs packing so much inferencing horsepower in the CPU, you could be limited to just 32 or 64 GB of RAM, which isn't big enough for some popular models that currently exist, let alone future ones.

As for the speed argument, I think the performance impact of paging in needed pieces of the model might be manageable, if you can keep the most frequently-used parts in RAM and predictively load most of the needed parts in advance of when they're referenced.

This is clearly just a gimmick to fleece some dumb people.

Maybe, but at least the specs meaningfully differ from other drives on the market and they have a story that matches up. It's not like some of the other cynical opportunists we've seen who will basically slap the term "AI" on existing products, just for buzzword compliance.

No, no we don't. We need to let it fade into obscurity where it belongs and hope that such idiotic naming doesn't catch with other AIBs.

Some of us are curious how they achieved the claimed endurance numbers, independent of whether we agree with their stated use case for the product.

If you're cynical, you might imagine this drive was originally designed for mining Chia (that cryptocurrency which stresses your SSD instead of your GPU) and now they resurrected it to capitalize on the AI boom. Doesn't change how impressive the endurance numbers are.

 

[bit_user]

I've found it very hard to find TOPS ratings (INT8) for some of the consumer discrete GPUs. For example, that figure isn't on Nvidia's website or TechPowerUp for the RTX 3060. I have to go to this Wccftech article to find out it's supposedly 101 TOPS. Any leads on this?

I used the wikipedia page. I'm not going to argue the specifics, as my point was just that you don't need a terribly exotic dGPU to meet Microsoft's threshold for an AI PC and I think we're in agreement on that.

Well you could argue that point about GPU's too! Lets hope nVidia/AMD don't start using AI TOPS in the already really long winded naming schemes currently in use. RTX4070 Ti Super AI TOPS!! Or AIB partners: 4070 Ti Super AI TOPS Windforce 57 SCC OC!!! Uggghh. No thanks.

They don't need to? They're practically synonymous with AI, these days.

 

[helper800]

What was the warranty TBW on that?

I scoured the internet multiple times to find such information and never could get an answer to that very question. It doesn't even have a DWPD stat associated with it. All I know is that when I was having OS stability issues for a the last 1.5 years of its use, I finally checked S.M.A.R.T. and it showed what I wrote above. It had to of had read or write errors. I reinstalled windows on it multiple times and it would be fine for a month or two and then start BSODing. I finally deduced it had to be the drive and got a new one. New windows install and 2.5 years later, no BSODs what-so-ever.

 

[USAFRet]

I scoured the internet multiple times to find such information and never could get an answer to that very question. It doesn't even have a DWPD stat associated with it. All I know is that when I was having OS stability issues for a the last 1.5 years of its use, I finally checked S.M.A.R.T. and it showed what I wrote above. It had to of had read or write errors. I reinstalled windows on it multiple times and it would be fine for a month or two and then start BSODing. I finally deduced it had to be the drive and got a new one. New windows install and 2.5 years later, no BSODs what-so-ever.

I looked as well. Found nothing.

A similar era 120GB Kingston HyperX 3K (I have one), has a warranty value of 290TB.
(not that this really means anything)

 

[helper800]

I looked as well. Found nothing.

A similar era 120GB Kingston HyperX 3K (I have one), has a warranty value of 290TB.
(not that this really means anything)

The OCZ Agility 3 came out in earlyish 2011 when I got it so its even older tech than the Kingston drive. I have also seen conflicting data on the TBW drive endurance for that drive. Maybe there were multiple revisions with different TBW endurance numbers? Here is one of the sources for that:

• Total Bytes Written (TBW)
  • 90GB: 57.6TB
  • 120GB: 76.8TB
  • 240GB: 153.6TB
  • 480GB: 307.2TB

Samsung drives of the same era were given anywhere between 35 and 73 TBW for their drives. I cannot imagine the budget OCZ drive of only 120gb having more.

 

[USAFRet]

Maybe there were multiple revisions with different TBW endurance numbers?

Quite possibly.

In any case, a regular user running past the TBW number is relatively rare.
Especially with newer drives.

I've seen people ridicule a drive with a warranty of 750TBW, in favor of a comparable one with 1200TBW.
In reality, a regular consumer will never see either number.

 

[helper800]

Quite possibly.

In any case, a regular user running past the TBW number is relatively rare.
Especially with newer drives.

I've seen people ridicule a drive with a warranty of 750TBW, in favor of a comparable one with 1200TBW.
In reality, a regular consumer will never see either number.

Yeah, I did a TON of redownloading games on that OCZ drive for years and years on end to get a meager 56 TBW. I added a bit to the bottom of my last post. Though I say that and I apparently have 49 TBW already on my 970 evo as my OS drive on this computer. I think I can sleep at night with 97% drive life remaining, to be fair.

 

[bit_user]

Yeah, I did a TON of redownloading games on that OCZ drive for years and years on end to get a meager 56 TBW.

This can't match what you'd see if you were continually under heavy swapping.

I've also watched the iostats for some web browsers without privacy add-ons, and some websites hit your cookies really hard. You might be thinking that shouldn't amount to much I/O, but the I/O Writes counter in Task Manager says otherwise. Consider that each of those will be subject to write-amplification and encompass at least a block, and now you're getting into serious territory. Right now, I have one Firefox process with 724 GB written, and that's before accounting for amplification. The number of I/O writes is 1.926B. SMART isn't telling me what the drive claims about TBW.

 

[CmdrShepard]

This can't match what you'd see if you were continually under heavy swapping.

I've also watched the iostats for some web browsers without privacy add-ons, and some websites hit your cookies really hard. You might be thinking that shouldn't amount to much I/O, but the I/O Writes counter in Task Manager says otherwise. Consider that each of those will be subject to write-amplification and encompass at least a block, and now you're getting into serious territory. Right now, I have one Firefox process with 724 GB written, and that's before accounting for amplification. The number of I/O writes is 1.926B. SMART isn't telling me what the drive claims about TBW.

That's not cookies, it's session store and history and it can be tuned down.

Anyway, I still maintain that if you are swapping you would benefit from more RAM, not from a disk more resilient to excessive writes. It's common sense.

Also the claim is so outrageous that it must come with a huge asterisk somewhere.

We have a saying here when someone makes outrageous claims "Yeah, and above a hole on a (wooden) fence it says p...."

 

[Li Ken-un]

Last I checked, enterprise M.2 drives are pretty much extinct. I bought the last one I could find, and that was a Samsung PM9A3. The only place I could still find them for sale is ebay.

They are fairly close to being extinct. They can be found, but you’d need deep enough pockets to meet their MOQ or pay ridiculous markups for a paltry capacity.

 

[bit_user]

Anyway, I still maintain that if you are swapping you would benefit from more RAM, not from a disk more resilient to excessive writes. It's common sense.

Yes, but there are limits to how much RAM different machines can accept and it might not be as much as you need. Then, there's the cost argument. NAND is way cheaper, GB/$ - probably even in this SSD!

If you look at something like a VM host, they eat RAM for breakfast. This might be good for having lots of VMs.

Also the claim is so outrageous that it must come with a huge asterisk somewhere.

You're probably right. That's one of the reasons I want to know the details!

 

[bit_user]

They are fairly close to being extinct. They can be found, but you’d need deep enough pockets to meet their MOQ or pay ridiculous markups for a paltry capacity.

To be honest, I'm not sure it was worth it. The thing is a 110 mm long, double-sided drive, with high idle power (I forget exactly, but maybe around 5 W?) and doesn't support ASPM. I could only find one SSD cooler that provided full coverage of both sides and it's not awesome (but probably good enough).

For another machine, I got a Samsung 990 Pro about 6 months later. Double the capacity for I think a similar price, much lower idle & peak power (even without ASPM) and better peak performance numbers. The PM9A3 probably has better tail latencies, but I'm not doing anything with it where I'd probably notice.

 

[Li Ken-un]

With theses new round of AI PCs packing so much inferencing horsepower in the CPU, you could be limited to just 32 or 64 GB of RAM, which isn't big enough for some popular models that currently exist, let alone future ones.

As for the speed argument, I think the performance impact of paging in needed pieces of the model might be manageable, if you can keep the most frequently-used parts in RAM and predictively load most of the needed parts in advance of when they're referenced.

I got myself a degree in Data Science on a whim a couple years ago. In the middle of the coursework, I found that my machine could barely complete the training loads I threw at it. And that was with toy datasets. What did I do? I made the swap file take the brunt of the infrequently accessed memory just so the Python script wouldn’t die. I don’t quite recall the algorithm I used, but using the SSD as a memory extension did the trick. *

I recall for that machine it could not accommodate any more than 32 GiB RAM. So the RAM capacity supported is a legitimate limitation to consider. The next step up (HEDT/workstation) costs a great enough premium that there is most definitely a space for this “AI TOP” SSD to squeeze in.


* Using a NAND SSD as memory had to have shortened the endurance materially, but alas, I did not think to check at the time, and I also don’t have that SSD.

 

[Pierce2623]

The capacities would suggest it's MLC, not SLC. It's probably the same NAND chips modern enthusiast SSD use, but just configured all to run in pseudo-MLC mode. When you do that on NAND capable of usable endurance in QLC mode, it's probably not too surprising (TBH, I wouldn't have guessed these endurance levels would be possible outside of SLC).


No, only enthusiast drives are TLC. Most of the consumer world is using QLC.


Okay, so 182.5 TBW vs. 219,000 TBD? You're off by 3 orders of magnitude.

Anyway, I think you're reading too much into the 2.5x discrepancy. Part of it is that the manufacturer needs to build in some margin of error, because the endurance is going to follow something like a bell curve. They'd want to be sure that they guarantee about 2-3 sigmas less than the mean endurance.

Also, they probably allow for it to be operated at higher temperatures than TechRepublic's testing used, and that's very detrimental to endurance!


I'm pretty sure it doesn't need to get that bad, before they'll accept it for return under warranty.


Last I checked, enterprise M.2 drives are pretty much extinct. I bought the last one I could find, and that was a Samsung PM9A3. The only place I could still find them for sale is ebay.

Yeah enterprise has mostly switched to U.2 but during the heyday of the 970 Evo(so pcie3 days) it was still fairly easy to find enterprise M.2s. I eventually quit buying them because I realized consumer drives now have enough over-provisioning to generally go well past their TBW rating before they show ANY sort of issues.

 

[bit_user]

Yeah enterprise has mostly switched to U.2

They mostly went straight to U.2, I think. Now, they seem to be all in on the E1.S and E3 form factors.

during the heyday of the 970 Evo(so pcie3 days) it was still fairly easy to find enterprise M.2s. I eventually quit buying them because I realized consumer drives now have enough over-provisioning to generally go well past their TBW rating before they show ANY sort of issues.

It's not only that, but also things like low tail latencies that enterprise drives tend to optimize for. Unfortunately, that also means they tend to idle pretty hot.

 

[Pierce2623]

They mostly went straight to U.2, I think. Now, they seem to be all in on the E1.S and E3 form factors.


It's not only that, but also things like low tail latencies that enterprise drives tend to optimize for. Unfortunately, that also means they tend to idle pretty hot.

Well enterprise had <Mod Edit> lots of 2.5” SATA SSDs before they went U.2. Or at least the first data centers I worked in around 2012 were packed full of them <Mod Edit>. To be clear, these were fairly small AT&T installations not some huge Google site.

 

[Pyrolite]

If this is an all-pSLC(or even pMLC) drive, I think random read performance could be very nice.

 

[bit_user]

If this is an all-pSLC(or even pMLC) drive, I think random read performance could be very nice.

Why? Because no need to re-read if an error is encountered (i.e. since no errors should occur)?

Naively, one might expect higher read speeds to come along with higher density.

 

[Li Ken-un]

If this is an all-pSLC(or even pMLC) drive, I think random read performance could be very nice.

For random reads, true.

The TechPowerup SSD database appears to have the details now, and from what I can tell, the page size will be 8 KiB. If that’s accurate, then random writes may suffer from read-modify-write and a greater write amplification factor. Clever controllers will probably just coalesce several random writes into one big write anyway; the 40% overprovisioning will help with that. But we’ll see once it gets reviewed.

Spoiler

One might say “just use Optane” for random I/O, but for the mini PC and laptop markets, there aren’t many great choices. I excised a 500 GB NAND SSD from my laptop and replaced it with 118 GB Optane. To my mild annoyance, I had to give up hibernation, a page file, and a whole bunch of software just to recover enough free space to work comfortably in.

Depending on the performance characteristics of this drive, if it’s got good enough random write and latency, I’ll swap out my Optane for it. The 380 GB Optane is 110 mm long, and if I don’t have to, then I’m going to postpone buying a Framework just to use them.

 

[bit_user]

One might say “just use Optane” for random I/O, but for the mini PC and laptop markets, there aren’t many great choices. I excised a 500 GB NAND SSD from my laptop and replaced it with 118 GB Optane. To my mild annoyance, I had to give up hibernation, a page file, and a whole bunch of software just to recover enough free space to work comfortably in.

Not only that, but isn't its interface just PCIe 3.0 x2?

Depending on the performance characteristics of this drive, if it’s got good enough random write and latency, I’ll swap out my Optane for it. The 380 GB Optane is 110 mm long, and if I don’t have to, then I’m going to postpone buying a Framework just to use them.

I'd suggest you check its idle power consumption. I think it's going to be a fair bit worse than most NAND-based SSDs.

 

[Li Ken-un]

I'd suggest you check its idle power consumption. I think it's going to be a fair bit worse than most NAND-based SSDs.

That was something I already had to give in to since the 800P wasn’t recognized by my laptop. The P1600X―which idles at 1.7 W (far more than the 8 microwatts of the 800P)―but surprisingly less power hungry than I anticipated, and I can get through the day with no perceptible loss of uptime.

The 2.52 W idle of the 905P might actually make a difference, but I’ll have to give it a spin before I draw any conclusions. I’ve yet to get around to sanding off the internal protrusions in my laptop so that I can fit a M.2 22110 in it.

If dev workloads don’t take forever, I’d say it’s worth it. 😎 But I’d trade up a bit of the latency advantage for more capacity.

 

[bit_user]

I’ve yet to get around to sanding off the internal protrusions in my laptop so that I can fit a M.2 22110 in it.

I just went down a similar path about a month ago. In my case, I wanted to mount heatsinks on the backside of a mini-ITX motherboard. The underside of the motherboard had some surface-mount components, but what worried me most were the leads of the VRM capacitors poking through their solder blobs.

So, the first thing I did was to grab a file. That was slow going and, after scratching the PCB a few times (exposing internal traces, a couple times) and a couple worrying bumps into some of those surface-mount components, I finally gave up on the idea of using a file. I remembered I had a Dremel tool, somewhere, that I had only ever used once or twice in the decades since I bought it. The grinding wheels made quick work of the leads and solder blobs. I did nick the motherboard a couple times with it, but not in worrying ways or places.

In the end, the patient survived the surgery. I then used a coat of this to cover the remaining solder bumps and those surface mount components I mentioned:

https://www.amazon.com/MG-Chemicals-Insulating-Varnish-Liquid/dp/B0BMNR4L3F/
​

Pro Tips:

• Make sure to wear proper PPE. That's goggles or a face shield and a good dust mask. I found that my half face respirator with organic vapor filters did a superb job of filtering out the varnish fumes. Something like that only costs $20 on Amazon.
• If you use the varnish, do so outdoors and let the item dry overnight. Then, operate it in a somewhat ventilated area, to accelerate the remaining drying/curing phase.
• Seal the varnish and brushes in double ziplock bags, when your done, to keep any leaking fumes safely contained. I always do this with hazardous chemicals.
• Seal your respirator (or just the filters, if you detach them) in a ziplock bag, or else you'll find they degrade rather quickly.

If dev workloads don’t take forever, I’d say it’s worth it. 😎 But I’d trade up a bit of the latency advantage for more capacity.

IMO, a decent SSD and plenty of RAM is probably all you need, but suit yourself.

Good luck!

 

[Pierce2623]

If this is an all-pSLC(or even pMLC) drive, I think random read performance could be very nice.

There’s no way they’re running TLC as pSLC. After you take over provisioning into account too, you’re talking 7-8 TB of flash for a 2TB drive.