Question USB thumb drive hideous real-world performance?

[David Taber]

Running Win10 Pro on an HP Envy laptop with plenty of memory, spare capacity, etc. All latest patches and drivers installed. No known hardware issues.

I have put two applications and all their data on a 64 GB SanDisk USB 3 SSD. There are about 400,000 files consuming about 40 GB of the space, a large number of files below 10 kB. Everything works but application performance is meh. Looked a task manager, and never saw a read or write above 20 MB/s, and that was for bigger files. For small ones, more like 80 kB/s. Looked at resource monitor, found disk queue length of 2-3, with disk utilization running at 60% even when I wasn't doing anything with the apps. Response time was sometimes as bad as 3000 ms.

In the interests of science, I took a 64 GB Patriot USB 2 SSD and copied all the data over to it. Fired up the apps and saw much worse performance, nearly unusable at the app level. Task manager showed peak reads and writes of less than 1 MB/s, resource monitor showed 100% disk utilization continuously, disk queue length pinned at 5, and response times as long as 10 seconds.

Obviously, USB 2 vs 3 makes a difference...but there were no other devices on that USB port...hard to believe that port contention was that much of an issue. And obviously, SSD benchmarks are always showing the best possible numbers using highly tailored data sets.

But is the small-file throughput really that bad for a USB SSD? Is the response time/latency/queue length typical of a thumb drive???

Does anyone have a recommendation for a faster USB SSD?

 

[Colif]

SSD or Flash drive?

their write speed is not that great, but read can be

 

[dwd999]

Running Win10 Pro on an HP Envy laptop with plenty of memory, spare capacity, etc. All latest patches and drivers installed. No known hardware issues.

I have put two applications and all their data on a 64 GB SanDisk USB 3 SSD. There are about 400,000 files consuming about 40 GB of the space, a large number of files below 10 kB. Everything works but application performance is meh. Looked a task manager, and never saw a read or write above 20 MB/s, and that was for bigger files. For small ones, more like 80 kB/s. Looked at resource monitor, found disk queue length of 2-3, with disk utilization running at 60% even when I wasn't doing anything with the apps. Response time was sometimes as bad as 3000 ms.

In the interests of science, I took a 64 GB Patriot USB 2 SSD and copied all the data over to it. Fired up the apps and saw much worse performance, nearly unusable at the app level. Task manager showed peak reads and writes of less than 1 MB/s, resource monitor showed 100% disk utilization continuously, disk queue length pinned at 5, and response times as long as 10 seconds.

Obviously, USB 2 vs 3 makes a difference...but there were no other devices on that USB port...hard to believe that port contention was that much of an issue. And obviously, SSD benchmarks are always showing the best possible numbers using highly tailored data sets.

But is the small-file throughput really that bad for a USB SSD? Is the response time/latency/queue length typical of a thumb drive???

Does anyone have a recommendation for a faster USB SSD?

A lot depends on what amount of money you wish to allocate to this project. There are certainly faster usbs but most of them are larger capacities and obviously more costly. The Kingston XS2000 is supposedly the fastest but the smallest one is 500gb and around $100. Alternately you could get a small m.2 drive for around $50 and an external enclosure like the Sabrent ENV-E which is around $30. But you would need to set a budget target first.

 

[Colif]

the size was what made me question if they using flash drives as not many 64gb ssd around anymore. I guess they could be old models...

 

[hotaru.hino]

But is the small-file throughput really that bad for a USB SSD? Is the response time/latency/queue length typical of a thumb drive???

Small file throughput destroys performance no matter what storage media you use. It's just the nature of the beast.

 

[David Taber]

Small file throughput destroys performance no matter what storage media you use. It's just the nature of the beast.

Well, sure...but this is MUCH worse than an HDD on the same USB port and my internal SDD. Seems like there's something going on with the firmware in the thumbdrive...

 

[David Taber]

the size was what made me question if they using flash drives as not many 64gb ssd around anymore. I guess they could be old models...

It's a thumdrive, it's definitely an SSD. It is about three years old, but it still shows as being a "good" drive.

 

[Lafong]

But is the small-file throughput really that bad for a USB SSD? Is the response time/latency/queue length typical of a thumb drive???

Does anyone have a recommendation for a faster USB SSD?

Never owned one, but it doesn't surprise me at all.

Sounds like the product cycle has taken the same path as USB flash drives where there is a very wide difference in performance between the best and the worst...much wider than 2 random spinning drives or 2 random internal SSDs. Most buyers are largely indifferent to performance in the interest of price and portability. They are just seen as commodities.

I've given up on flash drives for the same reason. Get an external enclosure and buy your own M.2 2280 connected by cable. A little bulkier but likely better performance.

 

[David Taber]

With some further testing found that:

• Thumb drives' performance varies significantly by manufacturer and even model.
  
  
• The advertised speeds in no way resemble real-world throughput, reading or writing (duh). Surprisingly, observed performance was significantly farther away from the advertised speed on reads. (In the immortal words of Sybase Founder Bob Epstein, "A benchmark number is what the manufacturer guarantees you will never attain.")
  
  
• If you copy files from a high-quality internal SSD to a thumbdrive, you'll notice that the throughput periodically dips by 50% or more, whether you're reading or writing to the thumbdrive. The effect is most frequent when working with small files, but on writing it can be quite dramatic with big files. The write-slowdown phenomenon is well documented and inherent to SSD's flash cycle.
  
  
• Why would there be a severe dip in read performance? My working theory: in the thumdrive there is an interface to the USB port, an interface to the memory cells, and a processor to do housekeeping and to present a "disk" for the file system to work with. While this may all be on a single chip, it has to be a cheap one and when working with a lot of file requests it gets overloaded. Under load, it must momentarily pause the USB bus so it can work on its backlog. This issue also occurs in HDDs, but the access times (averaging 10 ms or more) and read/write speeds give the internal processor lots of time to catch up under most circumstances.
  
  
• For mass file copying, the only result is uneven throughput. But if you're trying to run applications that store all their data on the thumbdrive, this can lead to very erratic performance ("application not responding" for 10 seconds or more). If the application is really picky in its file reading (i.e., it reads just a portion of the file, then re-reads the file again to get the next portion), the application may behave erratically or, in extreme cases, crash inexplicably.
  
  
• These effects can be even more dramatic if, like some of Sandisk's USB 3 thumbdrives, the SSD warms up a lot under heavy load. In this case, the entire drive may disconnect itself from the USB port, inevitably leading to application crashes and potential data corruption.

The bottom line: conventional thumbdrives are fine for archival storage and file transfer, but really aren't suitable for actively storing application data, and can be a really bad idea for actually running big applications stored on them. Heaven help you if you run your OS there...