[SOLVED] Extremely slow DNG file transfer with Windows

5 OClock Charlie

Distinguished
Jul 3, 2009
2
0
18,510
We've been fighting absurdly slow dng file transfers (30-50MB/s max), both inbound from media cards and between fast internal drives, which often handle larger conventional video files at 500MB/s to over 1000 sustained transfer.

So it can take days to intake and backup a single day's multi-cam video shoot if RAW DNG was required.

This wasn't so drastic on a Mac to eHDD tested recently (If forget the exact speed).

I'm assuming the sheer number of files is central to the issue (the files are smallish at around 5MB or less per file usually).

Working theory was that the LaCie Big8 Thunderbolt 2 RAID 0 racks, though fantastic for large files, were slow given the use of 8 conventional HDDs seeking. Tried smaller than recommended stripe size (no help), disabling indexing and thumbnails (no help).

However, even transfer of DNGs to a fast M2 NVMe on the same PC seems to peek around 100MB/s at best.

Just curious if this is out of line with others' experiences, or an inevitable speed bottleneck when dealing with sequences of this individual file size.

I've found very few complaints about this specific problem on other forums, and no explanation or solution, so it's made me wonder if my experience is an outlier with a remedy.
 
Solution
Transfer speed depends on the slowest device in the chain.
The NVMe drive is just loafing while it waits for something to serve up the data. Either the SD cards or the USB interface.

kanewolf

Titan
Moderator
We've been fighting absurdly slow dng file transfers (30-50MB/s max), both inbound from media cards and between fast internal drives, which often handle larger conventional video files at 500MB/s to over 1000 sustained transfer.

So it can take days to intake and backup a single day's multi-cam video shoot if RAW DNG was required.

This wasn't so drastic on a Mac to eHDD tested recently (If forget the exact speed).

I'm assuming the sheer number of files is central to the issue (the files are smallish at around 5MB or less per file usually).

Working theory was that the LaCie Big8 Thunderbolt 2 RAID 0 racks, though fantastic for large files, were slow given the use of 8 conventional HDDs seeking. Tried smaller than recommended stripe size (no help), disabling indexing and thumbnails (no help).

However, even transfer of DNGs to a fast M2 NVMe on the same PC seems to peek around 100MB/s at best.

Just curious if this is out of line with others' experiences, or an inevitable speed bottleneck when dealing with sequences of this individual file size.

I've found very few complaints about this specific problem on other forums, and no explanation or solution, so it's made me wonder if my experience is an outlier with a remedy.
You are transferring from SD cards via USB? Have you looked at a thunderbolt SD card reader ?