- Dec 22, 2016
- 290
- 55
- 18,940
So I recently bought a new Lenovo E570, which includes an OEM Toshiba XG4 model M.2 NVMe SSD. The Toshiba XG4 has model number "thnsf5256gpuk". See Toshiba's catalog at https://toshiba.semicon-storage.com/info/docget.jsp?did=55015.
Long story short, without proper drivers, the performance is abysmal. Absolutely horrifically awfully terrible. Luckily, I was able to jury-rig a workaround. Compare my benchmarks using Microsoft's NVMe drivers and using OCZ's NVMe drivers for a non-OCZ drive: http://imgur.com/a/fAGil/ The primary takeaway lesson is that nobody should buy an NVMe drive until they have verified that the manufacturer provides proprietary NVMe drivers. Check the manufacturer's driver support page before purchasing.
Anyway, let me tell my story in detail, because hopefully it will benefit others.
In my Lenovo E570, I added a retail Kingston KC400 SATA SSD as personal storage. So now the computer has two SSDs: an OEM Toshiva M.2 NVMe as the Windows boot drive, and a retail Kingston SATA SSD as personal storage.
I ran some benchmarks to compare the two SSDs, and I found that the Toshiba had extremely subpar 4K random write performance. Basically, what this means is that if you try to write a lot of very small files (instead of a few large files), your Toshiba SSD will struggle and crawl. The Toshiba performed decently in 4K read, and it trounced the Kingston in sequential tasks (because it is NVMe), but in 4K write, the Toshiba was absolutely abysmal.
Specifically: whereas the Kingston KC400 got about 70 to 90 MB/s in 4K write, the Toshiba was getting about 1.5 to 1.6 MB/s. I benchmarked the Hitachi mechanical HDD in my old Lenovo E545, and it got about 0.5 to 0.75 MB/s in 4K write. So the Toshiba was still 3 times faster than a mechanical drive, but it was about 60 times slower than the Kingston SSD.
Needless to say, this is a serious problem.
I went to look up benchmarks for the Toshiba XG4 to see if my results were abnormal. Unfortunately, I couldn't find anything about the XG4 online (since it's an OEM drive), but I did see that the Toshiba XG3 (also NVMe) was rebranded as the OCZ RD400. So I looked up benchmarks for the OCZ drive, and it performed comparably to the top-of-the-line Samsung 950/960 Pro in 4K random writes!!! So why was my XG4 performing so abysmally?
The next thing I tried to do was find updated drivers or firmware for the Toshiba. Again, no dice. As an OEM drive, there was nothing. Then I updated my BIOS, chipset drivers, Intel RST, etc. (Of course, the Intel RST is for AHCI SATA, not M.2 NVMe, but I figured, you never know! Updated drivers rarely hurt!) Nothing helped. My performance was still awful.
Then I Googled around for anyone else having trouble with 4K writes, and a common denominator seemed to be people disabling write caching. Once they enabled write caching, their performance improved. I checked Device Manager, and write caching was already enabled for my Toshiba. But out of curiosity, I disabled write caching on the Kingston, and its 4K writes dropped down to the same 1.5-1.6 MB/s. So at least I knew that my Toshiba was performing *as if* write caching was disabled, even though it was enabled in the Device Manager. Still no solution to the problem, but at least, closer to a diagnosis.
So I started reading anything I could find about SSDs and write caching. I couldn't find much. But finally, I came across several extremely helpful articles:
http://www.sqlskills.com/blogs/glenn/beware-of-the-native-microsoft-nvme-driver
http://www.anandtech.com/show/10754/samsung-960-pro-ssd-review/2
http://www.anandtech.com/show/10909/the-plextor-m8pe-512gb-ssd-review
http://www.anandtech.com/show/9166/intel-nuc5i7ryh-boradwellu-iris-nuc-review/5
http://www.legitreviews.com/samsung-sm951-nvme-m-2-pcie-ssd-review_162219/9
Basically, what these four articles found was that the Microsoft NVMe drivers effectively disabled write caching by using FUA (forced unit access). So this means that if you use the Microsoft NVMe drivers, you won't be able to use the cache that built into your SSD. So whereas the Intel RST drivers successfully implement write caching for the SATA drives, the Microsoft NVMe drivers fail to implement write caching for NVMe drives. According to the articles I just posted, several NVMe drives (including Samsung's) performed poorly until their manufacturers released their own proprietary NVMe drivers to replace Microsoft's NVMe drivers. In the meantime, people had been having to disable write cache flushing to get decent performance out of their NVMe drives. (Disabling write cache flushing is a VERY bad idea because it creates a serious risk of data corruption in the event of a power failure or blue screen.) SATA drives were performing fine, but NVMe drives were crippled by the Microsoft drivers.
I called Toshiba and Microsoft and asked them both if they had any plans to release new NVMe drivers anytime soon. Both of them said they had no such plans.
So this put me in a tight bind. What was I supposed to do if Toshiba and Microsoft weren't releasing any new drivers?
Well, I started Googling to find out what would happen if I used another manufacturer's NVMe drivers. Several people posted online that they had successfully used Intel NVMe drivers for non-Intel SSDs. And I saw that even though Samsung has separate firmwares for every distinct drive, all of their drives use a common Windows NVMe driver. So it appeared to be the case that while every drive needed its own unique firmware to manage the SSD on the lowest hardware level, the Windows NVMe drivers were high enough level to be more or less interchangeable. So I figured, hey, let me try this. Since I'm not touching the firmware, worst case, I just go into safe mode and roll back my drivers to the original Microsoft drivers. As long as I don't touch the firmware, I'm unlikely to damage the hardware. So I backed up all my data and started installing other manufacturer's NVMe drivers.
The first one I tried was OCZ's RD400 NVMe driver, since the OCZ RD400 is a re-branding of the Toshiba XG3 (whereas my Lenovo has the Toshiba XG4). I installed it via the EXE wizard installer, rebooted my computer, and ran some benchmarks. No improvement. So I went into Device Manager and checked under "Storage Controllers". I saw that my system was still using Microsoft's "stornvme.sys". Apparently, OCZ's EXE wizard installer hadn't done the job. So I manually told Device Manager to update the driver for my NVMe storage controller, and I told it to show me drivers that are NOT compatible with my device. I looked under "Toshiba" and chose the XG3 driver. I installed it and rebooted my computer. I ran benchmarks. No improvement. So I went back into Device Manager and this time, I chose the OCZ RD400 driver listed under "Toshiba". I installed it and rebooted. Now, Device Manager showed it was using "ocznvme.sys" instead of Microsoft's "stornvme.sys". When I ran the benchmarks, I saw an ENORMOUS improvement. ABSOLUTELY GIGANTIC.
Let me show you all some benchmarks, and you'll see what I mean. So I think this is a valuable story about the importance of replacing Microsoft's NVMe drivers. As I said, don't purchase an NVMe drive unless the manufacturer's website indicates driver support.
Now then, the benchmarks. Again, pay special attention to 4K random write: http://imgur.com/a/fAGil
Long story short, without proper drivers, the performance is abysmal. Absolutely horrifically awfully terrible. Luckily, I was able to jury-rig a workaround. Compare my benchmarks using Microsoft's NVMe drivers and using OCZ's NVMe drivers for a non-OCZ drive: http://imgur.com/a/fAGil/ The primary takeaway lesson is that nobody should buy an NVMe drive until they have verified that the manufacturer provides proprietary NVMe drivers. Check the manufacturer's driver support page before purchasing.
Anyway, let me tell my story in detail, because hopefully it will benefit others.
In my Lenovo E570, I added a retail Kingston KC400 SATA SSD as personal storage. So now the computer has two SSDs: an OEM Toshiva M.2 NVMe as the Windows boot drive, and a retail Kingston SATA SSD as personal storage.
I ran some benchmarks to compare the two SSDs, and I found that the Toshiba had extremely subpar 4K random write performance. Basically, what this means is that if you try to write a lot of very small files (instead of a few large files), your Toshiba SSD will struggle and crawl. The Toshiba performed decently in 4K read, and it trounced the Kingston in sequential tasks (because it is NVMe), but in 4K write, the Toshiba was absolutely abysmal.
Specifically: whereas the Kingston KC400 got about 70 to 90 MB/s in 4K write, the Toshiba was getting about 1.5 to 1.6 MB/s. I benchmarked the Hitachi mechanical HDD in my old Lenovo E545, and it got about 0.5 to 0.75 MB/s in 4K write. So the Toshiba was still 3 times faster than a mechanical drive, but it was about 60 times slower than the Kingston SSD.
Needless to say, this is a serious problem.
I went to look up benchmarks for the Toshiba XG4 to see if my results were abnormal. Unfortunately, I couldn't find anything about the XG4 online (since it's an OEM drive), but I did see that the Toshiba XG3 (also NVMe) was rebranded as the OCZ RD400. So I looked up benchmarks for the OCZ drive, and it performed comparably to the top-of-the-line Samsung 950/960 Pro in 4K random writes!!! So why was my XG4 performing so abysmally?
The next thing I tried to do was find updated drivers or firmware for the Toshiba. Again, no dice. As an OEM drive, there was nothing. Then I updated my BIOS, chipset drivers, Intel RST, etc. (Of course, the Intel RST is for AHCI SATA, not M.2 NVMe, but I figured, you never know! Updated drivers rarely hurt!) Nothing helped. My performance was still awful.
Then I Googled around for anyone else having trouble with 4K writes, and a common denominator seemed to be people disabling write caching. Once they enabled write caching, their performance improved. I checked Device Manager, and write caching was already enabled for my Toshiba. But out of curiosity, I disabled write caching on the Kingston, and its 4K writes dropped down to the same 1.5-1.6 MB/s. So at least I knew that my Toshiba was performing *as if* write caching was disabled, even though it was enabled in the Device Manager. Still no solution to the problem, but at least, closer to a diagnosis.
So I started reading anything I could find about SSDs and write caching. I couldn't find much. But finally, I came across several extremely helpful articles:
http://www.sqlskills.com/blogs/glenn/beware-of-the-native-microsoft-nvme-driver
http://www.anandtech.com/show/10754/samsung-960-pro-ssd-review/2
http://www.anandtech.com/show/10909/the-plextor-m8pe-512gb-ssd-review
http://www.anandtech.com/show/9166/intel-nuc5i7ryh-boradwellu-iris-nuc-review/5
http://www.legitreviews.com/samsung-sm951-nvme-m-2-pcie-ssd-review_162219/9
Basically, what these four articles found was that the Microsoft NVMe drivers effectively disabled write caching by using FUA (forced unit access). So this means that if you use the Microsoft NVMe drivers, you won't be able to use the cache that built into your SSD. So whereas the Intel RST drivers successfully implement write caching for the SATA drives, the Microsoft NVMe drivers fail to implement write caching for NVMe drives. According to the articles I just posted, several NVMe drives (including Samsung's) performed poorly until their manufacturers released their own proprietary NVMe drivers to replace Microsoft's NVMe drivers. In the meantime, people had been having to disable write cache flushing to get decent performance out of their NVMe drives. (Disabling write cache flushing is a VERY bad idea because it creates a serious risk of data corruption in the event of a power failure or blue screen.) SATA drives were performing fine, but NVMe drives were crippled by the Microsoft drivers.
I called Toshiba and Microsoft and asked them both if they had any plans to release new NVMe drivers anytime soon. Both of them said they had no such plans.
So this put me in a tight bind. What was I supposed to do if Toshiba and Microsoft weren't releasing any new drivers?
Well, I started Googling to find out what would happen if I used another manufacturer's NVMe drivers. Several people posted online that they had successfully used Intel NVMe drivers for non-Intel SSDs. And I saw that even though Samsung has separate firmwares for every distinct drive, all of their drives use a common Windows NVMe driver. So it appeared to be the case that while every drive needed its own unique firmware to manage the SSD on the lowest hardware level, the Windows NVMe drivers were high enough level to be more or less interchangeable. So I figured, hey, let me try this. Since I'm not touching the firmware, worst case, I just go into safe mode and roll back my drivers to the original Microsoft drivers. As long as I don't touch the firmware, I'm unlikely to damage the hardware. So I backed up all my data and started installing other manufacturer's NVMe drivers.
The first one I tried was OCZ's RD400 NVMe driver, since the OCZ RD400 is a re-branding of the Toshiba XG3 (whereas my Lenovo has the Toshiba XG4). I installed it via the EXE wizard installer, rebooted my computer, and ran some benchmarks. No improvement. So I went into Device Manager and checked under "Storage Controllers". I saw that my system was still using Microsoft's "stornvme.sys". Apparently, OCZ's EXE wizard installer hadn't done the job. So I manually told Device Manager to update the driver for my NVMe storage controller, and I told it to show me drivers that are NOT compatible with my device. I looked under "Toshiba" and chose the XG3 driver. I installed it and rebooted my computer. I ran benchmarks. No improvement. So I went back into Device Manager and this time, I chose the OCZ RD400 driver listed under "Toshiba". I installed it and rebooted. Now, Device Manager showed it was using "ocznvme.sys" instead of Microsoft's "stornvme.sys". When I ran the benchmarks, I saw an ENORMOUS improvement. ABSOLUTELY GIGANTIC.
Let me show you all some benchmarks, and you'll see what I mean. So I think this is a valuable story about the importance of replacing Microsoft's NVMe drivers. As I said, don't purchase an NVMe drive unless the manufacturer's website indicates driver support.
Now then, the benchmarks. Again, pay special attention to 4K random write: http://imgur.com/a/fAGil
Facebook
Twitter
Instagram