[SOLVED] Extremely slow write speed M.2 NVMe ?

tek3195

Prominent
Feb 7, 2021
67
4
545
I have a Samsung 970 EVO 500GB with an average read rate of 3.2 GB/s and average write of 292 MB/s. I know it says speeds are "up to" 3500/2500 MB/s but there has to be something wrong with my write speed. Don't have any NTFS partitions on it so can't check it with Samsung software, although other drive software(sk hynix) shows it as good and healthy from an external windows.. I don't know how to find the issue, the limited info I got from nvme-cli and smartctl aren't showing anything that helped me. I've included nvme smartlog and inxi output and link to gnome-disk-utility benchmarks with diff size samples. Can someone help me find where the issue is ?

specs: core i7-9700, ASUS Prime B365M-A, TEAM 32GB 4x8@3000MHz, Samsung 970 EVO 500GB, SK Hynix S31 Gold SSD 1TB, WD Blue 2.5" HDD 2TB, MSI GTX 970
For OS I use an Ubuntu Cinnamon remix that I built from source as a daily driver, the benchmark results are from LMDE4 that is installed on 2.5"SSD.

screenshots

Code:
root@lmde4:/home/tony# nvme  smart-log /dev/nvme0n1
Smart Log for NVME device:nvme0n1 namespace-id:ffffffff
critical_warning                    : 0
temperature                         : 50 C
available_spare                     : 100%
available_spare_threshold           : 10%
percentage_used                     : 2%
data_units_read                     : 15,791,618
data_units_written                  : 35,521,921
host_read_commands                  : 140,700,643
host_write_commands                 : 135,467,542
controller_busy_time                : 825
power_cycles                        : 744
power_on_hours                      : 1,166
unsafe_shutdowns                    : 339
media_errors                        : 0
num_err_log_entries                 : 2,449
Warning Temperature Time            : 0
Critical Composite Temperature Time : 0
Temperature Sensor 1                : 50 C
Temperature Sensor 2                : 29 C
Thermal Management T1 Trans Count   : 0
Thermal Management T2 Trans Count   : 0
Thermal Management T1 Total Time    : 0
Thermal Management T2 Total Time    : 0

root@lmde4:/home/tony# inxi -v 7
System:    Host: lmde4 Kernel: 4.19.0-18-amd64 x86_64 bits: 64 compiler: gcc v: 8.3.0 Desktop: Cinnamon 5.0.5
           wm: muffin 5.0.1 dm: LightDM 1.26.0 Distro: LMDE 4 Debbie base: Debian 10.2 buster
Machine:   Type: Desktop Mobo: ASUSTeK model: PRIME B365M-A v: Rev X.0x serial: 201177395101436
           UEFI: American Megatrends v: 1605 date: 07/14/2020
Battery:   Device-1: hidpp_battery_0 model: Logitech MX Keys Wireless Keyboard serial: 408a-dd-7d-ca-6e
           charge: 100% (should be ignored) rechargeable: yes status: Discharging
           Device-2: hidpp_battery_1 model: Logitech MX Ergo Multi-Device Trackball serial: 406f-fb-ba-a2-a0
           charge: 55% (should be ignored) rechargeable: yes status: Discharging
Memory:    RAM: total: 31.35 GiB used: 1.63 GiB (5.2%)
           Array-1: capacity: 64 GiB slots: 4 EC: None max module size: 16 GiB note: est.
           Device-1: ChannelA-DIMM1 size: 8 GiB speed: 2666 MT/s type: DDR4
           detail: synchronous unbuffered (unregistered) bus width: 64 bits total: 64 bits
           manufacturer: Team Group Inc. part-no: TEAMGROUP-UD4-3000 serial: 01A82AEF
           Device-2: ChannelA-DIMM2 size: 8 GiB speed: 2666 MT/s type: DDR4
           detail: synchronous unbuffered (unregistered) bus width: 64 bits total: 64 bits
           manufacturer: Team Group Inc. part-no: TEAMGROUP-UD4-3000 serial: 01A40B0B
           Device-3: ChannelB-DIMM1 size: 8 GiB speed: 2666 MT/s type: DDR4
           detail: synchronous unbuffered (unregistered) bus width: 64 bits total: 64 bits
           manufacturer: Team Group Inc. part-no: TEAMGROUP-UD4-3000 serial: 01A82A8D
           Device-4: ChannelB-DIMM2 size: 8 GiB speed: 2666 MT/s type: DDR4
           detail: synchronous unbuffered (unregistered) bus width: 64 bits total: 64 bits
           manufacturer: Team Group Inc. part-no: TEAMGROUP-UD4-3000 serial: 01A4203E
CPU:       Topology: 8-Core model: Intel Core i7-9700 bits: 64 type: MCP arch: Kaby Lake rev: D
           L2 cache: 12.0 MiB bogomips: 48000
           Speed: 800 MHz min/max: 800/4700 MHz Core speeds (MHz): 1: 800 2: 800 3: 800 4: 800 5: 800 6: 800
           7: 800 8: 800
           Flags: 3dnowprefetch abm acpi adx aes aperfmperf apic arat arch_capabilities arch_perfmon art avx
           avx2 bmi1 bmi2 bts clflush clflushopt cmov constant_tsc cpuid cpuid_fault cx16 cx8 de ds_cpl dtes64
           dtherm dts epb ept ept_ad erms est f16c flexpriority flush_l1d fma fpu fsgsbase fxsr ht hwp
           hwp_act_window hwp_epp hwp_notify ibpb ibrs ibrs_enhanced ida intel_pt invpcid invpcid_single
           lahf_lm lm mca mce md_clear mmx monitor movbe mpx msr mtrr nonstop_tsc nopl nx pae pat pbe pcid
           pclmulqdq pdcm pdpe1gb pebs pge pln pni popcnt pse pse36 pts rdrand rdseed rdtscp rep_good sdbg sep
           smap smep smx ss ssbd sse sse2 sse4_1 sse4_2 ssse3 stibp syscall tm tm2 tpr_shadow tsc tsc_adjust
           tsc_deadline_timer tsc_known_freq vme vmx vnmi vpid x2apic xgetbv1 xsave xsavec xsaveopt xsaves
           xtopology xtpr
Graphics:  Device-1: NVIDIA GM204 [GeForce GTX 970] vendor: Micro-Star MSI driver: nvidia v: 450.66
           bus ID: 01:00.0 chip ID: 10de:13c2
           Display: x11 server: X.Org 1.20.4 driver: nvidia unloaded: fbdev,modesetting,nouveau,vesa
           alternate: nv resolution: 1920x1080~60Hz
           OpenGL: renderer: GeForce GTX 970/PCIe/SSE2 v: 4.6.0 NVIDIA 450.66 direct render: Yes
Audio:     Device-1: Intel 200 Series PCH HD Audio vendor: ASUSTeK driver: snd_hda_intel v: kernel
           bus ID: 00:1f.3 chip ID: 8086:a2f0
           Device-2: NVIDIA GM204 High Definition Audio vendor: Micro-Star MSI driver: snd_hda_intel v: kernel
           bus ID: 01:00.1 chip ID: 10de:0fbb
           Sound Server: ALSA v: k4.19.0-18-amd64
Network:   Device-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet vendor: ASUSTeK driver: r8169
           v: kernel port: d000 bus ID: 04:00.0 chip ID: 10ec:8168
           IF: enp4s0 state: up speed: 1000 Mbps duplex: full mac: 3c:7c:3f:c2:e8:cc
           IP v4: 192.168.0.19/24 type: dynamic noprefixroute scope: global broadcast: 192.168.0.255
           IP v6: fe80::ca22:f317:9165:3e2c/64 type: noprefixroute scope: link
           IF-ID-1: docker0 state: down mac: 02:42:7d:f5:99:76
           IP v4: 172.17.0.1/16 scope: global broadcast: 172.17.255.255
           WAN IP: 69.73.110.142
Drives:    Local Storage: total: 4.09 TiB used: 444.59 GiB (10.6%)
           ID-1: /dev/nvme0n1 vendor: Samsung model: SSD 970 EVO 500GB size: 465.76 GiB speed: 31.6 Gb/s
           lanes: 4 serial: S5H7NS0NA72460H rev: 2B2QEXE7 temp: 50 C scheme: GPT
           ID-2: /dev/sda vendor: Smart Modular Tech. model: SHGS31-1000GS-2 size: 931.51 GiB speed: 6.0 Gb/s
           serial: FN08N851910809159 rev: 0Q00 temp: 28 C scheme: GPT
           ID-3: /dev/sdb vendor: Western Digital model: WD20SPZX-22UA7T0 size: 1.82 TiB speed: 6.0 Gb/s
           rotation: 5400 rpm serial: WD-WXQ2E704AVC3 rev: 1A01 temp: 34 C scheme: GPT
           ID-4: /dev/sdc type: USB vendor: Western Digital model: WD My Passport 2626 size: 931.48 GiB
           serial: 575847314135393650455532 rev: 1028 scheme: GPT
           Message: No Optical or Floppy data was found.
RAID:      Message: No RAID data was found.
Partition: ID-1: / size: 194.96 GiB used: 13.68 GiB (7.0%) fs: ext4 dev: /dev/sda2 label: N/A
           uuid: 6555f6cd-095f-4115-8a2c-64067a09da14
           ID-2: /boot/efi size: 96.0 MiB used: 40.0 MiB (41.6%) fs: vfat dev: /dev/sdc1 label: N/A
           uuid: 0000-69F9
           ID-3: /media/tony/WinToUSB size: 195.31 GiB used: 177.05 GiB (90.7%) fs: ntfs dev: /dev/sdc2
           label: WinToUSB uuid: 01D6A181B75C2210
           ID-4: /media/tony/lmde size: 421.06 GiB used: 43.82 GiB (10.4%) fs: ext4 dev: /dev/sdc5 label: lmde
           uuid: af939e5c-cd03-4171-bb36-9d435c4b5db5
           ID-5: /media/tony/share size: 199.85 GiB used: 151.42 GiB (75.8%) fs: ntfs dev: /dev/sdc3
           label: share uuid: 57B046996023DE60
           ID-6: /media/tony/split size: 107.42 GiB used: 58.58 GiB (54.5%) fs: ntfs dev: /dev/sdc4
           label: split uuid: 7346F6B249CDC16F
Unmounted: ID-1: /dev/nvme0n1p1 size: 500.0 MiB fs: vfat label: N/A uuid: 27D4-CF0C
           ID-2: /dev/nvme0n1p2 size: 458.98 GiB fs: ext4 label: N/A uuid: 920139da-3701-4549-acd4-c459a9282e07
           ID-3: /dev/nvme0n1p3 size: 6.29 GiB fs: swap label: N/A uuid: 9ad9a38e-2f74-471b-ad9e-dfb0e492eea6
           ID-4: /dev/sda1 size: 732.42 GiB fs: ext4 label: N/A uuid: d29f29f2-4961-4cf8-b5a3-dc6d7806d4e8
           ID-5: /dev/sdb1 size: 656.81 GiB fs: ext4 label: stowage uuid: 4705edc2-24f5-48b8-a6ff-20568e7b8d7d
           ID-6: /dev/sdb2 size: 41.91 GiB fs: ext3 label: N/A uuid: 7f25e84d-2db1-4e42-9424-3f35a02ddbc0
           ID-7: /dev/sdb3 size: 93.13 GiB fs: ext4 label: N/A uuid: f7ef2292-e390-4ef8-b170-6dacb2ca7420
           ID-8: /dev/sdb4 size: 465.52 GiB fs: ntfs label: share uuid: 6568A3E4497A7EAA
USB:       Hub: 1-0:1 info: Full speed (or root) Hub ports: 16 rev: 2.0 speed: 480 Mb/s chip ID: 1d6b:0002
           Device-1: 1-8:2 info: Logitech Unifying Receiver type: Keyboard,Mouse,HID
           driver: logitech-djreceiver,usbhid interfaces: 3 rev: 2.0 speed: 12 Mb/s chip ID: 046d:c52b
           Hub: 2-0:1 info: Full speed (or root) Hub ports: 8 rev: 3.0 speed: 5 Gb/s chip ID: 1d6b:0003
           Device-2: 2-7:2 info: Western Digital type: Mass Storage driver: usb-storage interfaces: 1 rev: 3.1
           speed: 5 Gb/s chip ID: 1058:2626 serial: 575847314135393650455532
Sensors:   System Temperatures: cpu: 29.8 C mobo: 27.8 C gpu: nvidia temp: 40 C
           Fan Speeds (RPM): cpu: 0 gpu: nvidia fan: 0%
Info:      Processes: 233 Uptime: 33m Init: systemd v: 241 runlevel: 5 Compilers: gcc: 8.3.0 alt: 8
           Shell: bash (su) v: 5.0.3 running in: gnome-terminal inxi: 3.0.32
 
Solution
Ok, so I could test write speed on the same drive actually. I have a working directory for building android set up on my / partition on nvme and my /home is on ssd. I can write from / to /work directory, or any sub-directory to any sub-directory on the nvme drive to get speeds using dd can't I ? I don't know why I got hung up on using disk-utility benchmark tool, which of course can't right to active disk. I was trying to run it instead of just checking speeds.
Performance depends on the slowest device in the chain.

HDD -> SSD = speed of the HDD.

Copying to and from spaces on the same drive? Still slow. The drive is doing both things at the same time...read-write.

The only way you may see the advertised uber numbers is in...
So, if you're writing to this NVME PCIe drive FROM a SATA drive, you are going to be limited to SATA bus speeds. There is no getting around that. The only way, even theoretically, you could achieve anything resembling the theoretical write speeds would be if you were writing from NVME to NVME, and if both drives were capable of similar speeds.

And, since you're using a custom solution, there might also be driver issues involved that are making it somewhat more problematic than just that.
 

tek3195

Prominent
Feb 7, 2021
67
4
545
So, if you're writing to this NVME PCIe drive FROM a SATA drive, you are going to be limited to SATA bus speeds. There is no getting around that. The only way, even theoretically, you could achieve anything resembling the theoretical write speeds would be if you were writing from NVME to NVME, and if both drives were capable of similar speeds.

And, since you're using a custom solution, there might also be driver issues involved that are making it somewhat more problematic than just that.

Ok, so I could test write speed on the same drive actually. I have a working directory for building android set up on my / partition on nvme and my /home is on ssd. I can write from / to /work directory, or any sub-directory to any sub-directory on the nvme drive to get speeds using dd can't I ? I don't know why I got hung up on using disk-utility benchmark tool, which of course can't right to active disk. I was trying to run it instead of just checking speeds.
 

USAFRet

Titan
Moderator
Ok, so I could test write speed on the same drive actually. I have a working directory for building android set up on my / partition on nvme and my /home is on ssd. I can write from / to /work directory, or any sub-directory to any sub-directory on the nvme drive to get speeds using dd can't I ? I don't know why I got hung up on using disk-utility benchmark tool, which of course can't right to active disk. I was trying to run it instead of just checking speeds.
Performance depends on the slowest device in the chain.

HDD -> SSD = speed of the HDD.

Copying to and from spaces on the same drive? Still slow. The drive is doing both things at the same time...read-write.

The only way you may see the advertised uber numbers is in benchmarks, or maybe copying between two drives of the same type, and only if copying a single large file of sequential data.
 
Solution

tek3195

Prominent
Feb 7, 2021
67
4
545
Performance depends on the slowest device in the chain.

HDD -> SSD = speed of the HDD.

Copying to and from spaces on the same drive? Still slow. The drive is doing both things at the same time...read-write.

The only way you may see the advertised uber numbers is in benchmarks, or maybe copying between two drives of the same type, and only if copying a single large file of sequential data.

It's not as slow as I was thinking. I believe advertised is 'up to' 2300 MB/s for the 500GB drive, I can live with 2.1 GB/s real world. That's very near advertised.
I believe I went big enough on size to have accurate result.
Code:
tek3195@desktop:/opt$ sudo dd if=/dev/zero of=/work/tmp/test1.img bs=5G count=5 iflag=fullblock
5+0 records in
5+0 records out
26843545600 bytes (27 GB, 25 GiB) copied, 12.6961 s, 2.1 GB/s
tek3195@desktop:/opt$
 

tek3195

Prominent
Feb 7, 2021
67
4
545
Best answer unselected. It is against forum guidelines to select your own answer as the solution. Thanks.

I was wondering about marking it that way, now I know. But, in all fairness, I did answer my question with desired results. The answers I was getting made it sound like one couldn't test write speed on active drive. Turns out it's actually easy to do and get real results, nothing theoretical about them. I do appreciate the responses and maybe someone else can mark an answer for me.
 
The person who starts the thread can never have one of their answers be listed as the BA. It would simply break our rules against gaming the system in a way that would circumvent the rule to the point of being unenforceable.

Regardless, good you got it sorted.
 
The answers I was getting made it sound like one couldn't test write speed on active drive.
I don't believe anybody ever said, or insinuated that here at all. I see nothing even remotely similar to that. What was said was you can do an accurate test of the write speak to a drive that is also reading that information. Sure, it can be done, but it's not going to be anywhere near "theoretical" (AND, "theoretical" being "the specs the manufacturer supplied on paper for that drives performance, which are technically theoretical because nobody EVER achieves those actual speeds except perhaps on a machine designed specifically for testing drive speed which doesn't have to deal with things like SATA or PCIe buses, congested or bloated operating systems, background processes, etc.) when it comes to what is actually, realistically attainable.

If the drive is trying to read the information, it's got to stop in order to write the information, thus, the performance can never be as good as when it is being read from one drive while being written to another drive with similar speed capabilities.
 

tek3195

Prominent
Feb 7, 2021
67
4
545
I don't believe anybody ever said, or insinuated that here at all. I see nothing even remotely similar to that. What was said was you can do an accurate test of the write speak to a drive that is also reading that information. Sure, it can be done, but it's not going to be anywhere near "theoretical" (AND, "theoretical" being "the specs the manufacturer supplied on paper for that drives performance, which are technically theoretical because nobody EVER achieves those actual speeds except perhaps on a machine designed specifically for testing drive speed which doesn't have to deal with things like SATA or PCIe buses, congested or bloated operating systems, background processes, etc.) when it comes to what is actually, realistically attainable.

If the drive is trying to read the information, it's got to stop in order to write the information, thus, the performance can never be as good as when it is being read from one drive while being written to another drive with similar speed capabilities.
I'm fine with it and I guess maybe if it didn't sound that way, it felt that way in frustration trying to sort it out. I thought and still think that 2.1 GB/s is close or near the advertised 2300 MB/s, which I don't know where the line is between close, near and nowhere near.