[SOLVED] Replace SATA SSDs with NVME on my Lenovo E32 Tower PC ?

[KpuCko]

Hi there,
I have Lenovo E32 Tower PC, on my home lab. I run Proxmox on it and play with different Open-source technologies. In summary I have more than 15+ (VMs+Containers) which does a lot of random I/O. That's why I decided to replace my old SSD drives with new PCIe disks.

As you may know, there a plenty of choices on the market, when we speak about SSD/NVMe. Some of the disks comes with M2, others with U2, there are also different sizes, the PCI bus has its speed, for instance I have PCIe 3.0.
So can you help me to choose the right so called "converter" card and the right NVMe disk for my setup.

Here is the main-board: https://www.amazon.ca/Genuine-Lenovo-Thinkstation-Motherboard-03T6749/dp/B01NCSD38V

I might be wrong, but I think this guy can do the job: https://www.sabrent.com/product/EC-PCIE/nvme-m-2-ssd-to-pcie-x16-x8-x4-card-with-aluminum-heat-sink/

With such disk: https://www.sabrent.com/product/SB-...ernal-ssd-high-performance-solid-state-drive/

Thanks in advance.

 

[Bob.B]

Hi there,
I have Lenovo E32 Tower PC, on my home lab. I run Proxmox on it and play with different Open-source technologies. In summary I have more than 15+ (VMs+Containers) which does a lot of random I/O. That's why I decided to replace my old SSD drives with new PCIe disks.

As you may know, there a plenty of choices on the market, when we speak about SSD/NVMe. Some of the disks comes with M2, others with U2, there are also different sizes, the PCI bus has its speed, for instance I have PCIe 3.0.
So can you help me to choose the right so called "converter" card and the right NVMe disk for my setup.

Here is the main-board: https://www.amazon.ca/Genuine-Lenovo-Thinkstation-Motherboard-03T6749/dp/B01NCSD38V

I might be wrong, but I think this guy can do the job: https://www.sabrent.com/product/EC-PCIE/nvme-m-2-ssd-to-pcie-x16-x8-x4-card-with-aluminum-heat-sink/

With such disk: https://www.sabrent.com/product/SB-...ernal-ssd-high-performance-solid-state-drive/

Thanks in advance.

What storage is in the machine now?

Is there a gpu installed?
If so which pcie slot is it using?

 

[KpuCko]

I have 4 SSD disks connected directly on the mainboard via SATA cables.

No GPU card, all PCI/E slots are empty.

 

[SkyNetRising]

I have 4 SSD disks connected directly on the mainboard via SATA cables.

What's the capacity of those 4 SATA drives?
Will one 256GB M.2 drive be enough to replace them all?

Note - it's highly unlikely for your system to be able to boot from NVME device. BIOS support is required for that.

 

[KpuCko]

I don't plan to boot from NVMe at all.

I can leave two SSD disks as system boot drives, and replace the other two of them with 1 NVMe 512G size.

 

[Bob.B]

I don't plan to boot from NVMe at all.

I can leave two SSD disks as system boot drives, and replace the other two of them with 1 NVMe 512G size.

What do you hope to gain out of this disk swap?
If it's speed you might be disappointed.
Run this and post a LINK to the results page let's take a peek under the hood.
PC Benchmark

 

[KpuCko]

Ok, let me share with you some results I made in the past related to performance checks:

Samsung EVO 860 SSD - With cache:

root@proxmox-node-1.home.lan:~# dd if=/dev/zero of=/tmp/output bs=8k count=10k; rm -f /tmp/output
10240+0 records in
10240+0 records out
83886080 bytes (84 MB, 80 MiB) copied, 0.0474977 s, 1.8 GB/s

Samsung EVO 860 SSD - Without cache:

root@proxmox-node-1.home.lan:~# dd if=/dev/zero of=/tmp/output conv=fdatasync bs=8k count=10k; rm -f /tmp/output
10240+0 records in
10240+0 records out
83886080 bytes (84 MB, 80 MiB) copied, 0.300222 s, 279 MB/s
root@proxmox-node-1.home.lan:~# dd if=/dev/zero of=/tmp/output conv=fdatasync bs=8k count=10k; rm -f /tmp/output
10240+0 records in
10240+0 records out
83886080 bytes (84 MB, 80 MiB) copied, 0.28048 s, 299 MB/s
root@proxmox-node-1.home.lan:~# dd if=/dev/zero of=/tmp/output conv=fdatasync bs=8k count=10k; rm -f /tmp/output
10240+0 records in
10240+0 records out
83886080 bytes (84 MB, 80 MiB) copied, 0.280084 s, 300 MB/s
root@proxmox-node-1.home.lan:~#

Sequential READ with FIO - Samsung EVO 860 SSD

root@proxmox-node-1.home.lan:~# fio --name TEST-Sequential-READ --eta-newline=5s --filename=/tmp/fio-tempfile.dat --rw=read --size=500m --io_size=10g --blocksize=1024k --ioengine=libaio --fsync=10000 --iodepth=32 --direct=1 --numjobs=1 --runtime=60 --group_reporting
TEST-Sequential-READ: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=32
fio-3.12
Starting 1 process
TEST-Sequential-READ: Laying out IO file (1 file / 500MiB)
Jobs: 1 (f=1): [R(1)][66.7%][r=899MiB/s][r=898 IOPS][eta 00m:04s]
Jobs: 1 (f=1): [R(1)][100.0%][r=896MiB/s][r=896 IOPS][eta 00m:00s]
TEST-Sequential-READ: (groupid=0, jobs=1): err= 0: pid=18846: Fri Jun 26 10:12:49 2020
  read: IOPS=902, BW=902MiB/s (946MB/s)(10.0GiB/11348msec)

Sequential WRITE with FIO - Samsung EVO 860 SSD

root@proxmox-node-1.home.lan:~# fio --name TEST-Sequential-WRITE --eta-newline=5s --filename=/tmp/fio-tempfile.dat --rw=write --size=500m --io_size=10g --blocksize=1024k --ioengine=libaio --fsync=10000 --iodepth=32 --direct=1 --numjobs=1 --runtime=60 --group_reporting
TEST-Sequential-WRITE: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=32
fio-3.12
Starting 1 process
TEST-Sequential-WRITE: Laying out IO file (1 file / 500MiB)
Jobs: 1 (f=1): [W(1)][25.0%][w=380MiB/s][w=380 IOPS][eta 00m:21s]
Jobs: 1 (f=1): [W(1)][46.4%][w=380MiB/s][w=380 IOPS][eta 00m:15s]
Jobs: 1 (f=1): [W(1)][70.4%][w=384MiB/s][w=384 IOPS][eta 00m:08s]
Jobs: 1 (f=1): [W(1)][92.6%][w=364MiB/s][w=364 IOPS][eta 00m:02s]
Jobs: 1 (f=1): [W(1)][100.0%][w=359MiB/s][w=359 IOPS][eta 00m:00s]
TEST-Sequential-WRITE: (groupid=0, jobs=1): err= 0: pid=20147: Fri Jun 26 10:17:41 2020
  write: IOPS=369, BW=369MiB/s (387MB/s)(10.0GiB/27726msec); 0 zone resets

Random READ with FIO - Samsung EVO 860 SSD

root@proxmox-node-1.home.lan:~# fio --name TEST-Random4k-READ --eta-newline=5s --filename=/tmp/fio-tempfile.dat --rw=randread --size=500m --io_size=10g --blocksize=4k --ioengine=libaio --fsync=1 --iodepth=1 --direct=1 --numjobs=1 --runtime=60 --group_reporting
TEST-Random4k-READ: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
fio-3.12
Starting 1 process
TEST-Random4k-READ: Laying out IO file (1 file / 500MiB)
Jobs: 1 (f=1): [r(1)][13.1%][r=38.7MiB/s][r=9894 IOPS][eta 00m:53s]
Jobs: 1 (f=1): [r(1)][23.0%][r=37.3MiB/s][r=9546 IOPS][eta 00m:47s]
Jobs: 1 (f=1): [r(1)][32.8%][r=38.6MiB/s][r=9879 IOPS][eta 00m:41s]
Jobs: 1 (f=1): [r(1)][42.6%][r=38.6MiB/s][r=9880 IOPS][eta 00m:35s]
Jobs: 1 (f=1): [r(1)][52.5%][r=37.9MiB/s][r=9692 IOPS][eta 00m:29s]
Jobs: 1 (f=1): [r(1)][62.3%][r=38.6MiB/s][r=9883 IOPS][eta 00m:23s]
Jobs: 1 (f=1): [r(1)][72.1%][r=37.0MiB/s][r=9718 IOPS][eta 00m:17s]
Jobs: 1 (f=1): [r(1)][82.0%][r=38.6MiB/s][r=9875 IOPS][eta 00m:11s]
Jobs: 1 (f=1): [r(1)][91.8%][r=37.3MiB/s][r=9552 IOPS][eta 00m:05s]
Jobs: 1 (f=1): [r(1)][100.0%][r=38.6MiB/s][r=9879 IOPS][eta 00m:00s]
TEST-Random4k-READ: (groupid=0, jobs=1): err= 0: pid=20584: Fri Jun 26 10:19:58 2020
  read: IOPS=9738, BW=38.0MiB/s (39.9MB/s)(2282MiB/60001msec)

Random WRITE with FIO - Samsung EVO 860 SSD

root@proxmox-node-1.home.lan:~# fio --name TEST-Random4k-WRITE-SYNC --eta-newline=5s --filename=/tmp/fio-tempfile.dat --rw=randrw --size=500m --io_size=10g --blocksize=4k --ioengine=libaio --fsync=1 --iodepth=1 --direct=1 --numjobs=1 --runtime=60 --group_reporting
TEST-Random4k-WRITE-SYNC: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
fio-3.12
Starting 1 process
TEST-Random4k-WRITE-SYNC: Laying out IO file (1 file / 500MiB)
Jobs: 1 (f=1): [m(1)][11.7%][r=508KiB/s,w=468KiB/s][r=127,w=117 IOPS][eta 00m:53s]
Jobs: 1 (f=1): [m(1)][21.7%][r=448KiB/s,w=492KiB/s][r=112,w=123 IOPS][eta 00m:47s]
Jobs: 1 (f=1): [m(1)][31.7%][r=468KiB/s,w=468KiB/s][r=117,w=117 IOPS][eta 00m:41s]
Jobs: 1 (f=1): [m(1)][41.7%][r=500KiB/s,w=436KiB/s][r=125,w=109 IOPS][eta 00m:35s]
Jobs: 1 (f=1): [m(1)][51.7%][r=460KiB/s,w=464KiB/s][r=115,w=116 IOPS][eta 00m:29s]
Jobs: 1 (f=1): [m(1)][61.7%][r=572KiB/s,w=468KiB/s][r=143,w=117 IOPS][eta 00m:23s]
Jobs: 1 (f=1): [m(1)][71.7%][r=452KiB/s,w=464KiB/s][r=113,w=116 IOPS][eta 00m:17s]
Jobs: 1 (f=1): [m(1)][81.7%][r=404KiB/s,w=508KiB/s][r=101,w=127 IOPS][eta 00m:11s]
Jobs: 1 (f=1): [m(1)][91.7%][r=472KiB/s,w=468KiB/s][r=118,w=117 IOPS][eta 00m:05s]
Jobs: 1 (f=1): [m(1)][100.0%][r=496KiB/s,w=496KiB/s][r=124,w=124 IOPS][eta 00m:00s]
TEST-Random4k-WRITE-SYNC: (groupid=0, jobs=1): err= 0: pid=21543: Fri Jun 26 10:23:30 2020
  read: IOPS=118, BW=474KiB/s (486kB/s)(27.8MiB/60005msec)

More about FIO here

 

[KpuCko]

From the tests above I consider that only RANDOM reads/writes are quite slow, and this is completely normal, because during the tests CPU load is almost 100% usage.

The block size also matters, not sure for those SSDs the block size is 4k (filesystem)