[SOLVED] Upgrading PCIe

[mujmuj]

I learned from another thread I posted that PCIe4 is better than PCIe3 for performing statistical and mathematical analyses using big data.

Currently everything I have is PCIe3 components.

Due to compatibility issue, I need to upgrade all the main components (motherboard, SSD, CPU) to PCIe-compatible models. (right?)

Will PCIe5 components be available for motherboard, SSD, CPU anytime soon? In that case, perhaps it makes sense to wait a bit and buy them.

But if PCIe5 components will be out 1 or more years later, then switching everything to PCIe4-compatible models now makes sense.

 

[Eximo]

You get more bandwidth, but that only works if you can use that bandwidth. As you mentioned 100GB data sets. PCIe 4.0 SSDs would be the biggest benefit.

Intel's latest supports a single PCIe 5.0 x16 slot or dual 8x. There are currently no consumer available PCIe 5.0 cards.

I would take a look at what hardware you have now vs what you propose to upgrade to see how it would benefit you in the now.

 

[Eximo]

There is an entire enterprise world of hardware where PCIe 5.0 hardware may be available soon, but you are looking at extreme costs compared to consumer hardware.

 

[Eximo]

I haven't seen any block diagrams for any potential HEDT options from Intel. Alder Lake X will be the big brother of Intel 12th gen, it would make sense for them to add lots of PCIe 5.0 lanes, but only the name has shown up so far, no details.

AMD's latest Threadripper Pro support huge amounts of PCIe 4.0 lanes, but they are OEM only, so you would have to buy from Dell, HP, etc to get one.

 

[mujmuj]

You get more bandwidth, but that only works if you can use that bandwidth. As you mentioned 100GB data sets. PCIe 4.0 SSDs would be the biggest benefit.

Intel's latest supports a single PCIe 5.0 x16 slot or dual 8x. There are currently no consumer available PCIe 5.0 cards.

I would take a look at what hardware you have now vs what you propose to upgrade to see how it would benefit you in the now.

"Intel's latest supports a single PCIe 5.0 x16 slot or dual 8x. "

You mean CPU?


"There are currently no consumer available PCIe 5.0 cards."

What does "cards" mean here? Graphic card? I probably don't need a graphic card at all because I won't play games or design things. I guess a graphic card is not necessary for running statistical computing on big data?

 

[hotaru.hino]

I learned from another thread I posted that PCIe4 is better than PCIe3 for performing statistical and mathematical analyses using big data.

Having a look at your other thread and if you're working on data sets that large, while it may be beneficial to have higher bandwidth, you also have to consider how much data the processor can crunch in a second.

For example, when I do Handbrake runs (processing video files, basically), the best my processor can average is around 40-50FPS. So let's call it 2x real time compared to the actual file. Considering a standard 1080p Blu-Ray movie rip is about 40Mbps and the output hovers around a quarter of that (so 80Mbps + 20Mbps = 100Mbps total, or about 12.5 megabytes a second) , I could do all the processing I want from a hard drive because it has sufficient bandwidth to keep up .

If you don't have the processing chops such that processing the data is purely memory bound (data still has to reside in RAM before it can be processed), then it's a waste of time to consider the storage bandwidth requirements because it's not going to be a bottleneck.

 

[mujmuj]

There is an entire enterprise world of hardware where PCIe 5.0 hardware may be available soon, but you are looking at extreme costs compared to consumer hardware.

Yes I am not that rich enough to buy enterprise version.

How do prices of SSD, CPU, motherboard usually change over time when new PCIe level is updated? If they go down quickly over time, then maybe it makes sense to wait more.

 

[Eximo]

"Intel's latest supports a single PCIe 5.0 x16 slot or dual 8x. "

You mean CPU?


"There are currently no consumer available PCIe 5.0 cards."

What does "cards" mean here? Graphic card? I probably don't need a graphic card at all because I won't play games or design things. I guess a graphic card is not necessary for running statistical computing on big data?

Yes. 12th Gen Intel CPUs support PCIe 5.0, but only on the x16 slots. Their M.2 storage slots are still PCIe 4.0.

Pricing for storage is mostly about performance/capacity. New PCIe 4.0 drives are as much as 50% more expensive than PCIe 3.0 drives. Aside from sales on lower end hardware, the price doesn't really fall that much. They just drop off the market when newer options are available.

Well, you can plug storage drives into x16 adapters for example. So get 4 x 4 PCIe 4.0 NVMe drives on a card and slot that in. There are no PCIe 5.0 cards or PCIe 5.0 drives to plug into them at the moment.

 

[Eximo]

Not a recommendation, but things like this exist:

https://www.amazon.com/Adapter-4x32Gbps-Individual-Indicator-Support/dp/B09BJ163KY?th=1

There are plenty of motherboards that offer 3 or more M.2 slots on their own.

The key here is that you would be going to directly to the CPU, in most instances, vs through the chipset, which would be limited by the connection between the CPU and chipset. 4x PCIe 4.0 in the case of Intel Z690.

 

[Eximo]

What does "cards" mean here? Graphic card? I probably don't need a graphic card at all because I won't play games or design things. I guess a graphic card is not necessary for running statistical computing on big data?

Well, only if you are using the CPU for calculations. There are many situations where using a graphics card makes more sense. Look into CUDA or OpenCL coding, Deep Learning/AI.

 

[mujmuj]

Having a look at your other thread and if you're working on data sets that large, while it may be beneficial to have higher bandwidth, you also have to consider how much data the processor can crunch in a second.

For example, when I do Handbrake runs (processing video files, basically), the best my processor can average is around 40-50FPS. So let's call it 2x real time compared to the actual file. Considering a standard 1080p Blu-Ray movie rip is about 40Mbps and the output hovers around a quarter of that (so 80Mbps + 20Mbps = 100Mbps total, or about 12.5 megabytes a second) , I could do all the processing I want from a hard drive because it has sufficient bandwidth to keep up .

If you don't have the processing chops such that processing the data is purely memory bound (data still has to reside in RAM before it can be processed), then it's a waste of time to consider the storage bandwidth requirements because it's not going to be a bottleneck.

The statistical package that I'd use is Stata. And on its "hardware requirement" info section, it says

"Stata loads all of your data into RAM to perform its calculations. You must have enough physical RAM to load Stata and allocate enough memory to it to load and analyze your datasets."

Does this mean higher bandwidth does not matter to speed up Stata data processing?

 

[mujmuj]

"but only on the x16 slots" What does this mean? Does "slot" here mean RAM?

 

[Eximo]

Hmm, looks like a CPU only tool.

If you have a 100GB data set, does that mean you have that much memory? That would seem to be the most important factor for speed. Given the size it should also be ECC memory so that errors are corrected as they happen.

Faster storage will just decrease the amount of time it takes to start the work. How quickly the data set can be loaded.

It sounds like you want faster storage because you don't have enough memory and it is using virtual memory on the disk to do the calculations. In which case there would be a benefit, but the better solution would be more RAM.

No, memory slots are separate from PCIe slots.

PCIe lanes are a fixed measurement of bandwidth. For each generation of PCIe, this speed is basically doubled. PCIe 1.0 x16 = PCIe 2.0 x8 = PCIe 3.0 4x = PCIe 4.0 2x = PCIe 5.0 1x.

So PCIe 3.0 x4 (approx 4GB/s) is half the speed of PCIe 4.0 x4 (Approx 8GB/s). Your fastest SSDs are around 7GB/s

PCIe 1x, 2x, 4x, 8x, 16x. Typically there are two lengths of slots on standard motherboards. 16x slots for full length cards (like graphics cards). And 1x slots for expansion, like 1Gbps ethernet cards, sound cards, WiFi cards, USB cards, etc. 2x, 4x and 8x cards are generally plugged into 16x slots. Not all 16x slots can operate at that speed. The most common configuration for a motherboard is two x16 slots connected directly to the CPU. The second slot is usually only wired for 8x. When in use these 8x lanes are taken from the primary x16 slot. Giving you two 8x slots effectively.

 

[Eximo]

I would probably just pick up something like this:

https://www.newegg.com/dell-precision-t5810/p/1VK-0001-5KJB9?quicklink=true

Has a decently fast quad core processor in it, but room for improvement. But the important thing here is 128GB of DDR4 memory.


Add a PCIe to NVMe storage adapter and maybe stripe a pair of drives. These would be PCIe 3.0 drives, but doubling them up will get you in the ball park of a PCIe 4.0 drive. This would only be used as the working drive. The system should boot off the drive it comes with.


Optional 16 core CPU...

https://www.ebay.com/itm/2035156416...50rI/bYJgYuRUopGdDP/J+5xRXGn|tkp:BFBMyte02Ylg

 

[hotaru.hino]

The statistical package that I'd use is Stata. And on its "hardware requirement" info section, it says

"Stata loads all of your data into RAM to perform its calculations. You must have enough physical RAM to load Stata and allocate enough memory to it to load and analyze your datasets."

Does this mean higher bandwidth does not matter to speed up Stata data processing?

Considering this sounds like it preloads all of the data, all a faster storage speed would get you is a reduction of loading the data onto RAM. Assuming you have enough RAM to cover your data set (> 100 GB), you're looking at a load time of about 30 seconds for PCIe 3 speeds and 15 for PCIe 4 speeds. PCIe 5 would shave this down to 7.5, but unless your time's worth can be measured in thousands of dollars per hour, I don't think it's worth trying to spend money on faster storage.

Though that's just reading. Writing is a different story.

Either way at the end of the day, with that explanation from the software vendor, I don't think storage affects processing performance itself.

 

[mujmuj]

Considering this sounds like it preloads all of the data, all a faster storage speed would get you is a reduction of loading the data onto RAM. Assuming you have enough RAM to cover your data set (> 100 GB), you're looking at a load time of about 30 seconds for PCIe 3 speeds and 15 for PCIe 4 speeds. PCIe 5 would shave this down to 7.5, but unless your time's worth can be measured in thousands of dollars per hour, I don't think it's worth trying to spend money on faster storage.

Though that's just reading. Writing is a different story.

Either way at the end of the day, with that explanation from the software vendor, I don't think storage affects processing performance itself.

OK I learned SDD reading speed doesn't matter.

What about overall change from PCIe3 to PCIe4 or PCIe5? I mean changing motherboard, CPU, SDD to a higher PCIe compatibility while keeping other specs like read/write speed same. Will this help run Stata faster?

 

[USAFRet]

OK I learned SDD reading speed doesn't matter.

What about overall change from PCIe3 to PCIe4 or PCIe5? I mean changing motherboard, CPU, SDD to a higher PCIe compatibility while keeping other specs like read/write speed same. Will this help run Stata faster?

From reading this thread, it seems you software is centered on CPU+RAM.
Not drive speed.

PCIe 3/4/5 won't make a real difference, reading or writing.

 

[mujmuj]

Hmm, looks like a CPU only tool.

If you have a 100GB data set, does that mean you have that much memory? That would seem to be the most important factor for speed. Given the size it should also be ECC memory so that errors are corrected as they happen.

Faster storage will just decrease the amount of time it takes to start the work. How quickly the data set can be loaded.

It sounds like you want faster storage because you don't have enough memory and it is using virtual memory on the disk to do the calculations. In which case there would be a benefit, but the better solution would be more RAM.

No, memory slots are separate from PCIe slots.

PCIe lanes are a fixed measurement of bandwidth. For each generation of PCIe, this speed is basically doubled. PCIe 1.0 x16 = PCIe 2.0 x8 = PCIe 3.0 4x = PCIe 4.0 2x = PCIe 5.0 1x.

So PCIe 3.0 x4 (approx 4GB/s) is half the speed of PCIe 4.0 x4 (Approx 8GB/s). Your fastest SSDs are around 7GB/s

PCIe 1x, 2x, 4x, 8x, 16x. Typically there are two lengths of slots on standard motherboards. 16x slots for full length cards (like graphics cards). And 1x slots for expansion, like 1Gbps ethernet cards, sound cards, WiFi cards, USB cards, etc. 2x, 4x and 8x cards are generally plugged into 16x slots. Not all 16x slots can operate at that speed. The most common configuration for a motherboard is two x16 slots connected directly to the CPU. The second slot is usually only wired for 8x. When in use these 8x lanes are taken from the primary x16 slot. Giving you two 8x slots effectively.

I have ASUS Rampage Extreme Encore, which has a huge 256GB RAM motherboard, and enough RAMs to fill all the 256GB slots. But this one supports only PCIe3. But the 4TB m.2 SSD I am considering buying supports PCIe4. So I am wondering if upgrading motherboard and CPU to PCIe4-compatible thing would give me speed boost for running Stata (in light of the fact that Stata says "Stata loads all of your data into RAM to perform its calculations.")

 

[Eximo]

It would be a minor change. 3000MB/s top speed drive to like 7000MB/s top speed.

Only PCIe 4.0 capable alternative would be Threadripper at the moment. Not worth the investment to change over.

Waiting for DDR5 HEDT platforms would make more sense.

 

[mujmuj]

It would be a minor change. 3000MB/s top speed drive to like 7000MB/s top speed.

Only PCIe 4.0 capable alternative would be Threadripper at the moment. Not worth the investment to change over.

Waiting for DDR5 HEDT platforms would make more sense.

OK Thanks. What about graphic card? Does that help for running statistical programs using huge dataset?

I will not play games. I don't need nice graphics on my computer. But I read graphic cards help non-graphic jobs as well so maybe it helps statistical program also (especially if I am simultaneously watching youtube while running the statistical program)?

 

[Eximo]

Not significantly in this case. Your software seems to run on CPUs only.

Not to say it couldn't be programmed to use a GPU, but someone would have to do that. It is interesting to note they programmed it for RISC CPUs and x86, so there is the possibility that the source code is around somewhere.

 

[mujmuj]

Not significantly in this case. Your software seems to run on CPUs only.

Not to say it couldn't be programmed to use a GPU, but someone would have to do that. It is interesting to note they programmed it for RISC CPUs and x86, so there is the possibility that the source code is around somewhere.

" It is interesting to note they programmed it for RISC CPUs and x86 "

Don't they actually say CISC is faster in general? Stata hardware requirement

"There are two main types of processors, RISC (reduced instruction set) and CISC (complex instruction set). In general, a CISC chip running at the same clock speed (such as 2 GHz) as a RISC chip will accomplish more in a given amount of time. RISC chips are fast at certain types of operations, such as integer math, which makes them faster for certain applications like graphics packages. However, for general computing, a CISC chip will be faster at a given clock speed. Intel and AMD are the main producers of CISC chips, whereas Sun SPARC and some other Unix computers use RISC chips. "

 

[Eximo]

They say that, yes. It is a very broad generalization. RISC is all about efficiency. But my point was more that they compiled it to run on different platforms, so that means the capability to do so exists.

I would be more curious how it utilizes the typical RISC based server with hundreds of CPU cores, if that could be translated to the multiple thousand 'cores' on a GPU it might be worth it. Of course, someone would need to do that.

 

[hotaru.hino]

" It is interesting to note they programmed it for RISC CPUs and x86 "

Don't they actually say CISC is faster in general? Stata hardware requirement

"There are two main types of processors, RISC (reduced instruction set) and CISC (complex instruction set). In general, a CISC chip running at the same clock speed (such as 2 GHz) as a RISC chip will accomplish more in a given amount of time. RISC chips are fast at certain types of operations, such as integer math, which makes them faster for certain applications like graphics packages. However, for general computing, a CISC chip will be faster at a given clock speed. Intel and AMD are the main producers of CISC chips, whereas Sun SPARC and some other Unix computers use RISC chips. "

I think that generalizing CISC vs RISC this way is misleading. It depends entirely on how the processor was implemented. In the early days, RISC was faster than CISC. However, CISC processors now have ways to process instructions faster and in fact, pretty much every implementation of x86 now uses a RISC-like execution core. The front-end can be considered another processor to handle the best way to execute these instructions.

As another blow to the generalization, Apple's CPUs are just as performant as AMD's or Intel's processors, despite being entirely RISC based.