I have some questions regrading the PCIe Root Complex, PCIe lanes and the PCH and I am hoping someone here will be able to help as I can't find a clear answer online.
I understand that the PCIe Root Complex is the Host Bridge in the CPU that connects the CPU and memory to the PCIe architecture. The 'inner' bus of the root complex is bus 0. The Root Complex contains Root Ports to connect to the PCIe devices.
Firstly, the PCH (southbridge) contains Root Ports and devices that are on bus 0, so is the PCH part of the Root Complex?
If so, what about the devices internal to the PCH (USB Cont., SATA Cont. etc.). They are not behind any Root Ports and are on Bus 0 so would they be part of the Root Complex.
My next question is regarding the PCIe lanes originating from the PCH, my Z87 chipset has 8 PCIe lanes from the PCH. Do these lanes then 'use up' some of the lanes from the CPU, for example if I have a 28 PCIe lane CPU, will 8 of those lanes connect to the 8 lanes in the PCH?
Again, what about the devices internal to the PCH, do these use up any PCIe lanes?
I know there not easy questions to answer but if someone has any information it would be greatly appreciated.
The PCIe root complex is to the PCIe architecture as the PCI host bridge is to the PCI architecture. The root complex (PCIe) and host bridge (PCI) provide a stateful translation layer between the PCIe/PCI logic on one side, and the system specific logic on the other. This allows a PCIe/PCI device to connect to any system that has a compliant root complex / host bridge without regard to the architecture of the rest of the system. The PCI devices and PCIe endpoints need not care about the specifics of the system's memory subsystem, endianness, etc...
Intel uses an architecture that they call Integrated IO (IIO) to merge all of the previous platform components into one physical part of the CPU. Although the components are now integrated, they are still [at least logically] connected via PCI and are exposed as such to the PC BIOS / UEFI firmware.
Bus 0 joins the DMI root complex, PCIe root complexes (between one and three depending on the chip), DMA engine, and IIO Core to the processor cores. Collectively this forms the Processor IIO Devices. Similarly, Bus 1 joins the QPI links, interrupt handling, core broadcast, power control, IMC, and performance monitoring to the processor cores. Collectively this forms the Processor Uncore devices.
Since the DMI root port (not complex) and PCIe root port(s) bridge a PCIe layer to a PCI layer they are logically just PCI-to-PCI brides. The physical DMI2 connection which joins the PCH to the CPU is invisible to configuration software. By virtue of this, the PCH and the devices connected to it are just a logical extension of the CPU's internal PCI bus 0.
Note that CPUBUSNO 0 and CPUBUSNO 1 are just symbolic names and are processor relative. In multi-socket systems the actual bus number assigned to it may vary.
Intel has been somewhat vague about how all of the PCIe root ports and PCI devices communicate with the system memory via a root complex or host bridge.
As for the lanes, the 8x lanes originating from the PCH are in the form of an 8x port which can be subdivided all the way down to 8 1x ports which enables a large number of low-bandwidth peripherals to be connected at once. This is unlike the CPU which is a 16x port that can only be subdivided three ways (16/0/0, 8/8/0, 8/4/4 with each able to be downnegotiated independently). Logically, these devices all share the same internal bus.
The devices internal to the PCH do not use these lanes as PCIe is primarily used as an external point-to-point bus and has no advantage over PCI when used internally (in fact I would suspect that using PCIe internally would be quite a bit more difficult). However, motherboard manufacturers often utilize some of these lanes to connect additional peripherals such as NICs, Bluetooth, Audio codecs, and additional storage controllers. It's common for four of the lanes to be exposed in the form of a single 4x PCIe slot and three 1x PCIe slots with the other four lanes used for onboard peripherals.[/quotemsg]
Hi Pinhedd, thanks for the reply (I remember speaking to you about this earlier). I would have replied sooner but the forums were down...
Right so the PCIe Root Complex can be seen as the PCI Host Bridge between system logic and the PCIe hierarchy. The 'PCI Express system architecture' book says that bus 0 is the internal bus of the PCIe Root Complex, as PCI bus 0 crosses the DMI and into the PCH would the PCH be classed as part of the Root Complex, or just an extension if it?
I'm not aware of CPUBUSNO 0 and CPUBUSNO 1, I assume these don't have any relation to PCI bus numbers, as the PCI hierarchy will be created 'behind' the PCIe Root Complex/s?
I had assumed that the devices internal to the PCH didn't use PCIe lanes as they are not behind Root Ports, unlike the NIC which is. I had read this online in an article but it must be wrong.
As for the PCIe lanes from the the PCH, do the lanes have anything to do with the lanes in the CPU, for example do the 8 lanes in the PCH 'use up' 8 lanes of those in the CPU.
I understand the PCIe is packet based and that the Root Complex will generate a packet for the CPU. If the CPU wants to write to a device in the PCH, such as a USB or SATA Controller, what would the process be (on a very basic level)?
The the PCIe Root Complex would read the memory address and bridge it onto bus 0 as a packet, from there would the DMI Root Complex intercept the packet and send it across the DMI and then then the internal logic of the PCH would send it to the right device, something like that?
Would a PCIe packet be created for an internal device on bus 0 such as the USB Controllers? what about to a device that's on a PCIe lane/s from the PC, where would the packet be created?
As you can see I'm quite out of my depth with this but I would just like to get a clearer understanding.
Thanks again for your reply, it really is appreciated.