31

As part of integration testing for our product, we received a client machine with a Supermicro X10DRi motherboard in it, with 2 Xeon E5-2620 v4 CPUs and a Quadro P5000. That motherboard is using the Intel C612 chipset.

While doing basic research for the integration, I noticed that the motherboard specs say it has 3X PCIe 3.0 x16 and 3X PCIe 3.0 x8 slots. However, the C612 chipset apparently only supports PCIe 2.0, and only up to x4 connections.

While the machine does work, I would like to know if we are taking full advantage of the Quadro. I currently believe that we are limited by the maximum speed of the chipset, as it controls the speed of communication between the CPUs and the GPU.

  1. Am I correct in that belief?
  2. Is there something I am missing/unaware of/incorrect about relating to the communication between the CPU and GPU, whereby the chipset is not a limiting factor?
  3. Is there a way to test the speed of the connection to the GPU?
Improve this question

1 Answer 1

Reset to default
45

Your CPU has 40 PCIe 3.0 lanes: Intel® Xeon® Processor E5-2620 v4

enter image description here

The communications with your GPUs will be direct from your CPU. This is good, after all the memory controller is also integrated on the CPU meaning that the fastest route for data in memory is from RAM -> CPU -> GPU rather than RAM - CPU -> Chipset -> GPU

The motherboard connects the PCIe connections from the CPU to GPU, but that does not mean that they need to pass through the chipset. They go direct from the CPU to the CPU served PCIe slots.

Somewhere in your motherboard documentation it should tell you if any of the PCIe slots are served by chipset or CPU. Chances are the chipset PCIe lanes are used to serve an m.2 slot or SATA controller. Possibly even USB3 or Gigabit Ethernet. How it is wired up and what to is down to the motherboard designer.

Found your effective chipset/CPU/GPU layout on the Intel page for that chipset:

enter image description here

Improve this answer
5
  • 1
    I understand the CPU can support more lanes, but does not the chipset actually define those lanes? In other words, do the lanes go from the CPUs, through the chipset, and then on to the GPU?
    – William Handrigan
    Commented Feb 7, 2019 at 21:56
  • 3
    I've added details. The motherboard holds the PCIe connections from CPU to GPU, but they do not need to pass through the chipset. They are direct from the CPU to the CPU served PCIe slots. Somewhere in your motherboard documentation it should tell you if any of the PCIe slots are served by chipset or CPU. Chances are the chipset PCIe lanes are used to serve an m.2 slot or SATA controller.
    – Mokubai ♦
    Commented Feb 7, 2019 at 21:58
  • 4
    Perfect. Thanks for your concise, timely response!
    – William Handrigan
    Commented Feb 7, 2019 at 22:01
  • 9
    @WilliamHandrigan I found the Intel technical page that shows the effective layout.
    – Mokubai ♦
    Commented Feb 7, 2019 at 22:19
  • 1
    The X10DRi manual includes a block diagram on page 1-10 which shows how the lanes are split. Slot 2 gets 16 lanes to CPU 1, slots 1 and 3 get 8 lanes each to CPU 1, slots 4 and 6 get 16 lanes each to CPU 2, slot 5 gets 8 lanes to CPU 2. The PCIe 2.0 lanes from the chipset are used for the BMC. The X540/i350 LAN chipsets and the C612 are connected to CPU 1.
    – Stephen Kitt
    Commented Feb 8, 2019 at 19:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .