0

I tried to acquire 10GBit/s of UDP traffic most of the time lossless (the measurement device is not capable to hold so much data in the own cache, so TCP is not a solution here).

To test if I can acquire 10GBit/s of UDP traffic I bought 2 Intel X550-T2 NIC's and connected them directly over an 2 Meter Cat7 Cable. But I couldn't bring the receiver side to use more than one CPU to acquire the data over UDP. Using TCP strangely uses all 8 configured RSS queues also I get 10 GBit/s of traffic.
Using Ubuntu Linux 20.04 works also very well (10GBit/s UDP traffic 0% loss).

My question is now, how can I use the RSS also for UDP on Windows 10? And if it does not work, why?

Receiver hardware:
Processor: AMD Ryzen 3900X
Motherboard: X570a-pro
RAM: 64 GB

Configuration:
Windows 10 Professional
Hyperthreading / SMT: Disabled (I don't know if this is required, but Microsoft states, that RCC only uses non SMT processors)
DMA-Coalescing: OFF
Receivebuffer: 4096
Flow control: Rx+Tx
Speed and Duplex: Auto
Interrupt reduction: Enabled
Interrupt reduction rate: Adaptive
IPsec Offload: OFF
IPv4 - checksum offload: Rx + Tx
Jumbo packets: 9014
Large-Send-Offload V2 (IPv4+v6): Enabled
Max. Rss Queues: 8 (I tested also 16, but that changed nothing)
Package priority and VLAN: Deactivated both (activating both changed nothing)
RSS: Enabled
TCP - Checksum offload (IPv4+v6): Rx + Tx
UDP - Checksum offload (IPv4+v6): Rx + Tx
Sending buffer: 16384
Connection event log: Enabled

Method of testing:
Since RSS is based on hashing of S/R IP + S/R Port, I started 10 receiving iperf3 servers on the receiver, each listening on different ports and started 10 sending clients on the sender.
Sending UDP packages occupies one whole CPU/core of the receiver, bot non of the other cores. And only 5GBit/s could be acquired. Sending TCP packages occupies all the specified 8 receiving queues and 10Gbit's could be acquired.

Improve this question
5
  • Your question is very vague and lacking details and proper troubleshooting information so it is largely speculation. First try standardized testing software as a comparison, like iperf and if you suspect the OS try different one like Linux and compare your results. You have too many variables and not enough testing scenarios to rule out even a single variable. This could be anything from a poor quality patch cord to a bug in the OS itself, but there is nothing here to determine it.
    – acejavelin
    Commented Sep 5, 2020 at 14:45
  • Just done this. One run with ubuntu20.04 and 2 iperf3 instances on 2 ports: nearly 10Gbit/s throughput. Also nearly no pakage drops. Running the same scenario on windows: 4-6 Gbit/s and 50% drops.
    – Fabian Keßler
    Commented Sep 5, 2020 at 14:58
  • And this is why as a network engineer when testing network gear for a large scale enterprise, carriers, or other very high capacity/speed scenario, we don't use Windows ...
    – acejavelin
    Commented Sep 5, 2020 at 15:25
  • Doesn't Windows use some weird receive window scaling routine thats both awfully tuned and primarily aimed at home/desktop use? Jumbo packets usually aren't enabled by default as well if I remember correctly...
    – Mokubai ♦
    Commented Sep 5, 2020 at 16:49
  • It's true, that windows does not use Jumbopackages by default, but the sender is responsible for that. With linux and the default 1500MTU I get the throughput of 10GBit/s. Interrestingly with TCP instead of UDP it works also on windows. Seems like Windows / iperf3 does not use RSS for UDP.
    – Fabian Keßler
    Commented Sep 7, 2020 at 12:22

1 Answer 1

Reset to default
0

We found out, that this specific NIC does not send UDP Packages with the specified size of 9014 Bytes (receiving is not a problem). As soon as the UDP Package size exceeds the MTU (in this case 1500) it will fragment all packages into several IPv4 fragmented packages. Since RSS Queues are splitting their payload over protocol specific hashing algorithms, the NIC will use the hashing algorithm of the IP-Protocol, which does not take the Port into account.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .