NIC's and Hardware Acceleration

[mattlach]

Hey everyone,

So I have read this page in the docs, and have been considered trying to enable hardware offloading to see if it helps with some of my high CPU usage.

The goal here is to reduce some of the high CPU usage I have seen during high bandwidth (~gigabit) use over WireGuard.   I am not sure how much of that load is encryption vs. packet handling / interrupts and how much potential there is to help things by enabling NIC hardware offload.



Any hints, tips and tricks before I do?

The docs seem to warn of misbehaving hardware.  Is there a list anywhere of which hardware people have used with hardware offload successfully?

I use an Intel i210 for WAN, and an Intel x520 for LAN.    I understand older enterprise grade Intel NIC's are some of the most well behaved in Unix and Linux, but should these not be a good fit I have a drawer full of disused NIC's I could experiment with.

Just want to make sure there isn't any helpful info I am missing before I start spending a ton of time experimenting.

It looks like the biggest candidates for useful improvement are hardware CRC and Hardware TSO.

Hardware LRO maybe be helpful to CPU load, but the impacts to latency due to buffer bloat may be undesirable, so I wasnt planning on trying that one.

I do use VLAN's so Hardware VLAN filtering may help (if it works and is reliable)

ARP handling is more ambiguous to me.   Not sure what there is to gain here and what the implications are.

Anyway, I'd appreciate any tips, tricks, suggestions, references for further reading, or any other input anyone might provide.

Thanks,
Matt

[meyergru]

The goal here is to reduce some of the high CPU usage I have seen during high bandwidth (~gigabit) use over WireGuard.   I am not sure how much of that load is encryption vs. packet handling / interrupts and how much potential there is to help things by enabling NIC hardware offload.

Then why did you not try to find out by measuring without Wireguard? If the CPU load drops much and/or the speed goes up, it is highly likely that Wireguard is the culprit.

Given your CPU, I doubt that network load is much of a concern, however, you still have a third potential problem, namely that you run on Proxmox (albeit via IOMMU).

If I should guess, high CPU load can be credited to Wireguard, because the native FreeBSD implementation is not optimized for AMD64 instruction sets.