Three day already trying to solve a problem.
I have a two sites with OpnSenses on bare metal (fully identical, Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz (4 cores, 8 threads))
Sites connected with wireguard, on kmod implementation (os-wireguard), channel width between sites 1Gbps.
Testing bandwidth between sites on os-wireguard:
last pid: 13225; load averages: 1.92, 0.95, 0.66 up 0+08:06:23 18:25:46
435 threads: 11 running, 401 sleeping, 23 waiting
CPU 0: 0.0% user, 0.0% nice, 77.9% system, 0.0% interrupt, 22.1% idle
CPU 1: 0.0% user, 0.0% nice, 69.0% system, 3.9% interrupt, 27.1% idle
CPU 2: 0.4% user, 0.0% nice, 70.5% system, 5.4% interrupt, 23.6% idle
CPU 3: 0.8% user, 0.0% nice, 70.9% system, 3.9% interrupt, 24.4% idle
CPU 4: 0.0% user, 0.0% nice, 69.8% system, 7.8% interrupt, 22.5% idle
CPU 5: 0.0% user, 0.0% nice, 70.5% system, 7.8% interrupt, 21.7% idle
CPU 6: 0.0% user, 0.0% nice, 73.6% system, 0.0% interrupt, 26.4% idle
CPU 7: 0.4% user, 0.0% nice, 76.7% system, 0.0% interrupt, 22.9% idle
Mem: 107M Active, 293M Inact, 926M Wired, 40K Buf, 30G Free
ARC: 323M Total, 72M MFU, 183M MRU, 164K Anon, 5624K Header, 62M Other
180M Compressed, 447M Uncompressed, 2.49:1 Ratio
Swap: 8192M Total, 8192M Free
PID USERNAME PRI NICE SIZE RES STATE C TIME CPU COMMAND
0 root -76 - 0B 2176K - 0 4:42 73.87% kernel{wg_tqg_0}
0 root -76 - 0B 2176K - 7 5:13 70.79% kernel{wg_tqg_7}
0 root -76 - 0B 2176K CPU5 5 4:42 67.78% kernel{wg_tqg_5}
0 root -76 - 0B 2176K - 1 4:42 67.63% kernel{wg_tqg_1}
0 root -76 - 0B 2176K - 6 4:40 67.16% kernel{wg_tqg_6}
0 root -76 - 0B 2176K - 4 4:38 66.85% kernel{wg_tqg_4}
0 root -76 - 0B 2176K - 3 4:40 66.61% kernel{wg_tqg_3}
0 root -76 - 0B 2176K - 2 4:34 64.67% kernel{wg_tqg_2}
12 root -72 - 0B 384K RUN 5 1:04 30.90% intr{swi1: netisr 0}
11 root 155 ki31 0B 128K RUN 6 476:24 30.06% idle{idle: cpu6}
11 root 155 ki31 0B 128K RUN 1 476:45 27.30% idle{idle: cpu1}
11 root 155 ki31 0B 128K RUN 7 476:17 27.22% idle{idle: cpu7}
11 root 155 ki31 0B 128K RUN 3 476:25 26.75% idle{idle: cpu3}
11 root 155 ki31 0B 128K RUN 2 476:16 25.15% idle{idle: cpu2}
Huge CPU load on both sides, and bandwidth capped on ~500 Mbits:
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.00 sec 1.78 GBytes 509 Mbits/sec 38 sender
[ 5] 0.00-30.02 sec 1.78 GBytes 508 Mbits/sec receiver
On wireguard-go I have 800-850 Mbits/s and 20-30% cpu usage. What am I doing wrong?