Messages

The opnsense backend works with Cloudflare if you use the Global API token as the password.

I stand corrected, I don't know what changed, but with the latest update to 23.1.4 I am now able to max out my 10gig link using a single iperf3 stream.

This is achieved using inter-vlan routing using a single 10gig port as an uplink and one stream. It's a LXC container in a proxmox box routing over to a different vlan on a debian server.

I have a similar system to yours:

Xeon E-2234
16GB RAM
Intel x550 adapter

I get around 7.5gbps for a single stream iperf3 run; any number of streams above that (-P 2 or higher) I easily max out the 10gbps limit of the adapter.

I just don't think *BSD distros will be able to get much higher performance than this, though this performance should be more than enough for the majority of use cases. If you really need >10gbps routing you should think about layer 3 switching or maybe try a linux distro.

I didn't have to change anything to reach the above speeds out of the box, though I did get a small boost implementing RSS as described on this page:

https://docs.opnsense.org/troubleshooting/performance.html

https://forum.opnsense.org/index.php?topic=32920.0

I am seeing the same. I use three services: cloudflare, duckdns, and dyndns. All three were working as expected through the 22.7 series, but since I upgraded to 23.1.1_2 none of the columns update. I use the interface WAN method for updates for all three services.

Is DCO for OpenVPN in the pipeline at the moment?

To answer my own question, and just to have the answer for anybody else in the future, the Intel 8960 does indeed seem to be functioning. Running an iPerf3 over an IPSec tunnel and watching

Code Select Expand

vmstat -i | grep qat definitely increments the counters.

Two command outputs below that lead me to believe everything is working as it should.

Code Select Expand

root@OPNsense:~ # vmstat -i | grep qat
irq144: qat0                      296664          2
irq145: qat0                     1547616         13
irq146: qat0                      142510          1
irq147: qat0                      195715          2
irq148: qat0                      232107          2
irq149: qat0                      255745          2
irq150: qat0                       63838          1
irq151: qat0                     1632242         13
irq160: qat0                           1          0
irq161: qat1                      104049          1
irq162: qat1                      327807          3
irq163: qat1                       57244          0
irq164: qat1                      594580          5
irq165: qat1                      210576          2
irq166: qat1                          27          0
irq167: qat1                      189053          2
irq168: qat1                      576246          5
irq179: qat2                     1430647         12
irq194: qat2                           1          0

Code Select Expand

root@OPNsense:~ # sysctl -a | grep qat
qat0: <Intel C620/Xeon D-2100 QuickAssist PF> mem 0x91540000-0x9157ffff,0x91500000-0x9153ffff at device 0.0 on pci3
qat_ae_cluster_intr
qat1: <Intel C620/Xeon D-2100 QuickAssist PF> mem 0x91440000-0x9147ffff,0x91400000-0x9143ffff at device 0.0 on pci4
qat2: <Intel C620/Xeon D-2100 QuickAssist PF> mem 0x91340000-0x9137ffff,0x91300000-0x9133ffff at device 0.0 on pci5
qat_ae_cluster_intr
irq144: qat0:295 @cpu0(domain0): 296664
irq145: qat0:297 @cpu1(domain0): 1547627
irq146: qat0:299 @cpu2(domain0): 142510
irq147: qat0:301 @cpu3(domain0): 195715
irq148: qat0:303 @cpu4(domain0): 232107
irq149: qat0:305 @cpu5(domain0): 338682
irq150: qat0:307 @cpu6(domain0): 105541
irq151: qat0:309 @cpu7(domain0): 1632242
irq152: qat0:311 @cpu0(domain0): 0
irq153: qat0:313 @cpu1(domain0): 0
irq154: qat0:315 @cpu2(domain0): 0
irq155: qat0:317 @cpu3(domain0): 0
irq156: qat0:319 @cpu4(domain0): 0
irq157: qat0:321 @cpu5(domain0): 0
irq158: qat0:323 @cpu6(domain0): 0
irq159: qat0:325 @cpu7(domain0): 0
irq160: qat0:327 @cpu0(domain0): 1
irq161: qat1:329 @cpu0(domain0): 104049
irq162: qat1:331 @cpu1(domain0): 327807
irq163: qat1:333 @cpu2(domain0): 57244
irq164: qat1:335 @cpu3(domain0): 594580
irq165: qat1:337 @cpu4(domain0): 210576
irq166: qat1:339 @cpu5(domain0): 27
irq167: qat1:341 @cpu6(domain0): 189053
irq168: qat1:343 @cpu7(domain0): 576246
irq169: qat1:345 @cpu0(domain0): 0
irq170: qat1:347 @cpu1(domain0): 0
irq171: qat1:349 @cpu2(domain0): 0
irq172: qat1:351 @cpu3(domain0): 0
irq173: qat1:353 @cpu4(domain0): 0
irq174: qat1:355 @cpu5(domain0): 0
irq175: qat1:357 @cpu6(domain0): 0
irq176: qat1:359 @cpu7(domain0): 0
irq177: qat1:361 @cpu0(domain0): 0
irq178: qat2:363 @cpu0(domain0): 0
irq179: qat2:365 @cpu1(domain0): 1488722
irq180: qat2:367 @cpu2(domain0): 0
irq181: qat2:369 @cpu3(domain0): 0
irq182: qat2:371 @cpu4(domain0): 0
irq183: qat2:373 @cpu5(domain0): 0
irq184: qat2:375 @cpu6(domain0): 0
irq185: qat2:377 @cpu7(domain0): 0
irq186: qat2:379 @cpu0(domain0): 0
irq187: qat2:381 @cpu1(domain0): 0
irq188: qat2:383 @cpu2(domain0): 0
irq189: qat2:385 @cpu3(domain0): 0
irq190: qat2:387 @cpu4(domain0): 0
irq191: qat2:389 @cpu5(domain0): 0
irq192: qat2:391 @cpu6(domain0): 0
irq193: qat2:393 @cpu7(domain0): 0
irq194: qat2:395 @cpu0(domain0): 1
dev.qat.2.stats.sym_alloc_failures: 0
dev.qat.2.stats.ring_full: 0
dev.qat.2.stats.gcm_aad_updates: 0
dev.qat.2.stats.gcm_aad_restarts: 0
dev.qat.2.%parent: pci5
dev.qat.2.%pnpinfo: vendor=0x8086 device=0x37c8 subvendor=0x8086 subdevice=0x0001 class=0x0b4000
dev.qat.2.%location: slot=0 function=0 dbsf=pci0:5:0:0
dev.qat.2.%driver: qat
dev.qat.2.%desc: Intel C620/Xeon D-2100 QuickAssist PF
dev.qat.1.stats.sym_alloc_failures: 0
dev.qat.1.stats.ring_full: 0
dev.qat.1.stats.gcm_aad_updates: 0
dev.qat.1.stats.gcm_aad_restarts: 0
dev.qat.1.%parent: pci4
dev.qat.1.%pnpinfo: vendor=0x8086 device=0x37c8 subvendor=0x8086 subdevice=0x0001 class=0x0b4000
dev.qat.1.%location: slot=0 function=0 dbsf=pci0:4:0:0
dev.qat.1.%driver: qat
dev.qat.1.%desc: Intel C620/Xeon D-2100 QuickAssist PF
dev.qat.0.stats.sym_alloc_failures: 0
dev.qat.0.stats.ring_full: 0
dev.qat.0.stats.gcm_aad_updates: 0
dev.qat.0.stats.gcm_aad_restarts: 0
dev.qat.0.%parent: pci3
dev.qat.0.%pnpinfo: vendor=0x8086 device=0x37c8 subvendor=0x8086 subdevice=0x0001 class=0x0b4000
dev.qat.0.%location: slot=0 function=0 dbsf=pci0:3:0:0
dev.qat.0.%driver: qat
dev.qat.0.%desc: Intel C620/Xeon D-2100 QuickAssist PF
dev.qat.%parent:

Is there any way to check whether OPNsense is using an add-on QAT accelerator properly? I just installed an Intel 8960 QAT accelerator and turned on the setting to enable QAT. Is there a way to check that "it worked" and the card is supported and being utilized? I know the "other" platform explicitly lists which acceleration is being utilized and which ciphers are supported.

Code Select Expand

root@OPNsense:~ # kldstat
Id Refs Address                Size Name
1   96 0xffffffff80200000  215db18 kernel
2    1 0xffffffff8235e000     e4d0 if_bridge.ko
3    2 0xffffffff8236d000     7870 bridgestp.ko
4    1 0xffffffff82375000     ba48 if_gre.ko
5    1 0xffffffff82381000    11920 ipmi.ko
6    3 0xffffffff82393000     3c68 smbus.ko
7    1 0xffffffff82397000     e318 pfsync.ko
8    3 0xffffffff823a6000    741a8 pf.ko
9    1 0xffffffff8241b000     3b18 pflog.ko
10    1 0xffffffff8241f000     4b58 if_enc.ko
11    1 0xffffffff82424000     f460 carp.ko
12    1 0xffffffff82434000    181d0 if_lagg.ko
13    2 0xffffffff8244d000     3538 if_infiniband.ko
14    1 0xffffffff82a10000     2110 pchtherm.ko
15    1 0xffffffff82a13000     5e7c ig4.ko
16    1 0xffffffff82a19000     433c iicbus.ko
17    1 0xffffffff82a1e000     3250 ichsmb.ko
18    1 0xffffffff82a22000     3378 acpi_wmi.ko
19    1 0xffffffff82a26000     2340 uhid.ko
20    1 0xffffffff82a29000     4350 ums.ko
21    1 0xffffffff82a2e000     3380 usbhid.ko
22    1 0xffffffff82a32000     31f8 hidbus.ko
23    1 0xffffffff82a36000    16308 qat.ko
24    1 0xffffffff82a4d000    e2330 qat_c62xfw.ko
25    1 0xffffffff82b30000     20f0 coretemp.ko
26    1 0xffffffff82b33000     4700 nullfs.ko
27    1 0xffffffff82b38000     f418 ipsec.ko
28    1 0xffffffff82b48000    34568 if_wg.ko
29    1 0xffffffff82b7d000     39c0 ng_socket.ko
30    4 0xffffffff82b81000     aac8 netgraph.ko
31    1 0xffffffff82b8c000     31c8 ng_ether.ko
32    1 0xffffffff82b90000     53f8 ng_netflow.ko
33    1 0xffffffff82b96000     31e8 ng_ksocket.ko

and

Code Select Expand

qat0: <Intel C620/Xeon D-2100 QuickAssist PF> mem 0x91540000-0x9157ffff,0x91500000-0x9153ffff at device 0.0 on pci3
qat_ae_cluster_intr
qat1: <Intel C620/Xeon D-2100 QuickAssist PF> mem 0x91440000-0x9147ffff,0x91400000-0x9143ffff at device 0.0 on pci4
qat2: <Intel C620/Xeon D-2100 QuickAssist PF> mem 0x91340000-0x9137ffff,0x91300000-0x9133ffff at device 0.0 on pci5
qat_ae_cluster_intr