[Tutorial/Call for Testing] Enabling Receive Side Scaling on OPNsense

[MartB]

Not available to download, no signature found.

Make sure:

1. Default mirror is set
2. "-z" is being used as indicated

It looks like everything is there: https://pkg.opnsense.org/FreeBSD:12:amd64/snapshots/sets/

I double checked (copied your command before), no dice cant get it to work. Mirror is also set to default.
Do i need to go back to base kernel and then install the new rss version maybe?

Code Select Expand


Fetching kernel-21.7.2-rss-amd64-RSS.txz: .+ mkdir -p /var/cache/opnsense-update/41781
+ opnsense-fetch -a -T 30 -q -o /var/cache/opnsense-update/41781/kernel-21.7.2-rss-amd64-RSS.txz.sig https://pkg.opnsense.org/FreeBSD:12:amd64/snapshots/sets/kernel-21.7.2-rss-amd64-RSS.txz.sig
.+ exit_msg ' failed, no signature found'
+ [ -n ' failed, no signature found' ]
+ echo ' failed, no signature found'
failed, no signature found
+ exit 1


[franco]

Maybe you're having that other device type issue mentioned here. Helping is difficult if you don't paste the error, but try this one then:

    # opnsense-update -zfkr 21.7.2-rss -D ''


Cheers,
Franco

[MartB]

Yup it was the additional RSS breaking things, solved with the -D '' saw it only after running verbose output on the update command.

[madj42]

Just curious as it's been a while since this was initially posted for testing.  Will this eventually be rolled into an upcoming point release soon or an updated kernel with the latest changes in the repo?

Thank you so much for the fort and work you guys do!

[franco]

The issue with RSS in IPv6 was fixed and the Unbound situation looks like a problem in the kernel regarding SO_REUSEPORT/RSS enabled combination: Unbound is using this by default, but it could be made conditional upon RSS flag for example.

Nevertheless we will ship the RSS-capable kernel with 21.7.4 defaulting to off like in this test series and will continue to improve it and do further testing to see if it can be enabled by default later on.


Cheers,
Franco

[dinguz]

I just tested this and indeed the problem is gone when I disable so-reuseport in unbound.

Are there any other applications in general use with OPNsense that are known to use this socket option, and thus are possibly affected?

[bunchofreeds]

Hi,

Will RSS and multiqueue benefit PPPoE when used as the WAN connection with your ISP?

Thanks

[franco]

I added the RSS conditional for so-reuseport:

https://github.com/opnsense/core/commit/84d6b2acd5

After a bit of research PPPoE does not accelerate in RSS since the IP packets are higher up the stack and RSS will hash these to "0" so the distribution is one core only. I assume that also includes using the

Code Select Expand

net.isr.dispatch=deferred

tunable so for PPPoE users RSS would likely have to be kept off to get at least some performance increase from their connection. Linux does have RPS to deal with PPPoE acceleration but FreeBSD has no equivalent.


Cheers,
Franco

[franco]

PS: Thanks all for testing this, highly appreciated! We will be adding a note in the release notes for Suricata speedup testing in the development version for 21.7.4.

[bunchofreeds]

@Franco, thanks for the reply regarding PPPoE

[madj42]

Removed the question.  Sorry, lack of caffeine.  It's working great on 21.7.4.  Thank you for the work as always!

[dinguz]

The issue with RSS in IPv6 was fixed and the Unbound situation looks like a problem in the kernel regarding SO_REUSEPORT/RSS enabled combination: Unbound is using this by default, but it could be made conditional upon RSS flag for example.

I would consider this more of a workaround than a proper fix. Would this be a complicated issue to achieve a 'real' fix?

[franco]

Sure, did not claim it was a fix -- more of an opportunistic safeguard to prevent misconfiguring unbound when using RSS since it's not possible to turn so-reuseport off from the GUI.

Digging through the kernel code is going to take a lot more time than a two line change and Stephan is currently pushing the RSS modifications to our upcoming 22.1 branch on FreeBSD 13 so we can have that datapoint as well (it could have been fixed already?).


Cheers,
Franco

[Tupsi]

I wanted to thank you guys for getting this into the 21.7.x release lately. After doing the works with the mentioned tunables, I know finally get my full internet I currently should have (1000/500). Before these changes the download side throttled around 500, so I ended up with a 500/500 line.

so THANK YOU!

[Koldnitz]

Has anyone noticed significant (a full magnitude or greater slow down) on the average recursion time in unbound after enabling this option?

I tried with the latest stable build 21.7.5.

Cheers,