[Solved] Is RSS available on 22.7 series?

[cookiemonster]

I did some RSS testing when I was on 21.7.5 and there was no discernible impact so I left the tunables in when I upgraded to 22.7. I later saw some of the tunables labelled as unsupported so I removed from the UI. All good.

I am tracing a marked reduction in bitrate from WAN to LAN which made me look at tunables again and the status of RSS as part of the elimination process. Cables and similar are part of the process at the moment of course, and in the meantime, this question.

BEFORE:
I had OPN 21.7.5, OpenSSL, Hardware is APU4.
Upgraded BIOS beforehand, coreboot v 4.14.0.6 .
Network interfaces on this system are igb.

penguin@OPNsense:~ % sudo sysctl -a | grep -i 'isr.bindthreads\|isr.maxthreads\|inet.rss.enabled\|inet.rss.bits'
net.inet.rss.enabled: 1
net.inet.rss.bits: 2
net.isr.bindthreads: 1
net.isr.maxthreads: 4
penguin@OPNsense:~ % sudo netstat -Q
Configuration:
Setting                        Current        Limit
Thread count                         4            4
Default queue limit                256        10240
Dispatch policy                 direct          n/a
Threads bound to CPUs          enabled          n/a

Protocols:
Name   Proto QLimit Policy Dispatch Flags
ip         1   1000    cpu   hybrid   C--
igmp       2    256 source  default   ---
rtsock     3    256 source  default   ---
arp        4    256 source  default   ---
ether      5    256    cpu   direct   C--
ip6        6    256    cpu   hybrid   C--
ip_direct     9    256    cpu   hybrid   C--
ip6_direct    10    256    cpu   hybrid   C--


NOW:
Now on OPN 22.7.4, OpenSSL, same hardware.

root@OPNsense:/home/penguin # sysctl -a | grep -i 'isr.bindthreads\|isr.maxthreads\|inet.rss.enabled\|inet.rss.bits'
net.inet.rss.enabled: 0
net.inet.rss.bits: 2
net.isr.bindthreads: 0
net.isr.maxthreads: 1

root@OPNsense:/home/penguin # netstat -Q
Configuration:
Setting                        Current        Limit
Thread count                         1            1
Default queue limit                256        10240
Dispatch policy                 direct          n/a
Threads bound to CPUs         disabled          n/a

Protocols:
Name   Proto QLimit Policy Dispatch Flags
ip         1   1000   flow  default   ---
igmp       2    256 source  default   ---
rtsock     3    256 source  default   ---
arp        4    256 source  default   ---
ether      5    256 source   direct   ---
ip6        6   1000   flow  default   ---

So it looks like during the upgrade, RSS was disabled. Question 1: Is that expected?
Since I realised that today, I've re-added the tunables that are accepted "net.inet.rss.bits", "net.isr.maxthreads" and "net.inet.rss.enabled" and rebooted . I also had to but netstat -Q shows that threads aren't bound to CPU enabled.
Configuration:
Setting                        Current        Limit
Thread count                         4            4
Default queue limit                256        10240
Dispatch policy                 direct          n/a
Threads bound to CPUs         disabled          n/a

Protocols:
Name   Proto QLimit Policy Dispatch Flags
ip         1   1000    cpu   hybrid   C--
igmp       2    256 source  default   ---
rtsock     3    256 source  default   ---
arp        4    256 source  default   ---
ether      5    256    cpu   direct   C--
ip6        6   1000    cpu   hybrid   C--
ip_direct     9    256    cpu   hybrid   C--
ip6_direct    10    256    cpu   hybrid   C--

Question 2: Should I expect the setting to change ie. is wrong at present without threads bound to CPU?
Question 3: How do I make it so?

Thanks for the assistance.

[franco]

It says "NOW" but

net.inet.rss.enabled: 0

Is this correct?


Cheers,
Franco

[tuto2]

Since I realised that today, I've re-added the tunables that are accepted "net.inet.rss.bits", "net.isr.maxthreads" and "net.inet.rss.enabled" and rebooted . I also had to but netstat -Q shows that threads aren't bound to CPU enabled.
Question 2: Should I expect the setting to change ie. is wrong at present without threads bound to CPU?
Question 3: How do I make it so?

I'm missing the "net.inet.rss.bindthreads" tunable from your list of applied steps. Maybe you forgot it?

[cookiemonster]

That was the question Franco. It was enabled, then system upgraded, then disabled. The question was if it was expected for the upgrades to retain or remove the rss enablement.
Since I found it disabled, I've gone and set again to 1 via tunable.

The second question is if "Threads bound to CPUs         disabled          n/a" is correct and expected. If not, how do I re-enable it.

[cookiemonster]

Since I realised that today, I've re-added the tunables that are accepted "net.inet.rss.bits", "net.isr.maxthreads" and "net.inet.rss.enabled" and rebooted . I also had to but netstat -Q shows that threads aren't bound to CPU enabled.
Question 2: Should I expect the setting to change ie. is wrong at present without threads bound to CPU?
Question 3: How do I make it so?

I'm missing the "net.inet.rss.bindthreads" tunable from your list of applied steps. Maybe you forgot it?

Thanks Tuto2. I did try and just did now but I get "unsupported" appear once added in the tunables. Let me see if I can find a name change in sysctl list.

[cookiemonster]

Code Select Expand

root@OPNsense:/home/penguin # sysctl -a | grep bind
kern.ipc.tls.bind_threads: 1
net.isr.bindthreads: 0
user.posix2_c_bind: 0
So I've tried adding the tunable as net.isr.bindthreads and net.inet.rss.bindthreads
Both give an "unsupported" message.
Is that expected? Which is right for this kernel version?  13.1-RELEASE-p2 FreeBSD 13.1-RELEASE-p2 stable/22.7-n250229-c8a7fd0d5dc SMP amd64

[cookiemonster]

Correction.
I've used the net.isr.bindthreads tunable now instead. I'm guessing net.inet.rss.bindthreads was replaced?
I need to reboot later and check again if it sticks.

[tuto2]

So I've tried adding the tunable as net.isr.bindthreads and net.inet.rss.bindthreads
Both give an "unsupported" message.
Is that expected? Which is right for this kernel version?  13.1-RELEASE-p2 FreeBSD 13.1-RELEASE-p2 stable/22.7-n250229-c8a7fd0d5dc SMP amd64

My mistake, it should be "net.isr.bindthreads" (not net.isr.rss.bindthreads).

After applying and rebooting:

Code Select Expand


Configuration:
Setting                        Current        Limit
Thread count                         1            1
Default queue limit                256        10240
Dispatch policy                 direct          n/a
Threads bound to CPUs          enabled          n/a


I see no indication of unsupported anywhere.

[franco]

Which "unsupported" message you mean? In the log for runtime settings? The GUI says "boot-time" for net.inet.rss.enabled which is correct...


Cheers,
Franco

[cookiemonster]

It's been spotted what the problem was franco, thanks. It should be "net.isr.bindthreads" (not net.isr.rss.bindthreads). The wrong one caused the "unsupported" message on the UI when adding it.
In light of this if the tunable string changed between 22.1 and 22.7, and I found it gone, I wonder if the upgrade routines should have re-enabled the new one if the old one was enabled at upgrade time.
For now I just need to reboot and I'll be golden :)

[franco]

I think it was always net.isr.bindthreads as it pertains to more than just RSS, maybe some old forum post got this wrong? The other one likely did not work in any version of the RSS patching.


Cheers,
Franco

[cookiemonster]

OK. It probably is that then. Reboot done. All appears good again. I'll mark it as solved.
Thanks Franco and Tuto2 for your help as always.

[franco]

yay :)