18.1 Network Performance (17.7.11-12 was fine)?

[dcol]

Also slowdowns here. Using Intel i350T4
Here is a comparison using same ISP connection.
150 down with OPNsense
310 down with PFsense.

Upload speed does not seem affected
Was not like this with 17.7.12

Not happy, hope there is an explanation or patch soon

[franco]

Is this all measurement with NAT in place?

We can try two things:

(a) Use different (older) kernels, even 11.0 to see if the issue is there, or

(b) Some quirk in the NAT rule generation setup causes this slowdown which seems to be half of what is expected?



Cheers,
Franco

[dcol]

Not sure, but I had to do the NAT patch to get NAT to work.

What is the shell command to revert back to 17.7? Or should I reinstall 17.7 to test it again just to make sure. And if I reinstall 17.5, how do I prevent it upgrading to 18.1. I would want to get back to 17.7.12

Something else I noticed right off, the memory usage has dropped from 12% with 17.7 to 6% with 18.1

[franco]

Try this first, reverting to the RC kernel where changes are minimal:

# opnsense-update -kr 18.1.r1
# /usr/local/etc/rc.reboot

[dcol]

Applied it.
System>Firmware still shows 18.1 installed. And console shows 18.1_1 installed
Tried the speedtest again with no improvement.

Not sure the downgrade worked

[dcol]

If I reinstall 17.7.5, how do I update to 17.7.12 without going to 18.1?

[franco]

"uname -a" will reveal the src / kernel commit. Okay, so it's not the PCP change[1].

You can upgrade to 17.7.12 no problem, 18.1 is on a different update track, you can't escape 17.7.x in the "normal" updates.


Cheers,
Franco

[1] https://github.com/opnsense/src/commit/dabc3cf4

[dcol]

Ok, great I will try it and report back the results. Glad I take lots of configuration backups.

[franco]

The most useful test is probably 18.1 with a FreeBSD 11.0 kernel underneath, but I want to double-check this is stable first.

Otherwise, 17.7.12 performance matters not so much.... rather opnsense-devel and a clean reboot.

Last test is 17.7.12 with FreeBSD 11.1 kernel (I know this should be stable but is tricky to update to, but works from opnsense-devel).

That should give us enough info which puzzle piece is responsible for the slowdown.


Cheers,
Franco

[franco]

PS: From the looks of it and missing reports for beta and all it's either driver specific or something with the new way of unrolling the NAT rules.

[funar]

I have a solution for my Broadcom-NIC'ed Dell R610. It was definitely a driver issue more than a NAT or anything else.  The hw.bce.* settings only apply to Broadcom, but there's probably similar options for the others who have also replied to this thread.

In /boot/loader.conf.local:

hw.bce.verbose=0
hw.bce.tso_enable=0
hw.bce.rx_pages=8
hw.bce.tx_pages=8
hw.pci.enable_msix=1
hw.pci.enable_msi=1
net.inet.tcp.tso=0
net.inet.tcp.sendbuf_max=16777216
net.inet.tcp.recvbuf_max=16777216
net.inet.tcp.sendbuf_inc=16384
net.inet.tcp.recvbuf_inc=524288

I'm now getting full bandwidth in both directions via opnsense. After getting that sorted out, I enabled dual-stack IPv4/IPv6, IDS and upstream traffic shaping. No more issues.

[funar]

It should go without saying, but you must reboot after making changes to /boot/loader.conf.local ...

[dcol]

Tried 17.7.12 and all went back to expected speeds.
Then I realized that the upgrade had rewritten some of my performance tweaks.
All speeds look normal now running 18.1_1.

I also noticed a new suricata.yaml. I will have to dive into that because I also had changes in there as well.

[franco]

Did you use /boot/loader.conf for these? Or were they really gone from /boot/loader.conf.local ? It shouldn't touch .local ever...


Cheers,
Franco

[franco]

PS: For everyone else still having issues, please state your network driver name for reference.