[Solved] Very slow internet after upgrade to 22.1

[ednt]

After upgrading a master slave installation from 21.7.8 to 22.1 we have very slow internet access.
www.speedtest.net gives a fast ping time, but only 2MBits as download and 0 as upload, or it stops with a socket error.

The only thing what I can see is in 'Reporting'->'Health'->'System'->'States' that the states stay low:



Hardware is a HP Gen9 server with a QLogic NetXtreme II BCM57810 10GbE (2 port)
and an Intel I340 82580 1GbE (4 port)

Any ideas?

[ednt]

Just updated to 22.1.1 the problem is still there.

[phoenix]

Just updated to 21.1.1 the problem is still there.

I presume you mean 22.1.x? You also haven't mentioned what kind of internet connection you have from your ISP.  what speed is your service and what type is it: FTTP, ADSL etc? By "master slave installation" do you mean a 'failover' setup?

[ednt]

Of course 22.1.1

We have a direct link with 10Gbit Fiber up and down synchronuos speed.
An other Master/Slave CARP failover (still on 21.7.8 )  on the same WAN connection shows with the same test 900Mbit up and down.

Ok 21.7.8:



Not Ok 22.1.1:



And this was a good result actual test gives 1.94 Mbps and 0 Mbps.
Sometimes I get a socket error.

After I noticed this behaviour I stopped the upgrade of the other installation, since it uses the same hardware.

[ednt]

At the end of the day, if no solution is found, I need to revert this upgrade.

It is mentioned to use

Code Select Expand

opnsense-update -kr 21.7.8
opnsense-shell reboot

only after advise.

But is it a good idea to only go back to 21.7.8 with

Code Select Expand

opnsense-revert -r 21.7.8 opnsense

What would be the right order to go back?

[franco]

There's NO WAY to revert this SAFELY other than a reinstall due to the major upgrade. Make sure you read the migration notes and are not looking at some sort of driver issue on FreeBSD 13.

It would help to start with your setup (driver, wan structure VLAN/LAGG and connectivity for IPv4/IPv6), current version used (there is 22.1 and 22.1.1 already) and not with your generic problem description alone.


Cheers,
Franco

[ednt]

Btw. it is not the WAN connection.
I just used curl on the console to download a 50MB file and it was done in 2 seconds.

What exactly do you need?

I can not publish the config here in the forum.

loaded drivers:

Code Select Expand

kldstat
Id Refs Address                Size Name
1   52 0xffffffff80200000  2159b38 kernel
2    3 0xffffffff8235a000    73db0 pf.ko
3    1 0xffffffff823cf000     e4f0 if_bridge.ko
4    2 0xffffffff823de000     7870 bridgestp.ko
5    1 0xffffffff823e6000     e318 pfsync.ko
6    1 0xffffffff823f5000     4b58 if_enc.ko
7    1 0xffffffff823fa000   5b7420 zfs.ko
8    1 0xffffffff829b2000     ab48 opensolaris.ko
9    1 0xffffffff829bd000     3b18 pflog.ko
10    1 0xffffffff829c1000     ba48 if_gre.ko
11    1 0xffffffff829cd000     f460 carp.ko
12    1 0xffffffff829dd000    181d0 if_lagg.ko
13    2 0xffffffff829f6000     3538 if_infiniband.ko
14    1 0xffffffff82d21000     8da8 ioat.ko
15    1 0xffffffff82d2a000     2340 uhid.ko
16    1 0xffffffff82d2d000     4350 ums.ko
17    1 0xffffffff82d32000     3380 usbhid.ko
18    1 0xffffffff82d36000     31f8 hidbus.ko

One 1GB NIC is used for management
igb0: <Intel(R) I340 82580 (Copper)>

One 1GB NIC is used for sync
igb1: <Intel(R) I340 82580 (Copper)>

All other interfaces uses one 10GB NIC with VLANs
bxe0: <QLogic NetXtreme II BCM57810 10GbE (B0) BXE v:1.78.91>

All interfaces are in CARP mode to be redundant.

[franco]

Did you assign and enable all the VLAN parents as mentioned in the migration notes?


Cheers,
Franco

[ednt]

Hi Franco,

I only read the stuff below the changes and since we don't use mac spoofing I ignored the text behind.
Or do you mean something else with 'migration notes'.
Where can I read them after the installation?

[franco]

System: Firmware: Changelog: 22.1


Cheers,
Franco

[ednt]

As I told you, I read what is written below the changelog and we don't use spoofing and we don't use GRE.
I added now the parent of the VLAN interfaces as a dummy and enabled it.

It works now.
Thank you very much.

But this was not 'readable' for me.
I thought

Media and hardware offload settings are no longer shown for non-parent interfaces and need to be set individually on the parent interface to take effect.  This can introduce unwanted configuration due to previous side effects in the code.  If the parent interface was not previously assigned please assign it to reapply the required settings.

does not affect me, since I don't used any 'special' media and hardware offload settings.

It should clearly stated as an extra point:

• Asign the parent interface of VLAN interfaces and enable it if not already done, even it is not used.

This would make it much clearer.

Best regards,

Bernd

[Gary7]

FWIW, today I saw that 'Reporting'->'Health'->'System'->'States' had the wrong display and it started when I upgraded to 22.1. I've had a couple of reboots since.

I searched the forum and found this discussion.
My home firewall is a very simple 1 WAN & 1 LAN on an APU2.
My speed is exactly the same as pre-22.1 ( a little over 200 Mbps, my full Internet speed )
I'm running Unbound with blocklists and a few Firewall:Rules:LAN, i.e.spamhaus_drop
Nothing inbound.
I've set several tunables to try to get max performance.

I just noticed that Reporting:Settings still has Round-Robin-Database enabled (from pre-22.1). Should I shut-off RRD graphing backend ?
Since I have only 1 WAN, I believe that I can safely shut-off RRD graphing backend.
[Update] Looks like I need to have RRD graphing enabled to display Health graphs.

As always, Franco, you do outstanding work.
Thanks

[franco]

I think there is a bug/incompatibility in the way the pfctl stats are presented now on FreeBSD 13 which end up as flat(?) readings in the RRD, but we don't have a ticket on GitHub for it yet.


Cheers,
Franco