Routing issues after upgrading to 22.1

[hmronline]

I've had OPNsense properly working with version 21.7.8, which is installed as a VM in a Proxmox Environment.
I have a couple of VLANs (parent interface properly set) and almost no firewall rules set other than NAT related rules and WAN inbound traffic block.
Again, everything was working fine with v21.7.x.

After upgrading to 22.1 communication between VLANs is not working properly, as follows:
* Internet access is OK, meaning: communication is started from host on VLAN A and goes into the Internet through WAN interface.
* Ping/ICMP test is ok from host in VLAN A to another host in VLAN B.
* TCP connection on port 443 from same host in VLAN A to same host in VLAN B is not working and therefore timed out.
* Any other TCP port behaves the same, and are not working. Tested with netcat.
* Enabled logging of firewall rules and no blocked communication is shown. In fact tests are shown as PASS en Live View.
* Tried adding inbound/outbound allow all firewall rules and it is still not working.
* Reverting back to version 21.7.8 (from a Proxmox Backup) makes everything work OK again.
* Upgrading once again to 22.1.x with no configuration changes ends with the same result: no communication is possible on TCP ports between VLANs.


Haven't found anything related on changelog, nothing on recent posts on this forum, and haven't read nothing that I'm aware of that can produce the results mentioned above.

Hopefully this issue is something really simple to solve but I'm not being able to determine what is causing it, so here I am searching for help.

Thanks in advance!
Hernán.-

[mimugmail]

Did you assign parent interface when using Vlans?

[Stumie]

Hello folks,

I found out to have exactly the same issue like the thread creator after upgrading from 21ish to 22.1 with a similar setup (OPNsense in a Proxmox VM).
I also reverted the upgrade with a rollback of a full VM backup.

Let me know, how I can help with additional information.

- Stumie

[spark5]

hi guys,

we run in the same problem. icmp is working, tcp not.
i captured with tcpdump and saw the syn, syn-ack packets.
after this, the sender sends out retransmission packets.

something stuck with the last handshake part of tcp.

we can also send some informations for troubleshooting.

thanks for help, we do not find any solutions at this time.

kind regards,
ronny

[spark5]

hi ... again, i found a solution in the miracles of the release notes.

o Media settings are no longer shown for non-parent interfaces and need to be set individually to take effect.  This can introduce unwanted configuration due to previous side effects in the code.  If the parent interface was not previously assigned please assign it to reapply the required media settings.

and on the upgrade screen
... Media and hardware offload settings are no longer shown for non-parent interfaces ...

that means, that the hardware offload engines are enabled, until you enable the "parent" interface.

i our example:
parent - vtnet1
vlan1 - vtnet1_vlan16
vlan2 - vtnet1_vlan20

the vlan1 and vlan2 are assigned, enabled and had problems.
first, i assigned the interface vtnet1 and leave it disabled.
nothing happens.
after this, i enabled the vtnet1 interface, with ip type static.

and after this ... tada, tcp is working fine.

hope, i could help someone.

kind regards,
ronny

[SillyPosition]

I have a similar setup - opnsense in proxmox, and I see tcp retransmissions in connectivity between my LAN network to my BGP network.
I dont have VLANs though, nor bridges, just WAN, LAN, and a BGP network advertised in my LAN network with a different subnet, which, any connections to it ends up with TCP retransmissions after a few packets.

Ronny - do you think there is anything that I could do to mitigate it? It sounds awfuly similar in behavior to yours, just that Im without VLANs.

[spark5]

hi ... sounds, like there is something other ... but ok

first, look at Interfaces: Settings, the first 3 boxes
Hardware CRC    Disable hardware checksum offload
Hardware TSO    Disable hardware TCP segmentation offload
Hardware LRO    Disable hardware large receive offload

try to disable this. i had a lot of trouble with bgp in the past, using tso.
there is also something with the realtek nic, but not in proxmox setup.

actually, i have no idea

what kind of network card you chosse, virtio?
you do not use bridges? how is your setup looking?

ronny

[SillyPosition]

All three are already disabled (checkboxes are checked)
I use PCI passthrough to two network interfaces, directly managed by opnsense, so no virtio at all.
Simply two nics, igb0 assigned as LAN, and igb1 assigned as WAN.
No bridge for LAN since it is the only physical network card used for LAN, so I couldnt find reason for this at all (Should I create a bridge for only one nic?)

[hmronline]

ok, got this working by doing this:

* Go to Interfaces, Assignments.
* Add a new Interface, select the parent interface you use for the existing VLANs (vtnet0 in my setup). In Description field you can put anything, ie PARENT.
* Go to that new Interface (PARENT), check the "Enable Interface" option, and Save it.
* Now click the red/orange "Apply changes" button and that's it. Everything is working as expected.

I've later removed this unnecessary Interface and it continued working fine until I've rebooted.
It seems you can not delete this PARENT interface to keep everything working properly.

I do not fully understand why this PARENT interface is now required (even though hardware features are disabled in Interfaces/Settings and no other settings are applied into that interface) and why it is not fixed in the upgrade scripts, so it would be great if someone could further explain.

Thanks again!

[mimugmail]

Did you assign parent interface when using Vlans?

Like I asked .. did you assign parent vlans ..

You need to assign the parent, enable (without IP) and you are good.

[Stumie]

Creating a parent interface also worked for me. Thank you spark5 and hmronline.

But I'm still a bit puzzled, why it's now relevant, when it wasn't before...