troubleshoot firewall logs

[caplam]

Hello,

I recently transitioned to opnsense and I'm in the process of segmenting my network.
I installed opnsense on a N100 box with 6 2,5G network ports.
igc0 is assigned to wan
igc1 is assigned to lan
igc2&3 are assigned to lagg0 and vlans 9,10,20,30,40 are assigned to lagg0.
igc4&5 are unused.

i have 3 wifi ssid broadcasted by unifi ap.
one is on lan network
one is on vlan iot (10)
one is on vlan guest (40)
The ap are on switch ports with lan as native and vlan iot and guest tagged.

I have some old smart wifi outlets. They are on lan for now but I'm going to assign it to iot vlan.

So i was looking at firewall logs and saw something strange.
the 2 outlets have probably hardcoded dns.
But one of the 2 outlet make request to 1.1.1.1:53 and then to 192.168.2.1:53 (the firewall ip)
The most strange is that in the firewall logs the request occur on lagg0 interface and I don't understand why.
The outlet has an ip (assigned by a static mapping in khea) in lan subnet and shouldn't make requests on lagg0 which is tied to vlans.

How could this happen ?
The other outlet which is the same model communicate on the right interface (igc1)

[dseven]

It appears that your switch is sending those packets untagged over the LAGG, so I think you need to be looking at your switch config?

[caplam]

Thanks.
I checked port config on the switch and the trunk only carry tagged vlan. But its vlan native is 1 which is on LAN network.
i figured out there might be a mismatch in config of lacp
But I don't know how to configure it.

Opnsense let me choose between:
L2: src/dst Mac and optional vlan number
L3: src/dst adress for ipv4 or ipv6
L4: src/dst port for tcp/udp/sctp

on the switch I can select :
1: src-mac
2: dst-mac
3: dst-src-mac
4: src-ip
5: dst-ip
6: dst-src-ip
7: dst-src-mac-ip
8: ingress port

can I assume for example that src/dst mac on opnsense would  match dst-src-mac on switch ?

[Seimus]

This config for LACP only adjust the loadbalancing of traffic across links bundled in LAGG LACP. IT doesn't need to be the same on both ends.

Saying that, all links that are part of the SAME LAGG group needs to have the same VLANs configured on BOTH LAGs on BOTH ends.

If you have an IP/frame that is coming without a VLAN TAG, that means the device that should TAG the frame is not doing it.

Regards,
S.

[dseven]

I checked port config on the switch and the trunk only carry tagged vlan. But its vlan native is 1 which is on LAN network.

That's the problem - if your LAN is the native VLAN on the trunk, it will get sent over the trunk untagged. Can you set a different native VLAN on the trunk?

i figured out there might be a mismatch in config of lacp
But I don't know how to configure it.

Opnsense let me choose between:
L2: src/dst Mac and optional vlan number
L3: src/dst adress for ipv4 or ipv6
L4: src/dst port for tcp/udp/sctp

on the switch I can select :
1: src-mac
2: dst-mac
3: dst-src-mac
4: src-ip
5: dst-ip
6: dst-src-ip
7: dst-src-mac-ip
8: ingress port

I've not actually done this, but I would expect OPNsense's "L2" to align with switch option 3, and "L3" to align with option 6. "L4" might align with option 8, but less sure about that one.

[caplam]

it appears I have to review my switch config.
I have unifi switches (and their vlan management is quite confusing) and other cheap Chinese switches.

So if a trunk port is not supposed to carry the LAN network, which would be its native vlan or pvid ? can I set it to any vlan number ?
i suppose not as the vlans carried in the trunk have to be tagged.

[Patrick M. Hausen]

So if a trunk port is not supposed to carry the LAN network, which would be its native vlan or pvid ? can I set it to any vlan number ?

Create a VLAN that you do not use anywhere. Back when I ran my own data centre with dozens of Cisco switches I created VLAN 999 and named it "native-dummy". Use that as the PVID on the trunk port and carry VLAN 1 (LAN) tagged.

[caplam]

i configured the switch lag port to carry all my vlans with tag and native vlan is 9 which exists but not used for the moment ( will be later when I will finish moving all my devices to vlans)
I don't see anymore requests on lagg from an address on LAN.
So I guess it's solved but still I have to review the config from other switches (I have 6 in total).


edit: I still have traffic blocked but it's ipv6 and much more difficult to track.
I also have huge number of packets from 33.33.33.33 which are blocked.

[Patrick M. Hausen]

So I guess it's solved but still I have to review the config from other switches (I have 6 in total).

You can leave the inter-switch links with "native VLAN" 1 just fine. Tagged/untagged is only relevant on a link, not on an entire broadcast domain. You could use e.g. 5 ports with one untagged VLAN each on two switches and run 5 cables in parallel - all the other devices won't notice.

So with FreeBSD being a bit particular in that regard, use your OPNsense -- switch link without an untagged network but just leave the rest as Unifi seems to prefer, so e.g. mapping of SSIDs to VLANs and provisioning of APs over VLAN 1 continues to work.

Really do create that extra dummy VLAN - it's sort of embedded documentation :)

HTH,
Patrick

[caplam]

Thank you for your advice which I followed as I started populating my vlans with devices. I created dummy vlan 99 and use it only on the link (lacp carrying all the vlans except lan) between opnsense and the first switch.

[Patrick M. Hausen]

Thank you for your advice which I followed as I started populating my vlans with devices. I created dummy vlan 99 and use it only on the link (lacp carrying all the vlans except lan) between opnsense and the first switch.

The point of the dummy VLAN 99 is that now your LCAP link can also carry LAN aka VLAN 1 tagged on that particular link. I don't quite get if that is what you want to do or not.

On the Unifi side create VLAN 99 and assign it as PVID to the LACP port/bundle. On OPNsense do not create VLAN 99, but create VLAN 1 and assign to LAN. This way all VLANs including 1 are tagged over that link.

[dseven]

The point of the dummy VLAN 99 is that now your LCAP link can also carry LAN aka VLAN 1 tagged on that particular link. I don't quite get if that is what you want to do or not.

It's not about wanting to carry it tagged, but about needing to avoid carrying it untagged.

[Patrick M. Hausen]

It's not about wanting to carry it tagged, but about needing to avoid carrying it untagged.

Yeah, sure. My point exactly. The dummy VLAN is only for whatever is on the other side of the link if that particular device insists on having a native/untagged VLAN on every 802.1q enabled port.

Kind regards and happy holidays,
Patrick