Messages

Thanks,

I have captured packets on the firewall when VLAN 26 cannot ping its firewall interface.



The firewall parent interface sends an ARP Broadcast.

Appreciate all the help, will go through, and try and rebuild again, must be something I missed.

Thanks again..

Just adding..

As you can see the Distributed Switch Portgroup has 2 VLANs 25 and 26, only VLAN 26 faces this issue of not reaching its firewall interface, all traffic in VLAN 25 reaches its firewall interface without any issues.

Point being VLAN 25 faces no issue at all in any configuration, but VLAN 26 faces this issue, even though both uplink configurations are the same, except the VLAN ID.

[Network Diagram]

Could you do a quick and simple diagram of the two configurations, please?

Network Diagram



Distributed Switch Uplink Portgroup with All VLANs allowed (works)



Distributed Switch Uplink Portgroup with only VLANs 26, and 24 allowed (not working)

[when the Edge Uplink Portgroups are configured with all VLANs [0-4094], the Edge vNIC carrying VLAN 26 traffic can reach the firewall interafce]

That should work, but if you expect VLANs on OPNsense to be connected in sny way across multiple ports, then that is your problem. OPNsense is not a switch. There is no "fabric". You would need to create bridge interfaces manually.

E.g. VLAN 10 on igb0, VLAN 10 on igb1, and an untagged igb2 are NOT connected in any way. As I wrote you could create a bridge with all these interfaces as members.

I understand the VLANs are not connected in anyway even though they belong to the same parent interface.

I'm just trying to understand why when the Edge Uplink Portgroups are configured with all VLANs [0-4094], the Edge vNIC carrying VLAN 26 traffic can reach the firewall interafce but when specific VLANs are added in the portgroup [26, 24], the same Edge Uplinks that allow VLAN 26 can't reach the same firewall interface.

Just use the physical interface without the VLAN.

What exactly do you want to achieve?

Thanks, this is what I plan to do now.

Yes, exactly.

In order to access the untagged port on OPNsense you use the interface itself rather than a VLAN.

However, same question from me: Why?

Thanks, I though it was possible to set up a VLAN interface without tagging.

I was trying to setup NSX Edge to send traffic over Uplinks for all VLANs over Trunk Ports (works), and for specific VLANs over Trunk ports (does not work) as I found it should work both ways, so I'm trying to understand why it does not work with the 2nd way, solely for learning purpose. I'm sure its a configuration issue, so trying to troubleshoot it but couldn't find where the issue was.

Please don't change the scope from the original question or at least don't require me to comment on the moving goal post. I just want to help with a simple question and not troubleshoot packet dumps.

Apologies, you are mistaken, the original question is related to what ever has been added.

I don't recall asking to troubleshoot packet dumps.

Anyway, no worries, appreciate your help..

A packet capture on firewall reveals the firewall Parent Interface sends an ARP Broadcast even though the VLAN interface 10.10.26.1 and 10.10.26.101 are on the same network and directly connected to it.

[(opt9 in the diagram)]

VLAN 26 tagged traffic will be ignored unless you add a VLAN interface for it.

The firewall has a VLAN interface with IP address 10.10.26.1 for VLAN tag 26 (opt9 in the diagram).

[but only traffic from VLAN 25 can reach the firewall VLAN interface 10.10.25.1. Traffic from 10.10.26.101 or 10.10.26.102 cannot reach its gateway 10.10.26.1.]

So the reason for this question is the below.

This is how the network is.



Below is the Portgroup from Distributed Switch where Edge_UL1 and Edge_UL2 are VLAN trunks carrying only traffic with VLAN ID 25 and 26, but only traffic from VLAN 25 can reach the firewall VLAN interface 10.10.25.1. Traffic from 10.10.26.101 or 10.10.26.102 cannot reach its gateway 10.10.26.1.



However, when I change both the Edge_UL1 and Edge_UL2 Portgroups to carry all VLAN traffic as below.



Everything works fine, all IP addresses from 10.10.25.0/24 and 10.10.26.0/24 can ping their respective gateway of 10.10.25.1 and 10.10.26.1 on the firewall VLAN interfaces.

So the question :

When Distributed Switch Portgroup is adding VLAN 26 tag to traffic heading to the firewall interface 10.10.26.1, is the firewall stripping the VLAN tag 26, or adding another VLAN 26 tag, or what does it do because the traffic never reaches the gateway ?

Basically both.

Thanks,

If I understood your reply correctly, any packets coming to this firewall interface will have VLAN 10 tagged to it by the firewall if its leaving the interface to the outside only if its not already tagged by the switch.

Any packets coming with tag other than 10 will be dropped right ?

Hi All,

I have the following VLAN interface configured.


My question is whether the VLAN tag 10 is added to packets (like a switch does) that go through this interface to the outside

or

Whether the interface only allows traffic with this tag (packets not tagged with 10 are dropped).

I have gone through the https://docs.opnsense.org/manual/other-interfaces.html, and its not clear to me.

Would appreciate if someone can clarify this.

Set VLAN untagged on the switch

This is possible, vCenter Distributed Switch can be configured.

omit VLAN tagging on OPNsense.

Sorry not sure I understand how to omit tagging on OPNSense. Do you mean by creating regular interfaces ?

Any way to have VLAN interface without tagging at firewall ?

Seems like double tagging problem, 1 from vCenter Distributed Switch, and 2nd from OPNSense interface.

The issue is OPNSense VLAN interfaces cannot be created without tags, or cannot be set as 0 so tagging can be set at Distributed Switch level only.

I set the Edge Uplink portgroups to trunking.



And firewall ARP table now has the interface attached.



Now both interfaces are in Established state, and BGP peering on all Edge Interfaces successfully.

Code Select Expand

edge1> vrf 2
edge1(tier0_sr[2])> get bgp neighbor summary
BFD States: NC - Not configured, DC - Disconnected
            AD - Admin down, DW - Down, IN - Init, UP - Up
BGP summary information for VRF default for address-family: ipv4Unicast
Router ID: 10.10.25.101  Local AS: 65000

Neighbor                            AS          State Up/DownTime  BFD InMsgs  OutMsgs InPfx  OutPfx

10.10.25.1                          65555       Estab 00:12:58     UP  46      20      12     4
10.10.26.1                          65555       Estab 00:12:58     UP  46      20      12     14

Thu Aug 24 2023 UTC 17:54:55.772

Code Select Expand

edge2> vrf 1
edge2(tier0_sr[1])> get bgp neighbor summary
BFD States: NC - Not configured, DC - Disconnected
            AD - Admin down, DW - Down, IN - Init, UP - Up
BGP summary information for VRF default for address-family: ipv4Unicast
Router ID: 10.10.25.102  Local AS: 65000

Neighbor                            AS          State Up/DownTime  BFD InMsgs  OutMsgs InPfx  OutPfx

10.10.25.1                          65555       Estab 00:15:18     UP  48      23      12     12
10.10.26.1                          65555       Estab 00:15:18     UP  51      23      12     6

Thu Aug 24 2023 UTC 17:57:02.232

Appreciate you taking the time, and thanks for the feed back, definitely helpful in trying to narrow down the issue.

Will definitely recheck anything associated with VLANs.