Messages

Hi All,

I have the following VLAN interface configured.


My question is whether the VLAN tag 10 is added to packets (like a switch does) that go through this interface to the outside

or

Whether the interface only allows traffic with this tag (packets not tagged with 10 are dropped).

I have gone through the https://docs.opnsense.org/manual/other-interfaces.html, and its not clear to me.

Would appreciate if someone can clarify this.

Set VLAN untagged on the switch

This is possible, vCenter Distributed Switch can be configured.

omit VLAN tagging on OPNsense.

Sorry not sure I understand how to omit tagging on OPNSense. Do you mean by creating regular interfaces ?

Any way to have VLAN interface without tagging at firewall ?

Seems like double tagging problem, 1 from vCenter Distributed Switch, and 2nd from OPNSense interface.

The issue is OPNSense VLAN interfaces cannot be created without tags, or cannot be set as 0 so tagging can be set at Distributed Switch level only.

I set the Edge Uplink portgroups to trunking.



And firewall ARP table now has the interface attached.



Now both interfaces are in Established state, and BGP peering on all Edge Interfaces successfully.

Code Select Expand

edge1> vrf 2
edge1(tier0_sr[2])> get bgp neighbor summary
BFD States: NC - Not configured, DC - Disconnected
            AD - Admin down, DW - Down, IN - Init, UP - Up
BGP summary information for VRF default for address-family: ipv4Unicast
Router ID: 10.10.25.101  Local AS: 65000

Neighbor                            AS          State Up/DownTime  BFD InMsgs  OutMsgs InPfx  OutPfx

10.10.25.1                          65555       Estab 00:12:58     UP  46      20      12     4
10.10.26.1                          65555       Estab 00:12:58     UP  46      20      12     14

Thu Aug 24 2023 UTC 17:54:55.772

Code Select Expand

edge2> vrf 1
edge2(tier0_sr[1])> get bgp neighbor summary
BFD States: NC - Not configured, DC - Disconnected
            AD - Admin down, DW - Down, IN - Init, UP - Up
BGP summary information for VRF default for address-family: ipv4Unicast
Router ID: 10.10.25.102  Local AS: 65000

Neighbor                            AS          State Up/DownTime  BFD InMsgs  OutMsgs InPfx  OutPfx

10.10.25.1                          65555       Estab 00:15:18     UP  48      23      12     12
10.10.26.1                          65555       Estab 00:15:18     UP  51      23      12     6

Thu Aug 24 2023 UTC 17:57:02.232

Appreciate you taking the time, and thanks for the feed back, definitely helpful in trying to narrow down the issue.

Will definitely recheck anything associated with VLANs.

Sorry about that.

I wasn't sure how to get interface information from CLI.

Network Overview

This is a test environment, using 1 firewall, and all the Edge Node uplinks are setup for BGP Peering on the same firewall.

Hi,

Yes, there are 2 NSX-T Edges deployed, each with 2 Uplinks, and each uplink in a separate VLAN, its an Active/Active HA setup.

So :

Edge Node 1, Uplink 1 (10.10.25.101, VLAN 25), Uplink 2 (10.10.26.101, VLAN 26)

Edge Node 2, Uplink 1 (10.10.25.102, VLAN 25), Uplink 2 (10.10.26.102, VLAN 26)

Uplink 1 works with the firewall, while Uplink 2 does not.

10.10.25.1 and 10.10.26.1 are 2 VLAN interfaces that have the parent interface MAC address seen in the ARP table.

Ping from T0 VRF in Edge

Code Select Expand

edge1(tier0_sr[2])> ping 10.10.26.1
PING 10.10.26.1 (10.10.26.1): 56 data bytes
36 bytes from 10.10.26.1: Destination Host Unreachable
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
4  5  00 0054 0000   0 0000  40  01 3230 10.10.26.101  10.10.26.1

36 bytes from 10.10.26.1: Destination Host Unreachable
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
4  5  00 0054 0000   0 0000  40  01 3230 10.10.26.101  10.10.26.1

Traceflow

Both uplinks have the exact same configuration, and are in /24, and in the right VLAN as well.

Yesterday I did a firewall state table reset and the 2nd uplink started responding to ping normally, however, it still did not have any entry in the ARP Table, and upon reboot of the firewall and edge node the 2nd uplink went back to not responding to pings.

[I have Allowed Promiscuous Mode, MAC Address Changed, and Forged Transmits.]

Hi All,

The network is as follows with VLAN interfaces on the firewall.

VLAN 15 - 10.10.15.1
VLAN 25 - 10.10.25.1
VLAN 26 - 10.10.26.1

NSX-T Edge Node Management - 10.10.15.101
NSX-T Edge Node Uplink 1 - 10.10.25.101
NSX-T Edge Node Uplink 2 - 10.10.26.102

I have OPNSense running as a VM on ESXi, and NSX-T Edge Node VM with 3 interfaces, Management, Uplink 1, Uplink 2.

I have Allowed Promiscuous Mode, MAC Address Changed, and Forged Transmits.

There are no firewall rules denying any traffic.

The problem is Edge Node Uplink 1 (10.10.25.101) can ping the firewall interface and vice versa, but interface 2 (10.10.26.102) cannot ping the firewall interface.

I did a packet capture on the firewall and the firewall interface 10.10.26.1 is sending ARP Broadcast when traceroute was performed from 10.10.26.102.





I have checked the ARP table and MAC address from Uplink 1 is added but Uplink 2 there are no entries from the Edge Node. I have set up another VM (10.10.26.225) on the 10.10.26.0 network and it can reach firewall interface (10.10.26.1) without any issues, and has entries from the VM (10.10.26.225) as well.



Anyone experienced with this issue, or knows what's going on, or what can be checked ?

Sorry been on this for about a week now.

Found the issue to be name of Prefix List and Route Maps, both were same.

Changed them to different names, and now neighbors populating routes.

I'm not using IPv6, so I disabled it completely.

IPv4 Is set as static, but once changed (in my case I'm changing it from 192.168.3.10 to 192.168.9.21) the LAN is inaccessible because WAN IPv6 (shows as disabled in the Web-UI) but remains enabled in the CLI.

When the firewall VM is rebooted and IPv4 address is attempted to change and it does not prompt for DHCPv6 LAN is accessible with the new IP address.

[change the LAN interface IP address]

Hi,

I do a lot of testing on firewall deployment, and configurtions in a virtual environment.

23.1_6 does not have this issue, however, both 23.1.11_1 and 23.7 have this issue.

If I disable DHCPv6 on WAN interface, shutdown the firewall, export to OVF or OVA for import elsewhere, then import it, then change the LAN interface IP address, the LAN page is inaccessible.

Of note is that when changing LAN IP address it asks for DHCPv6 on WAN even though its disabled, it does not do this in 23.1_6.

Anyone knows of a solution/fix for this, tried many way to search but could not find.

I tested this configuration in version 23.1.9, and it worked fine, after upgrading a new install it does not work.

The problem is I can't install FRR in 23.1.9 without upgrading, if anyome knows how to do that, that would be great too..

Thanks..

[Router A]

Hi,

I'm losing my mind having setup FRR BGP on 2 routers.

Router A - 192.168.9.21
Networks - 192.168.11.0/24

Router B - 192.168.9.31
Networks - 10.10.13.0/24, 10.10.15.0/24, 10.10.17.0/24

I can see in the routing table neither of the routes are being populated.
Router A


Router B


Router A Configuration

Code Select Expand

Building configuration...

Current configuration:
!
frr version 7.5.1
frr defaults traditional
hostname MFW.localdomain
log syslog notifications
!
router bgp 65551
no bgp ebgp-requires-policy
no bgp default ipv4-unicast
neighbor 192.168.9.31 remote-as 65555
neighbor 192.168.9.31 bfd
neighbor 192.168.9.31 update-source em0
!
address-family ipv4 unicast
  redistribute connected
  neighbor 192.168.9.31 activate
  neighbor 192.168.9.31 prefix-list All in
  neighbor 192.168.9.31 prefix-list All out
  neighbor 192.168.9.31 route-map All in
  neighbor 192.168.9.31 route-map All out
exit-address-family
!
address-family ipv6 unicast
  redistribute connected
exit-address-family
!
route-map All permit 10
match ip address prefix-list All
!
line vty
!
bfd
peer 192.168.9.31
!
!
end

Router B Configuration

Code Select Expand

Building configuration...

Current configuration:
!
frr version 7.5.1
frr defaults traditional
hostname SFW.localdomain
log syslog notifications
!
router bgp 65555
no bgp ebgp-requires-policy
no bgp default ipv4-unicast
neighbor 192.168.9.21 remote-as 65551
neighbor 192.168.9.21 bfd
neighbor 192.168.9.21 update-source vmx0
!
address-family ipv4 unicast
  redistribute connected
  neighbor 192.168.9.21 activate
  neighbor 192.168.9.21 prefix-list All in
  neighbor 192.168.9.21 prefix-list All out
  neighbor 192.168.9.21 route-map All in
  neighbor 192.168.9.21 route-map All out
exit-address-family
!
address-family ipv6 unicast
  redistribute connected
exit-address-family
!
route-map All permit 20
match ip address prefix-list All
!
line vty
!
bfd
peer 192.168.9.21
!
!
end

Both routers are advertising

Router A

Code Select Expand

BGP neighbor is 192.168.9.31, remote AS 65555, local AS 65551, external link
Hostname: SFW.localdomain
  BGP version 4, remote router ID 192.168.9.31, local router ID 192.168.11.2
  BGP state = Established, up for 00:00:03
  Last read 00:00:02, Last write 00:00:02
  Hold time is 180, keepalive interval is 60 seconds
  Neighbor capabilities:
    4 Byte AS: advertised and received

Router B

Code Select Expand

BGP neighbor is 192.168.9.21, remote AS 65551, local AS 65555, external link
Hostname: MFW.localdomain
  BGP version 4, remote router ID 192.168.11.2, local router ID 192.168.9.31
  BGP state = Established, up for 00:00:05
  Last read 00:00:04, Last write 00:00:04
  Hold time is 180, keepalive interval is 60 seconds
  Neighbor capabilities:
    4 Byte AS: advertised and received

Nt sure where the problem lies, seeing the below in logs.

Router A

Code Select Expand

2023-08-07T20:29:35 Notice zebra client 31 says hello and bids fair to announce only vnc routes vrf=0
2023-08-07T20:29:35 Notice frr_carp FRR received carp configuration event.
2023-08-07T20:29:35 Notice zebra client 28 says hello and bids fair to announce only bgp routes vrf=0
2023-08-07T20:29:35 Error bgpd [EC 100663304] ERROR: No such command on config line 34: ip prefix-list All seq 10 permit Any
2023-08-07T20:29:34 Error bgpd [EC 100663304] ERROR: No such command on config line 34: ip prefix-list All seq 10 permit Any

Router B

Code Select Expand

2023-08-07T20:36:18 Error bgpd [EC 100663299] %bgp_getsockname() failed for accept from_peer 192.168.9.21 fd 22 (peer fd 19)
2023-08-07T20:36:18 Error bgpd [EC 100663299] Can't get remote address and port: Socket is not connected
2023-08-07T20:36:18 Error bgpd [EC 33554465] 192.168.9.21 [FSM] Ignoring event BGP_Start in state Connect, prior events ConnectRetry_timer_expired, ConnectRetry_timer_expired, fd 22
2023-08-07T20:28:55 Error bgpd [EC 100663304] ERROR: No such command on config line 34: ip prefix-list All seq 20 permit Any
2023-08-07T20:28:55 Error bgpd [EC 100663304] ERROR: No such command on config line 34: ip prefix-list All seq 20 permit Any
2023-08-07T20:28:17 Warning zebra [EC 4043309122] Client 'bfd' encountered an error and is shutting down.
2023-08-07T20:28:17 Warning zebra [EC 4043309122] Client 'bgp' encountered an error and is shutting down.

Any thoughts ?!