Bridged OpenVPN DOWN after reboot

[onet]

Hi all,

Please see second post for updated info.

Summary: Bridged OpenVPN server/client config network access only possible after re-init (save-apply) of the bridge after every reboot. Connection is up, but no traffic.

--> Since 23.7 we are facing this issue that after every reboot a bridged OpenVPN tunnel only works after going into Interfaces - <BridgeIF> -- save -- apply without changing any config, basically "re-initializing" the bridge I would assume.

Before doing the re-init, a client can establish a connection, but does not get any network connectivity.

It's two (legacy) OpenVPN servers bridged to a physical interface, assigning a static IP with ifconfig in the client config.

On 23.1, this worked fine, so we are forced to stay on that version at the moment.

I read about an older,  similar issue that has been resolved before, where the interface was not part of the bridge. But as far as I can see, it's part of the bridge already.

before reinit

Code: [Select]

bridge0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: 4_INSTRUMENT (opt2)
        options=0
        ether 58:9c:fc:10:ff:f1
        inet 192.168.150.254 netmask 0xffffff00 broadcast 192.168.150.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: ovpns1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 10 priority 128 path cost 2000000
        member: ovpns2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 9 priority 128 path cost 2000000
        member: igb2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 55
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>

In a diff with the working after, only thing that changed was the cost of igb2 --> 20000

Any ideas on how to get to the root of the issue?

Thanks!

Update: In the logs, I can see the following entries after reboot, but I don't understand why the interfaces show up as member with ifconfig?!

Code: [Select]

2024-10-08T08:44:32 Notice kernel <118> 4_INSTRUMENT (bridge0) -> v4: 192.168.150.254/24
2024-10-08T08:44:28 Notice kernel <6>bridge0: link state changed to UP
2024-10-08T08:44:23 Notice kernel <6>bridge0: link state changed to DOWN
2024-10-08T08:44:23 Notice kernel <6>bridge0: link state changed to UP
2024-10-08T08:44:23 Notice kernel <6>bridge0: Ethernet address: 58:9c:fc:10:ff:f1
2024-10-08T08:44:23 Notice opnsense /usr/local/etc/rc.bootup: Device bridge0 required for opt2, configuring now
2024-10-08T08:44:23 Notice opnsense /usr/local/etc/rc.bootup: Device ovpns1 cannot be added to non-existent bridge0, skipping now.
2024-10-08T08:44:22 Notice opnsense /usr/local/etc/rc.bootup: Device ovpns2 cannot be added to non-existent bridge0, skipping now.
2024-10-08T08:44:22 Notice opnsense /usr/local/etc/rc.bootup: Device igb2 cannot be added to non-existent bridge0, skipping now.

[onet]

Hi all

I spent quite some time investigating, checking all related forum entries and github issues I could find, but to no avail.

But I have an update on the investigation.

The most interesting observation:
--> I have several devices of same type plus some VMs. One only one of the devices, one of the two VPN Interfaces that are bridged on the same bridge shows as UP after reboot and works normally. On other devices (even same hardware with exact same config.xml loaded), both interfaces are down after reboot.

This seems like some kind of race condition to me.

Alss interesting: broadcast traffic, like a broadcast ping, comes through to the VPN client even on the interface shown as down. On both servers, the VPN clients can connect, but traffic is only normal on ovpns2/opt7.

I don't know of anything else to check/do tbh.

Any other information needed? Should I create an issue on github?

Any help is appreciated.

I went through the logs and there are different entries for both:
ovpns1/opt6 -- NOT WORKING after reboot

Code: [Select]

2024-11-03T10:27:31 Notice kernel <118> INSTRUMENT_VPN_UDP (ovpns1) ->
2024-11-03T10:27:28 Notice kernel <118>Reconfiguring IPv4 on ovpns1
2024-11-03T10:27:28 Notice kernel <6>ovpns1: link state changed to UP
2024-11-03T10:27:28 Notice opnsense /usr/local/etc/rc.newwanip: IP renewal deferred during boot on 'ovpns1'
2024-11-03T10:27:27 Notice kernel <6>ovpns1: promiscuous mode enabled
2024-11-03T10:27:27 Notice kernel <6>tap1: changing name to 'ovpns1'
2024-11-03T10:27:27 Notice opnsense /usr/local/etc/rc.bootup: Device ovpns1 cannot be added to non-existent bridge0, skipping now.
2024-11-03T10:27:27 Notice opnsense /usr/local/etc/rc.bootup: Device ovpns1 required for opt6, configuring now
2024-11-03T10:27:26 Notice kernel <6>ovpns1: link state changed to DOWN


Code: [Select]

2024-11-03T10:27:28 Notice opnsense /usr/local/etc/rc.newwanip: Failed to detect IP for interface opt6
2024-11-03T10:27:27 Notice opnsense /usr/local/etc/rc.bootup: Device ovpns1 required for opt6, configuring now

ovpns2/opt7 -- WORKING after reboot

Code: [Select]

2024-11-03T10:27:31 Notice kernel <118> INSTRUMENT_VPN_TCP (ovpns2) ->
2024-11-03T10:27:28 Error opnsense /usr/local/etc/rc.linkup: The command `/sbin/ifconfig 'bridge0' addm 'ovpns2'' failed to execute
2024-11-03T10:27:28 Notice opnsense /usr/local/etc/rc.linkup: Device ovpns2 requires reload for opt7, configuring now
2024-11-03T10:27:28 Notice kernel <118>Reconfiguring IPv4 on ovpns2
2024-11-03T10:27:28 Notice kernel <6>ovpns2: link state changed to UP
2024-11-03T10:27:28 Notice opnsense /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for opt7(ovpns2)
2024-11-03T10:27:28 Notice opnsense /usr/local/etc/rc.newwanip: IP renewal deferred during boot on 'ovpns2'
2024-11-03T10:27:27 Notice kernel <6>ovpns2: promiscuous mode enabled
2024-11-03T10:27:27 Notice opnsense /usr/local/etc/rc.bootup: Device ovpns2 cannot be added to non-existent bridge0, skipping now.
2024-11-03T10:27:27 Notice kernel <6>tap2: changing name to 'ovpns2'
2024-11-03T10:27:27 Notice opnsense /usr/local/etc/rc.bootup: Device ovpns2 required for opt7, configuring now
2024-11-03T10:27:26 Notice kernel <6>ovpns2: link state changed to DOWN


Code: [Select]

2024-11-03T10:27:29 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure newwanip_map:rfc2136 (,opt7)
2024-11-03T10:27:28 Notice opnsense /usr/local/etc/rc.newwanip: Failed to detect IP for interface opt7
2024-11-03T10:27:28 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (execute task : ipsec_configure_do(,opt7))
2024-11-03T10:27:28 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (,opt7)
2024-11-03T10:27:28 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: entering configure using opt7
2024-11-03T10:27:28 Notice opnsense /usr/local/etc/rc.linkup: Device ovpns2 requires reload for opt7, configuring now
2024-11-03T10:27:28 Notice opnsense /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for opt7(ovpns2)
2024-11-03T10:27:27 Notice opnsense /usr/local/etc/rc.bootup: Device ovpns2 required for opt7, configuring now

Despite the ERROR, the ovpns2 also shows as member of the bridge and traffic goes through as expected.