Routed IPSec BGP Invalid

[mrwizardno2]

Hi all,

I've been using opnsense for quite a while. Previously, I had a routed IPSec tunnel set up to Azure (using the documentation that was provided on the OPNSense site) that exchanged routes via BGP. This has continued to work fine through multiple updates - until I applied 21.1.3_3.

Now the tunnel comes up but the advertised BGP routes from Azure are marked invalid. I'm confused to why this stopped working - did the behavior change? How can I sort out what's happening? I can see in the firewall logs that Azure is sending me traffic (AD, DNS queries, etc) but I can't route back the other way.

Thanks in advance. This has really messed up my lab (and my SO is annoyed that some of the home automation stuff isn't working!) for the last week or two!

[mimugmail]

What was the last known working version?

[mrwizardno2]

I believe I was on 21.1 just before upgrading - but I am not 100% positive on the details there.

[mrwizardno2]

This morning I deleted my configuration in Azure as well as OPNSense and re-ran through the documentation on the Docs site (https://docs.opnsense.org/manual/how-tos/ipsec-s2s-route-azure.html). The documentation does not call out creating the interface (it appears to assume it is present, though) and setting its Dynamic Gateway Policy. You get a generic complaint about setting an IP on an interface where the network can not be determined without doing so. So the docs need updated at a minimum to include this step.

As far as the remainder of the configuration - it does not appear to work unless there is something more that needs done. I assume that at minimum a static /32 route must be added in order for OPNSense to understand how to get to the Gateway Subnet (which is outside the IPSec tunnel addresses). What's interesting is that creating that route under System > Routes > Configuration never shows up in the list of routes under Routing > Diagnostics > General > IPv4. I can see from Azure that BGP announced my local networks and they show up in Azure's routing table for my VNet Gateway, but nothing shows up in my local Routing > Diagnostics > General > IPv4. What am I missing?

[mrwizardno2]

I tried again today. No dice. So, anyone else have any thoughts?

[carpenike]

Howdy!

Did you ever get this working? I believe the "issue" is that the network provided by Azure over the BGP connection has a "next hop" of the VPN Gateway inside of Azure's VNET. This is not local to Opnsense as the VPN has it's own routed subnet (10.111.1.1 in the docs)

[mrwizardno2]

Heya!  No, sadly I couldn't get it to work. I redeployed to use tunnel / policy instead as a workaround.

Before I gave up I noticed the lack of a route to the VPNGW and added a static route for the VPN gateway - would that not have solved the problem? I also tried modifying the tunnel addresses to be the local IPs (LAN IP local, VPN GW IP for Azure) that didn't seem to work either even though BGP data was exchanged. Baffled as to what happened - as this all worked previously.

Not sure if it's related, but there was this discussion regarding route-based IPSEC connections on GitHub: https://github.com/opnsense/docs/pull/279#pullrequestreview-632969636

[olk2233]

Hi mrwizardno2

I have a similar issue or even the same.... I tested an OPNsense setup in February 2021 with the newest OPNsense version, if it's possible to create an IPSEC tunnel to Azure with BGP for dynamic routing. All worked like a charm (with the default Azure BGP peer IP address). In the phase2 of the IPSEC tunnel I used "Route-based", the local address 192.168.8.22 and the remote address 10.88.2.254 (the same as the default Azure BGP peer IP address) --> OPNsense sent the traffic to 10.88.2.254 successfully through the tunnel. Azure learned the routes from OPNsense and vice versa.

A few months later, I bought pyhsical OPNsense hardware and built the same configuration on the newest OPNsense version 21.1.8_1-amd64. I wasn't able to reach Azure with this configuration. OPNsense didn't create a route to Azure.

I tried the configuration with a Custom Azure APIPA BGP IP address. 169.254.21.89 on Azure and 169.254.21.88 on the OPNsense. I was able to ping the Azure IP 169.254.21.89 from the OPNsense. After configuring BGP, I also received the BGP routes from Azure. ICMP requests from an on-prem host to hosts within an Azure VNET found their target. But Azure does not learn the routes advertised by the OPNsense, so the way back doesn't work.

With tcpdump I saw, that the OPNsense does not send any BGP advertisements to Azure. I didn't spend to much time in this setup as I could solve it easily with static routes. But it would be interesting, if this hint helps you.

Best regards
olk

[lilsense]

May I ask, why not roll back to the older code and run your BGP... It's always recommended to test your config on another system prior to the upgrade, like a couple of VM's.

[davokr]

I was struggling with this today and the fix for me was to enable "Multi-Hop" on the BGP Peer.

I'm also using Azure's recommended tunnel IP which you can get from the "Download Configuration" on the connection in the Azure portal. Typically they'll tell you to use 169.254.0.1/30 on the opnsense side.

I think it's in the docs, but I also created a gateway for Azure's 169.254.0.2 IP on the IPSec interface, and added a static /32 route for the Azure VPN Gateway's BGP peer IP.

[nutcha]

before I give up I noticed there is no route to VPNGW and adding a static route for the VPN gateway - won't that solve the problem?
เล่นบาคาร่าออนไลน์
I also tried modifying the tunnel address to a local IP (Local LAN IP, VPN GW IP for Azure) that doesn't seem to work even though the BGP data is exchanged. this