Messages

and I set up query forwarding in unbound so that if the lookup is for an internal FQDN, unbound will forwards to port 53053 for DHCP server to resolve.

Is that really in the docs? This part looks weird to me.

https://docs.opnsense.org/manual/unbound.html#query-forwarding
https://docs.opnsense.org/manual/dnsmasq.html

What I'm currently seeing is that my DNS query from one site is indeed forwarded to the other end (tcpdump showing the traffic on enc0), but there is no response. I've tried fiddling with the firewall rules, but maybe I'm missing something else.

This looks like my situation: https://forum.opnsense.org/index.php?topic=48004.0

If I do:

`drill -4 host1.mysite.s2.internal` from s2, I get a successful response.
`host -4 host1.mysite.s2.interenal` from s2, I get a successful response.

`drill -4 host1.mysite.s2.internal` from s1, I get "not found: 3(NXDOMAIN)".
`host -4 host1.mysite.s2.interenal` from s1, sometimes I get "network error" other times I get a successful but empty record back.

I have made sure there is a domain entry for mysite.s2.internal on s2 under Services | Dnsmasq | Domains

I have two sites each with an OPNsense box communicating over IPSec VTI so LAN hosts can communicate. Each fw uses its own local FQDN for hosts so when an IP is leased through DHCP, they get a "host1.mysite.s1.internal" or "host1.mysite.s2.internal" depending on which LAN they're on (s1 vs s2) FQDN.

What I do at each site is I intercept port 53 traffic from LAN, re-route it to localhost for unbound, and I set up query forwarding in unbound so that if the lookup is for an internal FQDN, unbound will forwards to port 53053 for DHCP server to resolve. This is done per the docs with a destination NAT rule redirecting the traffic.

Would somebody walk me through how do I resolve FQDNs of hosts on the other end of the IPSec tunnel?

What I've tried so far is to put an entry in query forwarding table for the other site's FQDN, but I'm not sure what fw rules are needed or the config on the other end.

Having trouble redistributing routes between three devices so would appreciate pointing out my mistake, because presumably, it is me rather than the documentation...

Only seeing two routes from R1 populate on another R2, but not in R3.

I have:

R1 connects R2 & R3 via separate IPsec VTIs
R2 connects R3 with another VTI

R1 has two networks to populate in R2 & R3
R2 & R3 have one network each to populate into the adjacent two

It has also been a fiddle just getting OSPF to see the actual neighbors and exchange LSAs even though the firewall rules were present.

What I did was to put the routes on each router into a prefix list, assign that to a route map, then add the route map under 'Route Redistribution' in 'general' tab, per the docs. I had put R2 & R3 in area 0.0.0.1 and for the direction to R1 in 0.0.0.0.

If I do not specify the area under the interface, and I put it under the networks tab, there doesn't seem to be any way to associate an interface with an area so what is the point of using the networks tab? You can't put the area in both places per the help text warning.

Sounds logical to me : In order to connect two peers you need at least a /30 subnet :)

It's common to use /32 addresses for IPsec loopbacks and establish child over /32. What I'm disputing is the fact the traffic doesn't show on enc0 when /32 are used, not that they don't connect. The SA is installed for the peer - therefore they're connected (on /32). However, the peer reports ICMP echos, but enc0 doesn't show the traffic unless they're a /30. If pf isn't seeing traffic, that is a problem regarding pf rules.

Put it this way, I have zero trouble with GRE over IPsec when I take opnsense out of the picture and just do it between two devices of the same vendor.

[with a dedicated interface]

I guess your firewall rules are not sufficient for the GRE over IPsec to be permitted. But that is simply too much complexity for my tastes when IPsec with a dedicated interface exists and is well supported.

The firewall rules to allow GRE over IPsec are straightforward. Allow GRE protocol on IPsec interface, ICMP, and OSPF. Then make the GRE interfaces accept anything from anywhere just to test connectivity. Rules can be made more restrictive when the baseline connectivity is confirmed.

I'm looking at VTI. The guide says policies are attached to VTIs by reqid. So is this only an option for two opnsense firewalls? Or can the reqid be specified in the client's config somewhere irrespective of its vendor?

Also, I cannot interpret the topology in the doc (https://docs.opnsense.org/manual/how-tos/ipsec-s2s-conn-route.html#preparations). If both firewalls are separated by public internet, and have WAN IP's assigned dynamically, how does this change the required properties of the VTI config as shown?

I was hoping logging rules would at least provide you with some clues, but it seems that it isn't.

I suspect some traffic isn't being either picked up on enc0 or passed to pf and this is why I'm having difficulty pinning down the problem. See my reply to Cedrik.

Have you considered the suggestions from @Patrick & @Cedrik?

I actually used that how-to Cedrik wrote when I was setting this up. I don't know what to do. Maybe I'll look at the VTI, but the reason I didn't go with that option in the beginning was because of the caveat mentioned in the 'important' box in the doc (https://docs.opnsense.org/manual/vpnet.html#ipsec). I didn't want to go with VTI and later find this caveat to be an issue for my situation later down the line, when comparatively speaking, the GRE over IPsec seemed much simpler to do.

Thanks for the "no state" option. That's useful to see both in and out traffic in the live log. "keep state" doesn't show the return because it gets classed as related traffic in the state.

Heres the example setup from the documentation:

https://docs.opnsense.org/manual/how-tos/dynamic_routing_ospf.html#ipsec-failover-with-policy-based-tunnels-gre-and-ospf

I wrote that, but Im not an expert on GRE, but I didnt have issues with it not establishing with the firewall enabled.

Maybe what you have is a new issue, or something that wasnt noticed yet.

Though I know others who e.g., get Hurricane Electric IPv6 nets over GRE and there arent too many reports that something is wrong with that either.

So the simplest assumption is that the firewall rules are wrong somehow.

Yes, I don't think my firewall is operating correctly, but only because turning it off brings up the other side's GRE tunnel, but I feel like there could be other variables involved.

For example, let's ignore GRE, and just focus on IPsec. I have a number of peers and profiles for each peer. So I have multiple loopbacks for this purpose. If I set one of them to a /32 address and adjust the peer's loopback address the same, just like in your how-to, once the IPsec re-establishes, if I ping the opnsense IPsec loopback IP from the peer on the other /32, I cannot see the ICMP traffic on enc0 (tcpdump -nvv -i enc0) even though the peer reports receiving ICMP replies back! However, if I revert back to a /30 address for the peer and opnsense loopbacks, I can see the ICMP traffic on enc0!

If I have a floating firewall rule for ICMP and set "state type" = none, when the IP's are /32, I don't see anything in the live log, but I see plenty of IN and OUT when the IP's are /30.

Is that expected behaviour? Seems abnormal to me.

[Default block]

Since I'm running GRE over IPSec, do I need a GRE rule on WAN

In review, you don't mention having OPNsense logging of Default block being enabled. This is where I would expect PF to log the blocked packets arriving from the peer at the other end.

As you don't have an interface for IPsec, testing with a rule which is applied to all interfaces other than your WAN may help;

e.g. setting Invert interface in the rule I provided earlier. You may want to enable logging initially to verify the rule is being used.

Code Select Expand

@329 pass log quick on ! re0 inet proto gre all no state label "9dc1eb15-4dd3-44b1-adbc-04385d9f9955"

Under firewall | settings | advanced | logging, the option to log default deny/block rule is enabled.

My floating rule looks a little different to yours:

Code Select Expand

pass log quick on ! re0 inet proto gre from {any} to {any} no state label "<label>" # Test rule for GRE
pass log quick on ! re0 inet proto icmp from {any} to {any} keep state label "<label>" # Test rule for ICMP

Different after the protocol and before 'no state label'. I also added one for ICMP. What am I supposed to be seeing? All I see are hits against the 'IN' direction, nothing on 'OUT' except for OSPF to multicast on the GRE interface. The peer's GRE tunnel is still not coming up either.

Fyi, there is an interface for IPsec, it's called enc0 in ifconfig (from fw terminal) or just 'IPsec' in the GUI.

The most common scenario is either to use a policy based IPsec VPN in tunnel mode without a tunnel interface

'Without a tunnel interface'. Do you mean without something like GRE?

I've set up policy based IPsec, but IPsec is a unicast technology. To do dynamic routing over it where OSPF broadcasts to multicast address this needs to be encapsulated in GRE to pass over the IPsec tunnel. Hence why you run a ptp protocol over IPsec. It was also common at one time to run L2TP over IPsec too. Haven't you ever heard of that either?

Perhaps it is not common in 2026 but this detours from the issue at-hand. The issue being the GRE tunnel doesn't come up unless the firewall is disabled. That doesn't make sense -- I have appropriate firewall rules, so why does the firewall prevent communication? It should be far simpler to get to the explanation for why this is happening than what it will be to spend time removing all the config to set up route based IPsec with VTI.

WG is not possible at the moment.

I am using tunnel mode, not transport. That a problem?

That entirely depends on what you want to achieve and how the other side is configured.

In tunnel mode as used in most scenarios you would not run an additional GRE tunnel inside the IPsec tunnel. Why are you doing that?

Maybe because I'm inexperienced and haven't spent months reading the RFCs?

I'm running a site-to-site connection. I'm not sure what you mean by "inside", but my understanding is "GRE/IPSec" = "GRE over IPSec" meaning your local tunnel address is the IPSec loopback IP and the remote tunnel address is the IPSec loopback of the peer. Then you give the GRE interface its own private IP. Then you do dynamic routing via GRE. So for example:-

Device A:

loopback (for IPSec) - 10.22.22.1/30
GRE interface local address - 10.22.22.1/30
GRE interface remote address - 10.22.22.2/30
GRE interface IP - 172.25.1.1/30

Device B:

loopback (for IPSec) - 10.22.22.2/30
GRE interface local address - 10.22.22.2/30
GRE interface remote address - 10.22.22.1/30
GRE interface IP - 172.25.1.2/30

After the IPSec connection passes Phase 2 the GRE interface comes up (should...) and communication is via 172.25.1.0/30 network. Is this what you mean by running GRE 'inside' IPsec? Because if so, that's what I'm doing. What is the most common scenario you refer to?

[keep state]

I have fw rules on ipsec and WAN to allow GRE protocol.

I don't understand why you would be running GRE over an IPSEC tunnel, is this what you are doing?

Reading through one of the articles, during troubleshooting they mention not setting keep state for the GRE rule.

Have you tried a rule like this for GRE where you have Direction set to Both and set state to no state - advanced option under Stateful firewall?

Code Select Expand

@625 pass quick on re0 inet proto gre all no state label "c18f1b78-d4dc-46fb-9bb5-61c3ae3d8693"

Since I'm running GRE over IPSec, do I need a GRE rule on WAN, or only IPsec interface, and if only on the IPsec interface, does your suggestion still apply? My understanding is you have a GRE rule on the WAN if you're pointing GRE directly at your WAN IP. I'm not, as I'm using GRE through IPsec. IPsec certainly contacts the WAN IP on UDP 500/4500 though.

I can't tell whether you are having a joke here or not, but if not, you're telling me opnsense and neither freebsd have solved a 20 year old problem?...

I am telling you what I happen to know : That's all :)

GRE does not have ports. It's its own protocol on top of IP independent of TCP and UDP. Port 0 might be a historical frontend abstraction of some product for not having port numbers at all.

Could be... I can't remember anymore... Too long ago...

Also no further experience with GRE or IPSec :)

Appreciate the feedback nevertheless.

I don't understand why you would be running GRE over an IPSEC tunnel

At times when VTIs did not yet exist but you wanted a dedicated interface for your IPsec VPN, e.g. to run OSPF on it or similar, a common setup was to establish an IPIP or GRE tunnel and encrypt these packets in transport mode.

I am using tunnel mode, not transport. That a problem?

I can't tell whether you are having a joke here or not, but if not, you're telling me opnsense and neither freebsd have solved a 20 year old problem?...

...bring up peer's GRE interface?

As the title says, have been troubleshooting incoherent gre behaviour over last day or two, uttering bad words in frustration etc, and have finally found a common thread: if I disable the opnsense firewall (fw | advanced | miscellaneous), the peer's gre tunnel comes up immediately. If I then re-enable the firewall, the tunnel stays up. I can't accept this in a production environment.

I have fw rules on ipsec and WAN to allow GRE protocol.

What is the problem?

https://forum.opnsense.org/index.php?topic=6131.0 --- related?