Re: Multi WAN (was DoT in combo) - allow dpinger in firewall?

[lar.hed]

So I read the web page documentation about Multi-WAN:
https://docs.opnsense.org/manual/how-tos/multiwan.html

And then I started to think about DNS settings and DoT (DNS over TLS within Unbound), and I just got into that it is not possible from my point of view to enter the correct information either on the DNS settings on " System %u2023 Settings %u2023 General " (does not allow @853 after IP address so no way to enter correct info about DoT servers) and in the web settings for Unbound " Services: Unbound DNS: Miscellaneous " there is no way to set the gateway as described in MultiWAN page above.

So how am I supposed to set up: Multi-WAN (failover from fiber to LTE in my case) and DoT - or is this combo not possible?

[mimugmail]

You need also gateway switching active where system default gateway always points to primary, when this goes down it swithes to second. Its only failover and not balancing, but DNS traffic so small, should be fine though

[lar.hed]

Not sure I follow, or rather if I do the "gateway switching active where system default gateway always points to primary" I will lose DNS. And I have not even tried DoT yet. Turning gateway switching on, fail DNS requests - turning off, DNS requests works. However if I play around, back forth and back again or something, DNS simply never works again. And everytime I make a change Unbound takes 53 second to restart before anything can happen (if it will happen that is).

I simply do not get this to work - and I am just trying to get this working with everything in default settings. Just followed the how-to multiwan (which also mentions Default Gateway Switching) - and I still fail....

[lar.hed]

Okay - I think I got what is not working for some reasone:

WAN_FTTH (Primary) - Fiber To The Home
WAN_LTE (Secondary) - LTE mobile modem as backup
Default Gateway Switching IS active for this test.

Starting from reboot, WAN_FTTH marked as active (under System -> Gateway -> Settings) - everything works.
Pulling the ethernet cable for the interface for WAN_FTTH, failover to WAN_LTE is marked as active - everything works.
Attaching the ethernet cable I just pulled, WAN-FTTH is marked as active - DNS resolution does not work.
Pulling the ethernet cable for WAN_LTE - NOW everything works on WAN_FTTH.
I could also reboot the firewall to get it working - but that seems a bit wrong in my book.

So yes it fails over, but it can not recover - what am I doing wrong? And I am sure everything is as the how-to describes it. I can take screendumps if requested, just tell me what?

Oh and NO I have not gotten to DoT yet...

[mimugmail]

Screenshot of System : Gateways : Single

[lar.hed]

A few screenshots then :-)

[lar.hed]

And one final screenshot...

[mimugmail]

Both Gateways have the same IP??? This cant work

[lar.hed]

Both Gateways have the same IP??? This cant work

Correct - it does not. My fault, I grabbed my LTE router (I have only one) from my current firewall (I lost my previous OPNsense firewall due to hardware error = it just died one day, so had to go back to my trusty old Asus AX88 router...) and connected it. Works directly! Perfect, on less problem.

That being said: The LTE router is ALSO on 10.x.x.x network - so there is of course a risk in that I could get the same 10.x.x.x IP from both at the same time......

I will now move onto DoT - however I am not entirely sure how to validate DoT, since NAT will let anything out...

[mimugmail]

DoT via Unbound or your browser?

[lar.hed]

My plan is to close port 53 on OPNsense out, and only allow 853.
So forward all port 53 to 127.0.0.1 internal.
In Unbound custom option:
server:
local-zone: "use-application-dns.net." always_nxdomain

This should, in theory, stop my firefox from DoT out, and only use OPNsense for DNS requests.
And any DNS requests should be handled by Unbound, which should use DoT, no matter what gateway is used.

Or am I doing some sort of error in my thinking?

[mimugmail]

Then there is no outbound nat involved, it will work

[lar.hed]

Okay - new challenge or question:

The Multi WAN guide says rule on DNS port to destination 192.168.1.1/32
The rule I seem to need for handling all DNS by OPNsense and Unbound on DNS port to destination 127.0.0.1/32

Not sure it is any difference in the real world so to speak?

[mimugmail]

This is only for traffic going through the firewall, local intiiated packets are not controlled by pf

[lar.hed]

Hmmm I am not 100% sure, since well I did a test (sorry  ) and well I added a few rules to both my WAN interfaces, to control whats goes out... It killed both.

I think it is about dpinger not being able to connect since both gateways goes offline, and I have added rules for ICMP <any>, no difference...

However, if I disable all rules - and that is ALL rules - on one of my WAN interfaces, it starts to work again. So if Unbound is considered "local", why is not dpinger? And what is dpinger using so I could open a rule for it?