OPNsense Forum
Archive => 20.7 Legacy Series => Topic started by: lar.hed on November 27, 2020, 07:26:40 am
-
So I read the web page documentation about Multi-WAN:
https://docs.opnsense.org/manual/how-tos/multiwan.html (https://docs.opnsense.org/manual/how-tos/multiwan.html)
And then I started to think about DNS settings and DoT (DNS over TLS within Unbound), and I just got into that it is not possible from my point of view to enter the correct information either on the DNS settings on " System %u2023 Settings %u2023 General " (does not allow @853 after IP address so no way to enter correct info about DoT servers) and in the web settings for Unbound " Services: Unbound DNS: Miscellaneous " there is no way to set the gateway as described in MultiWAN page above.
So how am I supposed to set up: Multi-WAN (failover from fiber to LTE in my case) and DoT - or is this combo not possible?
-
You need also gateway switching active where system default gateway always points to primary, when this goes down it swithes to second. Its only failover and not balancing, but DNS traffic so small, should be fine though
-
Not sure I follow, or rather if I do the "gateway switching active where system default gateway always points to primary" I will lose DNS. And I have not even tried DoT yet. Turning gateway switching on, fail DNS requests - turning off, DNS requests works. However if I play around, back forth and back again or something, DNS simply never works again. And everytime I make a change Unbound takes 53 second to restart before anything can happen (if it will happen that is).
I simply do not get this to work - and I am just trying to get this working with everything in default settings. Just followed the how-to multiwan (which also mentions Default Gateway Switching) - and I still fail....
-
Okay - I think I got what is not working for some reasone:
WAN_FTTH (Primary) - Fiber To The Home
WAN_LTE (Secondary) - LTE mobile modem as backup
Default Gateway Switching IS active for this test.
Starting from reboot, WAN_FTTH marked as active (under System -> Gateway -> Settings) - everything works.
Pulling the ethernet cable for the interface for WAN_FTTH, failover to WAN_LTE is marked as active - everything works.
Attaching the ethernet cable I just pulled, WAN-FTTH is marked as active - DNS resolution does not work.
Pulling the ethernet cable for WAN_LTE - NOW everything works on WAN_FTTH.
I could also reboot the firewall to get it working - but that seems a bit wrong in my book.
So yes it fails over, but it can not recover - what am I doing wrong? And I am sure everything is as the how-to describes it. I can take screendumps if requested, just tell me what?
Oh and NO I have not gotten to DoT yet...
-
Screenshot of System : Gateways : Single
-
A few screenshots then :-)
-
And one final screenshot...
-
Both Gateways have the same IP??? This cant work
-
Both Gateways have the same IP??? This cant work
Correct - it does not. My fault, I grabbed my LTE router (I have only one) from my current firewall (I lost my previous OPNsense firewall due to hardware error = it just died one day, so had to go back to my trusty old Asus AX88 router...) and connected it. Works directly! Perfect, on less problem.
That being said: The LTE router is ALSO on 10.x.x.x network - so there is of course a risk in that I could get the same 10.x.x.x IP from both at the same time......
I will now move onto DoT - however I am not entirely sure how to validate DoT, since NAT will let anything out...
-
DoT via Unbound or your browser?
-
My plan is to close port 53 on OPNsense out, and only allow 853.
So forward all port 53 to 127.0.0.1 internal.
In Unbound custom option:
server:
local-zone: "use-application-dns.net." always_nxdomain
This should, in theory, stop my firefox from DoT out, and only use OPNsense for DNS requests.
And any DNS requests should be handled by Unbound, which should use DoT, no matter what gateway is used.
Or am I doing some sort of error in my thinking?
-
Then there is no outbound nat involved, it will work
-
Okay - new challenge or question:
The Multi WAN guide says rule on DNS port to destination 192.168.1.1/32
The rule I seem to need for handling all DNS by OPNsense and Unbound on DNS port to destination 127.0.0.1/32
Not sure it is any difference in the real world so to speak?
-
This is only for traffic going through the firewall, local intiiated packets are not controlled by pf
-
Hmmm I am not 100% sure, since well I did a test (sorry :'( ) and well I added a few rules to both my WAN interfaces, to control whats goes out... It killed both.
I think it is about dpinger not being able to connect since both gateways goes offline, and I have added rules for ICMP <any>, no difference...
However, if I disable all rules - and that is ALL rules - on one of my WAN interfaces, it starts to work again. So if Unbound is considered "local", why is not dpinger? And what is dpinger using so I could open a rule for it?
-
Your screenshot looks strange, you allow inbound DNS and ntp on WAN??
-
No I do not - it is OUT going ONLY. I have no IN rules at all.
-
I was not at my PC at my previous response, here is two screenshots on DNS and NTP rules detail.
-
For the fun of it I decided to put my new OPNsense (WAN_FTTH) router inside the ASUS firewall, and allowed all ports open in/out on the IP that the OPNsense firewall got. This shut down dpinger of course, and I lost WAN_FTTH interface = status offline. I then added ICMP packet typ 0 and 8 (which is Echo Reply and Echo) - and the result is that my WAN_FTTH interface got status = online.
So dpinger needs Echo (=ICMP packet type 8) and Echo Reply (=ICMP packet type 0) to work. I now know what to filter for in OPNsense - I thought...
...so I am still working on what I do wrong here...
-
Okay, can not work around this challenge...
All below is tested on my backup WAN interface, the WAN_LTE interface. Also, for all testing below there are NO block rules enabled! The ONLY rules I am testing is "allow out" kind of rules to try figuring this out...
If I add a rule that allows ICMP <any> out - dpinger will stop working.
If I add a rule that allow <any> protocol <any> everything out - dpinger will stop working.
Add a rule that allows HTTP / HTTPS / NTP / DNS / port 853 (DNS-over-TLS) out - dpinger works perfect.
So adding a ICMP or allow anything out rule stops dpinger. I can not do this wrong, there got to be something else involved here that I simply put don't understand. Adding a allow all everything out rule should not stop dpinger from working. And dpinger, since it is internal, is not supposed to be stooped anyway or?
-
Normally you dont need allow Rules for outgoing packets, everything initiated from the Firewall is allowed, incl dpinger
-
Okay. I understand. I think. Might need a night to sleep on it...
How should I then do whitelist everything that is allowed to exit from OPNsense? it sounds like that is impossible?
On 20.1 I got this working the same way I am trying now. Has something changed to 20.7?
-
Right, so I can ONLY control what is send/received on for example the LAN interface - but I have no way of controlling what goes thru the WAN interface? Have I got this correct?
-
You can control more or less everything, but maybe the concept is mit clear. Just give me an example with source and destination what you want to block
-
I think I can do better, I will (try to) draw a sketch that describes how I imagine my installation of a firewall. Just need a few moments.....
-
Btw I am not sure, but it feels like there is something not working on filtering on WAN interface. Since I had the problem with ICMP above, I decided to test the "rest". So I constructed a rule that allowed HTTP. Only one rule, and that was HTTP out pass. Nothing really happened. So I decided to add a rule for HTTPS out pass = internet dead - now again, I added an allow rule (pass) and I got a block rule.... Disabled that HTTPS rule = internet online again.
So from the looks of it, adding a rule, pass or block, will ALWAYS result in a BLOCK rule no matter what?!
I am pretty sure this worked in 20.1 - but it does not work in 20.7 - so something has changed?
-
Maybe it worked with 20.1 cause of a coinncidence. Really, I never Had to use Rules with direction outbound, never.
-
Well I have such a case - now there are more ways to handle what I now describe, so....
I use a software, LMS - Logitech Media Server, and for some (very uggly) reason it like so send stuff on port 9000. Like broadcast style. A very easy way to solve this is to simply block port 9000 on the (ex. LAN) interface that the LMS server is connected to. And it will for sure do the trick. I hope since I have actually never tried that on OPNSense...
Now this is how I have it on my ASUS router (it has a LAN-to-WAN netfilter GUI part, where one can choose blacklist or whitelist any TCP/UDP traffic (and ICMP)): I only run whitelist, port 80/443 and that is it. This works, since well it is like in OPNsense, you have no control of the WAN interface. So since the ASUS router handles all DNS requests on port 53 (or 853) - well I do not need to handle that since that is after LAN-to-WAN firewall so to speak. The same way it goes in OPNsense.
And this is where I am trying to improve. I would like to have that extra control of what actually leaves the WAN port(s), dead 100% control. Not only LAN (WORK, or what ever) rules but that extra last line of "defense".
And as I wrote I did have this up and running in 20.1 since I have screendumps on all my old 20.1 firewall rules, and Multi-WAN btw (what did not work on Multi-WAN in that hardware was gateway switch - it only worked from WAN-FTTH most likely since I used wrong gateway in my rules - yes my mistake). And I could use my work-PC for sure thru that OPNsense firewall. The same rules on 20.7 = internet blocked.
Now if my interpretation is correct, this is simply not possible (anymore?). We all assume that OPNsense is 100% safe att all time. Right? Since OPNsense always has 100% possibility to do what ever it likes - it is the firewall after all?
Don't get med wrong here, OPNsense is superb - it is just that last line of defense that used to work that now simply put does not work.
-
You still didnt post IPs and direction
-
Sorry, yes still trying to find enough time to draw that drawing I have in my head, on the right computer on top of all. This seems to be an issue for me, always working from home, but on the wrong computer....
-
Okay - when I draw this on paper yesterday evening I think I know why we "talk different languages" so to speak. I see this at "hardware"-level, and I think OPNsense developers sees this more like "software"-level. Why? Well you request, most likely correct, IPs and direction. I like to see this as hardware interface level, and of course direction. I think this could explain why I would very much like to have back the firewall rules that seems to have worked on 20.1 - and now no rules at all works on WAN-kind of interfaces (all rules at always interpreted as blocking rules, no matter what).
Anyway here is a very simple drawing. Do note that I am att interface level, and ports mentioned are the only one allowed out from each area so to speak.
-
When you dont use port forwards you can leave the WAN rules tab empty, just add the rules with direction incoming on LAN and/or WORK.
And please, dont repeat yourself that it worked before with 20.1, the concept of firewalling is to allow the packet closest to the source, so when LAN wants to travel via WAN, add the rule on LAN.
When a packet of WORK wants to go to LAN, add the rules to WORK.
Only portforwards are added to WAN, or if you want to allow VPN to the firewall.
-
Sorry for that - will not happen again.
You asked for example so you could maybe help me with rules. Here is the 2 I am currently struggling with:
1) DNS will be only port 53 on the inside (left if you will on earlier posted drawing) of OPNsense, so a rule to go from an interface, say WORK in this example my laptop with 192.168.2.10, to internal Unbound DNS only, and this needs to work with Multi-WAN of course.
2) For the WAN interfaces, WAN_FTTH and WAN_LTE, only allow DNS-over-TLS to IP address 1.1.1.1 and 9.9.9.9 from Unbound internal of OPNsense, thru port 853 - and no traffic on 53 (or 853 except this two IPs).
The first one I got I think, although I am still not 100% sure what destination IP I should use - currently I use "This Firewall" since it works - however that alias seems a bit to "large" so to speak.
The second one - well that is where I can not figure out how to get that working.
For me the left (on my drawing) and right (again on my drawing) are two separate "zones" - and I do not like to assume that everything works all the time. I liked that extra protection.
-
For the first you can set it on rules tab LAN with source LAN net and destnations LAN address and for WORK, WORK net and destination WORK address.
So every client in it's own network use the gateway address as DNS.
Regarding 2 you have to make sure within Unbound that only DoT is allowed, no idea how to do this, but I guess when you enabled DoT for zone "." it should force 853 always.
-
Thanks! And I mean it!
Sorry question 2 is not resolvable, but I guess that is how life is.