Messages

Done:
https://github.com/opnsense/core/issues/9262
https://github.com/opnsense/core/issues/9263
https://github.com/opnsense/core/issues/9264

Thanks for your work.

The way it works right now is intentional, since it guides the user implicitly that domains that are entered must be resolvable by dnsmasq. This is because most users also run unbound.

Yes, I am using Unbound as upstream.
Not quite sure about your point. If using IPset feature, then Unbound as upstream recursive resolver seems to be the preferred solution by docs - otherwise an additional external resolver would be needed to prevent loops.
In this case all queries are forwarded via wildcard / `*`. If a domain is resolvable will be determined by upstream. IPSet feature picks up those domains from the wildcard forwarding, that are interesting to be stored for firewall rules.
Or did you primarily have the case Unbound -> Dnmasq in mind?

Allowing a wildcard (#) to flush all resolved domains into an alias seems like its unecessary. The use case is clearly stated in the documentation, for allowlists regarding things like *.example.com or the like. It is just an extension of the other Alias types that exist for hostnames, not a solution for full alias management.

That wildcard is indeed intended for allowlists:
Allow everything, that explicitly has been resolved by primary DNS resolver of OPNsense. This enables whitelisting solely by domains, not static IPs.

If you want full control, you can import a custom dnsmasq configuration file.

Yeah, that's what I am doing now. But I hoped to finally get all settings smoothly integrated in OPNsense GUI for better maintainability, now Dnsmasq is the standard DNS/DHCP server.                                                                                                         

[Suggestion]

First of all thanks a lot for taking this feature directly into OPNsense user interface!

The general limitation I am seeing with current design in

Services > Dnsmasq DNS & DHCP > Domains 

is, that IPSet/"Firewall alias" is artificially bound to forwarded DNS servers/"IP address". In Dnsmasq IPsets are just tied to domain names.
Here is an example:


You cannot view this attachment.

creates following config in /usr/local/etc/dnsmasq.conf:

Code Select Expand

server=/*/127.0.0.1#5335
rebind-domain-ok=/*/
server=/example.org/127.0.0.1#5335
ipset=/example.org/dns_resolved_ips
rebind-domain-ok=/example.org/
server=/example.org/1.1.1.1
ipset=/example.org/dns_resolved_ips
rebind-domain-ok=/example.org/
server=/example.org/1.1.1.1
ipset=/example.org/my_orga_unit
rebind-domain-ok=/example.org/


• This design suggerates, that a firewall alias is only filled, if DNS server of same entry has been chosen for forwarding.
  But this is not the case. E.g. if example.org is resolved via 127.0.0.1:5335, then returned IP is stored both in `dns_resolved_ips` and `my_orga_unit` (despite latter alias defined for entry with 1.1.1.1).
• Work is repetitive, if IPs are to be stored in multiple IPsets, as you need to specify `IP address` and `Port` in each entry. But server and port don't matter for IPsets at all.
• There also are duplicate entries in dnsmasq config created:
  
  Code Select Expand
  # no copy-and-paste mistake by me, this gets auto-generated
server=/example.org/1.1.1.1
server=/example.org/1.1.1.1

ipset=/example.org/dns_resolved_ips
ipset=/example.org/my_orga_unit
(Does that actually send DNS requests *twice* due to double entries, as dnsmasq will do concurrent queries to all servers, if "Query DNS servers sequentially" is not checked?)
  
• Pure dnsmasq config would have entries like this:
  
  Code Select Expand
  server=/example.org/1.1.1.1
ipset=/example.org/dns_resolved_ips,my_orga_unit

• If you are used to dnsmasq notation `ipset=`, don't pay attention to hints and think "I just want to create an IPset for domain X, so leaving 'IP address' field for server empty", then domain won't be resolved at all, which seems a bit un-intuitive.
  
• An important case for me is to have one catch-all IPset, which captures *all* resolved IPs. dnsmasq allows this via # wildcard like `/#/dns_resolved_ips`. In OPNsense user interface, we currently can't do that. Setting `*` in `Domain` and selecting a "Firewall alias" results in error

  Top level wildcard entries are not allowed for Ipset.

• Note that `ipset=` and `server=` in dnsmasq have kinda their own notation, so with current design, handling these specifics in one form is difficult, as you would end up mixing both notations.
  
• Another thing I noticed: Setting domain `*` as suggested here leads to `rebind-domain-ok=/*/` entry be automatically created in config file. Wouldn't this disable DNS rebinding checks for all domains?

Suggestion
What about ditching "Firewall alias" in "Domains" and creating an additional tab "IPSet", which takes a domain (optional autocomplete from values in "Domains" tab) and a firewall alias name? This change would solve above issues and comes closer to the config model Dnsmasq has.

If you managed to read till here: Let know what you think!

You're welcome, let me know, if you found something out!
I will also test this (+ WAN restricting via interface OUT rule) later on, after having re-organized some of my other setup.

You could only block outbound connections with that, which is usually not what you want.

Yeah, I see. But sometimes there is still need for blocking of OUT traffic on WAN. An example is a VPN "killswitch" via NO_WAN_EGRESS tag or similar, which prevents data leaks as last barrier. If I am not mistaken, the guide even is in this forum. Two remaining questions:

1. Do I need a floating rule for this tagging case, or is it sufficient to use a plain OUT interface rule on WAN with the same rule content: block on "Match local tag = NO_WAN_EGRESS"?

2. Does an interface-specific rule with OUT direction on WAN also see the Post-SNAT IP address/public IP, like floating rules? Or is it still the internal address for rule matches? If former, then both floating and interface rule should be interchangeable here. Looking at pfSense docs that is still not quite clear to me (happily take an updated one from OPNsense, if there is one!). Described LAN-to-WAN traffic flow in their docs seem to only include outbound floating rules on WAN, no OUT WAN interface rules at all.

Hence, if migrating floating to interface-specific rules, one consideration is to check, whether you still need its higher priority with regards to DNAT - understood.

What about other potential uses - specifically, do I still need floating rules for tag/match or to block outgoing connections e.g. on WAN/SNAT? I only found these docs for pfsense, not sure if that's still a thing for OPNsense as a fork.

In other words: Can I equally create a WAN interface rule with OUT direction for this purpose? Note that pfSense originally solely had IN direction for interface rules, so there naturally was need to create special type floating rules for outbound connections. According to OPNsense interface rule creation form, OUT for interface can be chosen.

https://opnsense.org/roadmap/

One of the most well-arranged roadmaps I've seen so far - great job!

NAT is always processed before filtering.

Thanks for your answer. I should write that on a paper - netfilter/iptables/nftables first does dnat, then filtering/routing, then snat. So coming from that tool, this order is slightly confusing.

So e.g. if you use global blocklists like FIREHOL and friends, but still want easy inbound NAT, best place those blocklists in the floating category and not on the interface.

Yeah, it definitely makes sense to add global rules to Floating. But you also could define an interface group - or firewall group, as called in the GUI - for global rules and probably have more flexibility, as you can define multiple of those.
(Btw: This is why I have the feeling, floating rules seem redundant from user-facing perspective)

It's still my impression I can use interface and floating rules interchangeably. Or doe floating rules change anything fundamentally regarding NAT (like processing order)?

You are aware of the implications of doing that w/r to firewall rule priorities, especially when you use NAT?

To my knowledge System > Floating > Interface group > Interface rules.
So afaik I should be able to convert floating to an interface rules (all first match), if
- rule is placed at the top of interface-specific table, to have higher priority than existing rules
- there is no interfering interface group rule
But your comment about NAT made me curious. Can you clarify?

in the new Firewall - Automation - Filter all of this works already as Floating is determined differently.

Oh, thanks. Honestly I totally missed this new feature.
If I got you right, this is the new preferred rule processing component, side-by-side to legacy rules  - will look into it.

Hi,
I am in the process of migrating 30+ floating rules back to interface rules - they only have one interface assigned.
Cloning on a floating rule doesn't  allow to convert this rule back to an interface (group) rule.
So how might I do this efficiently without manual, error-prone re-creation of rules?
Not interesting to me, but probably the other way round interface rule -> floating rule would also be a good idea to enhance compatibility.

Kind regards

Thanks for the info. I guess Interfaces > Neighbors is just to maintain the ARP table in the original sense?
"Enable Static ARP entries" actually does a lot more than that, as described above.
In certain subnet,  I'd like to keep this setting's workflow of restricting a client (no WAN), as long as it hasn't been registered as static dhcp mapping yet.
And I agree with @meyergru that its scope would be better suited outside specific DHCP service. Is there e.g. an alias referencing "static DHCP client IPs of a subnet" or so, to manually craft a blocking rule - or will this feature be abandoned?

Despite its confusing name (*), I found

Services > ISC DHCPv4 > Static ARP > Enable Static ARP entries (*checked*)

quite practical in stricter environments/subnets to prevent unknown clients (no static DHCP mapping entry) from communicating with the firewall in any way incl. reaching out to WAN.
Now with upcoming migration away from ISC, I am asking: Is there an equivalent setting in dnsmasq?

---

(*) It does not only cause a static ARP table formed by DHCP mappings, nor prevents ARP requests to get firewall's MAC from IP - It actually completely blocks unknown clients.

Basically if you have rules on all 3 of these. From Interface rule view point, its created as a single list starting with rules from Floating at the top followed by Group rules Followed by Interface rules. In case a Floating rule is "last match" If any rule from this combined list is hit, the "last match rule" will not be preferred.

Thanks.
1. And system-defined/auto-generated rules are on the very top, right? This is the section, where rule related to "Disable force gateway" is in.
2. I guess, by "any rule from this combined list is hit" you are referring to  "first match" rules. I was wondering about priority in case of multiple "last match" rules - and  if I am not mistaken, here the rule located more at bottom of this combined list wins (given no "first match" rule can be applied).
3. So basically, there is an effective rule list built from System-defined > Floating > Group > Interface. And scope of system-generated + floating rules can further be limited to one or more interfaces.

(just curious about internal functioning of OPNsense, don't want to be nit-picky)

PBR is not a security feature, its a routing feature.

But what is the purpose of "Disable force gateway" then - firewall admins are responsible for proper setup and maintenance of routing table/firewall rules anyway.
Can it be seen as some kind of additional aid against accidental configuration?
At least from what I've learned about rule organization and precedence here, it cannot accomplish its purpose of enforcing used gateway for WAN-type interfaces.

#4: I don't think so.
Otherwise, the default deny/state violation rule would override all custom rules. 😉

First, thanks for the correction.
Docs in https://docs.opnsense.org/manual/firewall.html#processing-order are not too clear on this topic.
In desperation I also asked an AI, and it answered with " 'last match' means the last matched rule within the specific section (such as floating or interface) of the firewall rules being evaluated."  Apparently this wasn't a good idea 😉.

This said, I agree that the force GW might be one of the most obscure rules.
In particular, it's after another last match rule that's more generic thus should fire first. I suspect a non-visible element.

What does not make sense to me is:
Since system-defined rule induced by "Disable force gateway" is last match, and given last match is evaluated globally and not per section, it can't really enforce the gateway, right?
I then might just create a first (or even last) match rule in Floating or Interface section, and it should take higher priority.

A firewall is first and foremost a router with filtering capabilities on top. It runs on a single IP stack with locally connected interfaces and a routing table. And that should be all that is taken into consideration by default when forwarding packets.

[...]

Just default both to off and revert the UI so a ticked checkbox means "special feature only necessary in rare situations explicitly enabled".

Yeah, I am also inclined to *enable* the "Disable forced gateway" setting (what a boolean nightmare term lol).
So from security perspective, there should be no harm to do so, given you did not set up policy-based rules, that redirect to unintended other gateways?

[security implications]

I'll try to provide an answer myself (please correct, if there's something wrong).

1. Does unchecked value (default) for "Disable force gateway" always enforce WAN-type interface to use its associated gateway?
Yes. This setting creates a policy-based, auto-generated rule for each WAN-type interface (like let's say `WAN`), which enforces redirection to `WAN` and effectively overrides other policy-based rules and omits static routes.

2. Static routes still seem to work though?
No. But my case was different: A local process created packets, so static routing table was consulted first, before this packet was forwarded to WAN interface. And "Disable force gateway" only kicks in at the point of packet being associated with a WAN-type interface.

3. Given "Disable force gateway" is *enabled*: What does this option change in OPNsense configuration?
It only removes the auto-generated policy rule, everything else should be kept same and system should work as usual (could not test myself yet, as no testing system). A WAN-type interface now can have (multiple) gateways different from its configured gateway, and system routing table and other policy-based rules are respected again for routing decisions.

4. Rule `let out anything from firewall host itself (force gw)` is marked as "last match". But "last match" rules can be overwritten by "first match" rules, so it effectively would *not* enforce the gateway?
My bad. "Last match" rules are evaluated per section (system-defined/auto-generated, floating, interface group, interface). `let out anything from firewall host itself (force gw)` resides in the auto-generated section, which has highest priority. If there is no "first match" in this section, and above rule is last, it will always matched first.

---

That been said:
How do you assess security implications of enabling "Disable force gateway" setting?

[Quotes from the docs]

I also would like to know more about this setting. To recap, it can be found under

Code Select Expand

Firewall -> Settings -> Advanced -> Multi-WAN -> Disable force gateway

Quotes from the docs

https://docs.opnsense.org/manual/firewall.html#direction

Traffic leaving the firewall is accepted by default (using a non-quick rule), when **Disable force gateway** in `Firewall ‣ Settings ‣ Advanced` is not checked, the connected gateway would be enforced as well.

https://docs.opnsense.org/manual/firewall_settings.html#disable-force-gateway

Disable force gateway

By default OPNsense enforces a gateway on "Wan" type interfaces (those with a gateway attached to it), although the default usually is the desired behaviour, it does influence the routing decisions made by the system (local traffic bound to an address will use the associated gateway).

**Note**
This rule is responsible for the `let out anything from firewall host itself (force gw)` rule visible in the floating section, it forces a route to (`route-to`) on all non local traffic for the "Wan" type interface.

Example
WAN interface with IP 192.168.1.2.
WAN has DHCP-configured gateway 192.168.1.1 - i.e. there is an entry in System -> Gateways -> Configuration.
"Disable force gateway" is *disabled* (default value).
Use case: Try to re-route certain packets on WAN to a WireGuard gateway (^1) based on destination IP, via policy-based routing or static routes.

My observations so far
a) Manually routing to WireGuard gateway with static routes does work.

b) Policy-based routing to different gateway does *not* work
- Still "Live view" shows this rule as matched for a packet.
- I tried to start "Packet Capture", but could not see any traffic on the WAN interface for packets.
- What seems confusing: Rule `let out anything from firewall host itself (force gw)` is marked as "last match" (see ^2 and attachment). But "last match" rules can be overwritten by "first match" rules, so it effectively would *not* enforce the gateway, right?

Questions
1. Is my understanding correct, that "Disable force gateway" is meant to always enforce and restrict interface to use the associated gateway, if existent (here IF 192.168.1.2 -> GW 192.168.1.1)?
2. Static routes still seem to work though?
3. Given  "Disable force gateway" is *enabled*: What does this option change in OPNsense configuration? I'd be interested to have this option enabled only for a specific interface, while keeping everything else same. So there is a bit of customization need.

I would highly appreciate, if someone more knowledgeable could provide some clarification (Sorry for the wall of text).

---

Related, old post: https://forum.opnsense.org/index.php?topic=6242.0

^1: To be more accurate, this is a Gateway group, not a single gateway. But did not want to make this more complicate here.
^2: See Firewall -> Rules -> WAN -> Expand "Automatically generated rules".