Floating rules for grouped interfaces do not seem to work

[nautilus7]

I have several vlans on my local network. I am trying to use the Security Zones guide to only allow specific vlans to access other vlans and also allow internet access to all vlans.

I have created 3 groups, trusted internal (secure vlans), untrusted internal (unsecure vlans) and external (wan, wireguard).

I created the ICMP rules as described in the guide, as well as a floating rule which allows all internal networks (trusted and untrusted) to access the external ones (internet). This does not seem to work correctly, as none of my vlans have internet access. In live log, I never see my floating rules being mentioned and I mostly see the "Default deny / state violation rule" auto-generated rule that blocks traffic.

The floating rules are picked up correctly as they are listed for each vlan interface, but they show below the auto-generated rules. How are the floating allow rules supposed to work if the they are processed after the auto-generated ones (and specifically the ones that blocks everything)?

Can anyone help me understand what's wrong? Do I also need to add a separate rule for each interface or something?

EDIT: Do I also need to add rules for DNS and DHCP for each zone?

[OPNenthu]

How are the floating allow rules supposed to work if the they are processed after the auto-generated ones (and specifically the ones that blocks everything)?

It has to do with non-Quick vs. Quick rule types, indicated by the lightning bolt.  Keep in mind the "Default deny" rule is non-Quick.

The Quick rules are evaluated first, top to bottom, and in order of precedence by their priority levels (System/auto rules -> Floating -> Group -> Interface).

If, and only if, no Quick rule was matched in any of the sections, then the non-Quick ones are evaluated (again top to bottom).  The difference is that non-Quick rules are also last-match meaning later rules overrule earlier ones.  If you have a non-Quick rule in your Interface section that matches, it will take effect even if the system level "Default deny" rule was also matched.  Last one wins.

(This is the logical behavior at least; the rules might be processed differently in code.  You could imagine that they are all processed in a single pass and exits immediately on any Quick rule match, otherwise the last matched non-Quick rule action gets applied.)

The end result is that the system level "Default deny" rule action is only ever applied if there are no other rules (Quick or non-Quick) that are matched, anywhere in the hierarchy.

If you're hitting the Default deny then it's likely you have a misconfiguration in your rules.  Can you post them here?

EDIT: Do I also need to add rules for DNS and DHCP for each zone?

If you're using ISC, it registers automatic DHCP rules.  Kea and Dnsmasq do it by default unless you disable the option in the service settings.  I think there might be a nuance with Dnsmasq where it only auto-registers the DHCP rules if you select specific listen interfaces, but won't do it if you leave it on 'All' interfaces.

DNS rules may or may not be needed, depending on how you set up your access rules and where your DNS server is.  If you have a typical "Allow any" rule and your DNS isn't on a blocked network, then probably not needed.

[OPNenthu]

Quick update (pun intended):

Looking at the OpenBSD 'pf' manual, it seems non-quick is the default and 'quick' is a modifier which breaks the rule processing immediately on match.  https://www.openbsd.org/faq/pf/filter.html#quick

So the rules are processed in the same order as they are defined in one pass (which would make sense for things like packet tagging) but we can still think of the non-Quick rules as a separate pass logically, if we are only talking about the rule actions such as pass/block/reject.

[nautilus7]

Thanks for explaining the quick vs non-quick rules and their processing order.

If you're using ISC, it registers automatic DHCP rules.  Kea and Dnsmasq do it by default unless you disable the option in the service settings.  I think there might be a nuance with Dnsmasq where it only auto-registers the DHCP rules if you select specific listen interfaces, but won't do it if you leave it on 'All' interfaces.

I am using KEA DHCP and I have the option to create firewall rules enabled, so this should not be a problem.

DNS rules may or may not be needed, depending on how you set up your access rules and where your DNS server is.  If you have a typical "Allow any" rule and your DNS isn't on a blocked network, then probably not needed.

I think this might be the problem... Should I also have an "allow any" per each interface, apart of the floating ones? And this "allow any" rule should be quick or non-quick?


Anyway, I am using a WAN PPPoE interface (vdsl) and several VLAN interfaces. No plain LAN as opnsense is virtualized and runs with one physical adapter only. Below are photos of my 3 firewall groups, the 4 floating firewall rules related to these groups and the rules that are picked up by vlan20 interface. There is also an "allow any" quick rule specific for that interface. If I remove/disable the "allow any" rule from the vlan20 interface, then access to the internet is blocked.

[OPNenthu]

Yes, you're right.  Looking over the Zones guide, I see that it only walks you through setting up a few top-level rules for inter zone traffic.  I think it assumes that you will set up more specific rules for things like DNS and internet access:

Our unified ruleset will create a baseline that will always match on top-level. Afterwards, we can create more selective allow rules in the individual interface groups.

So I think you have a few options.  You could add a default allow rule (like LAN typically has) to the 'trust' group, or you can add more granular rules to specifically allow DNS(53), HTTP(80), and HTTPS(443). 

Quick is fine.  In that case, put any default 'Allow any' rule(s) at the bottom of the list.

(@Monviech, if you're reading, did I interpret the Zones guide correctly here?  Will be good to get the author's view on this.)

[nautilus7]

Man, not sure I follow, sorry.

If a default "allow all" rule, floating or per vlan, is needed to access the internet, how am I going to block my untrusted vlans to access my trusted ones?

[meyergru]

By making its target not "any", but a negated alias that contains RFC1918 or by putting a block rule for RFC1918 before the "allow any" rule for such less privileged VLANs.

I would not put these rules in the floating rules, but either into the VLAN rules or into a rule for a firewall group that contains all less privileged VLANs. The reason for this is that you may want to have some rules for well-known services on your LAN that are in RFC1918 and that must be specifically allowed in the floating rules (say, a file, a logging or NTP server).

[nautilus7]

Thanks for the suggestion. I have came across the concept of the !rfc1918 rule before. Haven't tried yet though.

So, the conclusion is that the zones setup guide in the docs is incomplete, the zones setup using floating rules is not optimal and better follow a more traditional approach regarding the vlan isolation I am looking for.

[OPNenthu]

Respectfully, no, I think you miss the point.  The zones guide is narrowly focused on the idea of zones and inter-zones traffic, but this is a more advanced topic and the guide requires that the reader already understands the basics of interfaces and interface rules.  It's the reader's exercise to configure their interface groups, as it says in the text I quoted.

Looking over your screenshots I think you're expecting that the top-level zone rules are providing internet to each other, which is not the case.  Allowing the internal networks to the reach the "external" network (where I presume you have your WAN interface) is not the same as allowing internet.  There's a common misconception that "WAN net" means "internet" but it's not.

There's an explanation of this and some more discussion about the allow rule with the RFC1918 exclusion here: https://forum.opnsense.org/index.php?topic=36865.0