"Inverting destinations is only allowed for single targets" rule error

[techturtle]

Setting multiple aliases as "Destination" and ticking "Invert destination" within a firewall rule declaration currently triggers error:

Inverting destinations is only allowed for single targets to avoid mis-interpretations

I am a bit buffled, what is meant by "mis-interpretations" - isn't this the application of De Morgan's laws?

Let's say, two firewall aliases A and B exist, each with couple of IPs. Then setting A and B in "Destination" creates the union of those two aliases A ∨ B ("match, if destination is in any of those aliases"). Additionally enabling "Invert destination" should lead to ¬ ( A ∨ B ) = ¬ A ∧ ¬ B ("match, if destination neither is in A nor in B").

I am not asking from a theoretical or academical standpoint, but would really like to express:

• If destination does not match any of the hosts in those aliases, block connection => block rule.
• Fail fast => quick/first match rule is to be used.
• Keep firewall rules strict.

Especially with regards to point 3, if splitting up into

• a pass rule for A ∨ B
• followed by blocking rule for non-matching hosts

, then anything with destination in either A or B is immediately allowed. But according to principle of least privilege, it would be better to preserve the possibility to block traffic for other reasons by subsequent rules. Current rule logic cannot express this pattern AFAIK.

I definitely agree, these logic expressions sometimes can get confusing. So it might be worth to add a help message for "Invert destination":

Without inversion, the union of destinations is matched = "match if any destination A OR B matches".
With inversion, selected destinations A and B are processed as follows: ¬ ( A ∨ B ) = ¬ A ∧ ¬ B = "match, if destination neither is in A nor in B"

Btw: https://forum.opnsense.org/index.php?topic=51467.msg263889#msg263889 is a bit similar, at least error message. But my issue does not have to do anything with migration. Above error already appeared with the old firewall rules format.

Happy to read any feedback.

[Patrick M. Hausen]

Source or destination invert used to be allowed for multiple entries, but lead to undesired effects confusion because the underlying logic created "not this OR not that" which essentially applied to anything.

Since the logic cannot easily be changed, the check and error message was created and the solution for your requirement is to create a group alias and use that with invert.

[techturtle]

Thanks for the answer - you mean Network group alias, right?

Group alias seems to only support other Network and host aliases - though I also need to nest External and URL Table. In the past I had some timing issues, where values of child aliases (ipsets; external type alias) weren't quickly enough populated to parent alias. Apparently group alias is only  guaranteed to be eventually consistent with all child aliases, not immediately, as all values are copied over and not referenced. Or has this synchronization been improved in recent versions? Otherwise unfortunately that's not a viable solution for me.

Since the logic cannot easily be changed

Can you (or someone else) elaborate, why not / what would need to be done?

[Patrick M. Hausen]

I have network group aliases with multiple URL tables exactly for the purpose of source invert to express "not in any of these block lists".

As for the underlying logic - I'm not a core developer, sorry.

[techturtle]

Thanks, no worries.

My case is IPset firewall alias with dnsmasq. A parent network alias nesting this ipset (amongst others) of course would be required to have its values immediately updated, after dnsmasq puts some ip into child ipset alias, in order to make whitelisting work properly. Your case might be slightly different - quick realtime updates of aliases vs. a bit more slow paced url table aliases. I am gonna re-test this current OPNsense version.

Regarding OP error: may some dev can comment and discuss here. Otherwise I am fine creating a separate issue on GitHub.

[techturtle]

Multiple sources/destinations become multiple rules under the hood - that's a bit different than I anticipated. Probably this is the cause, why implementation of negation gets complicated.

So back to nested aliases: Is there any info, how these are synced?
Given parent alias P and nested aliases C1, C2, I'd like to have P synced immediately in a blocking fashion, as soon as either C1 or C2 is changed. I.e. if a rule using P is to be consulted for a packet, application of this rule should wait till P has fully flushed / synced.

Is that possible?

[Patrick M. Hausen]

If you have an active connection which you intend to block the problem is possibly not that your aliases are not updated in a timely fashion, but that you need to flush the firewall states.

An active TCP connection that once was permitted stays that way because of the stateful firewall, even if you change the rules.

Of course flushing the firewall states will disrupt all active connections which is exactly why it does not happen by default.

[pfry]

[...] Of course flushing the firewall states will disrupt all active connections which is exactly why it does not happen by default.

Looks like single rule(set) operations are possible, but the mechanism is rather obtuse ("anchors") and does not appear to apply to associated states/sessions. A bit of a feature oversight.

[techturtle]

If you have an active connection which you intend to block the problem is possibly not that your aliases are not updated in a timely fashion, but that you need to flush the firewall states.

Active connections or firewall states are rather unrelated here, no need to touch anything on these. I am just asking for a immediate consistency between parent and nested child alias entries at any point in time with regards to firewall rule matching.

My case is IPset firewall alias with dnsmasq.

The flow is like:
> declare parent alias P
> declare external alias C1
> nest C1 under P (workaround for rule negation with multiple dests; not sure, if it works with external)
> optionally nest more aliases C2, CX under P, e.g. for static IP whitelists

Then for each request:
> dnsmasq resolves domain
> IP is put in C1
> Firewall whitelists packets, whose dest IP is in P, blocks everything else (*1)

For (*1), it is crucial, that resolved IP is immediately synchronized from C1 to P and available for the rule. IIRC last time I checked, there only was eventual consistency, and this causes packets to be blocked on first try. Connection only worked on second try, as now P was synced with C1.

If C1 is not nested, it works fine. But then I can't express "block, if destination neither is in C1 nor in C2" due to limitation with negation and multiple dests.

[Monviech (Cedrik)]

Dnsmasq cannot add an IP to an alias faster than the client tries the initial connection. First tries most likely fail for the client and it has to do another connection attempt. On second try the alias is hydrated so the connection succeeds.

Dnsmasq just cannot look into the future, it just sees a DNS query, but then the client has already resolved it and tries the first connection before the PF alias is even touched the first time.

Timeline of the race:

1. client asks DNS
2. dnsmasq resolves / observes result
3. client immediately opens TCP/UDP connection
4. dnsmasq hook updates pf table / alias
5. pf evaluates packet

Step 3 can easily beat step 4.

You would need a divert-to feature so PF waits here until dnsmasq can make a decision, but thats currently not supported. Suricata has it, as example.

[techturtle]

Thanks Monviech,

Timeline of the race:

1. client asks DNS
2. dnsmasq resolves / observes result
3. client immediately opens TCP/UDP connection
4. dnsmasq hook updates pf table / alias
5. pf evaluates packet

are you saying race condition occurs, because:
a) dnsmasq reports resolved IP to client *before* it writes it to pf table/alias
b) dnsmasq reports resolved IP to client *after* it writes it to pf table/alias, but update of pf table is not immediately available
?

From own experience, dnsmasq resolution and subsequent IP connection works just fine, when not nesting aliases, hence a bit buffled. My assumption was:
1. client asks DNS
2. dnsmasq resolves / observes result
3. dnsmasq hook updates pf table / alias, table update is directly populated, given unnested alias
4. dnsmasq notifies client about resolved IP (after triggering table update)
5. client opens TCP/UDP connection
...
This also worked for dnsmasq on Linux with nftset/ipset. Hence I would rather see delayed pf table update as cause.

Btw: I am not talking about the case, where client already has resolved IP in its local cache, but dnsmasq hasn't resolved domain yet. Of course, client connection would get blocked here.

[Monviech (Cedrik)]

What Im saying is that some users experienced this here in the past with dnsmasq and ipset.

DNS resolved faster for the client than the pf table and pf got the change. First connection is then blocked/allowed by a different rule, a state is created. Therefore more packets matching that state will be allowed/blocked.

Maybe on linux iptables is not a stateful firewall so it doesnt matter as much there as every packet is evaluated statelessly (though I guess it should be due to conntrack).

Just delete all states in pf, flush all tables in questions, delete a dns cache of a client, do a first connection to some site and observe:

pfctl -t <table> -T show

For all tables you expect the IP to appear, watch it with unix timestamps so you know how much the delay is. Use tcpdump as well so you have exact timestamps everywhere.

But I won't analyse this, if you find something give a filtered summary where the issue is.

[techturtle]

It definitely is the nested alias in my case.

Firewall alias type "Network group" does not allow a child alias of type "External". So I chose "Network" as parent P, which referenced external child alias C in content.
After C had been updated, it took ~ 30 sec for P to be synced/updated with values from C. This suggests, values are getting copied over from child to parent at some point in time and are not referenced.

1. Would it be feasible to implement a "Reference type" alias, whose purpose is to be immediately consistent with nested child aliases? For example, this type might dynamically look up values from a child alias (with cost of slightly decreased performance), pointing to PF table of external alias instead of copying values.

2. @Monviech judging by your post in https://github.com/opnsense/core/issues/8559, am I correct current OPNsense architecture isn't ready yet (too much effort) to support inversion of multiple sources/destination in firewall rule creation?

Thanks.

[Monviech (Cedrik)]

@techturtle

It would be best if you open a new issue on github, specifying:
- your exact usecase
- what doesn't work
- what you think would be the most pragmatic choice to fix it

I cannot say if what you do is something supported or not. The indirect nesting sounds like an unintended side effect to me, but lets see what others on github say.

If you open an issue on github, please use the KISS (Keep it short and simple) principle, that would help a lot. Thanks :)