how to create floating (and group rules) after migrating to new firewall rules?

[defaultuserfoo]

Hi,

before migrating to the new firewall rules, I used to have a floating rule blocking all trafic between all interfaces in an interface group and a rule on the LAN interface allowing all traffic from a particular IP address in the LAN to everywhere on all of the interfaces in the interface group.

After migrating to the new rules, these rules didn't get lost, but the one allowing all traffic doesn't work anymore.  That kinda makes sense because the floating rule would block the traffic before the rule allowing it is reached (leaving me wondering why it worked before).

I tried to move the allowing rule before the blocking rule but I only get the message that I can't move an interface rule before a group rule.

I don't see a way in the new rules to make floating rules, and group rules seem to have been silently introduced by the migration, with no way to create them either.

So how do I now make a rule that would allow all traffic from a certain IP address on the LAN interface to everywhere on all interfaces in the interface group that goes before the blocking rule?  I guess it would have to be a floating rule which I can't create ...

[defaultuserfoo]

Hm this migration is extremely bad :((  It has destroyed ALL firewall rules except for the floating ones because they ALL don't work anymore, and ALL of them need to be changed :(((

You need to create a migration assistant that actually understands what the rules do and converts them accordingly.  Without that, you can't call it 'assistant' but only 'disaster'.

Maybe you should make making firewall rules even more confusing, more obfuscating and more user-unfriendly from now on ...

[dirtyfreebooter]

[Monviech (Cedrik)]

Start by reading our up to date documentation:

https://docs.opnsense.org/manual/firewall.html

[viragomann]

After migrating to the new rules, these rules didn't get lost, but the one allowing all traffic doesn't work anymore.  That kinda makes sense because the floating rule would block the traffic before the rule allowing it is reached (leaving me wondering why it worked before).

Maybe the LAN rule was created as a floating rule as well.
But floating rules for a single interface are not possible anymore in the new rules.

So how do I now make a rule that would allow all traffic from a certain IP address on the LAN interface to everywhere on all interfaces in the interface group that goes before the blocking rule?  I guess it would have to be a floating rule which I can't create ...

I get two possible ways in mind.

• Turn the floating rule into a "last match" rule by removing the Quick check.
• Exclude the certain LAN devices from the source in the block rule. You can achieve this by checking "Invert Source" and entering the IP or alias.

[defaultuserfoo]

Thanks, I've been looking at that documentation.  How is it supposed to help?  Does it say somewhere how to create floating and group rules?

It doesn't say that the migration will destroy the whole firewall.  The floating rules that are remaining don't seem to work anymore either :(

Migration is not possible :(  You have to completely start over.  The existing rules are merely a reminder of what the intentions were.

Change the documentation.  Warn users about this change and tell them they need to start over and set up an entirely new firewall.  Stop pretending this is a 'migration'!

[Monviech (Cedrik)]

Sure here is the relevant section. Please note group rules are not a new concept, they also exist in the old firewall rules since ages.

Floating is an annoying concept that still has to be decided on most likely at some point. For now it is what it is.

Also you are not forced to migrate yet. The old rules stick around for 2 more years at least. No rush.

-----

Since Firewall ‣ Rules [new] and Firewall ‣ Rules implementations exist side by side, there are some additional considerations regarding the processing order of rules.

If a Firewall ‣ Rules [new] filter rule has:

a single interface defined, it is an Interface Rule
a group interface defined, it is a Group Rule
any number of interfaces or one inverted interface defined, it is a Floating Rule

[defaultuserfoo]

After migrating to the new rules, these rules didn't get lost, but the one allowing all traffic doesn't work anymore.  That kinda makes sense because the floating rule would block the traffic before the rule allowing it is reached (leaving me wondering why it worked before).

Maybe the LAN rule was created as a floating rule as well.
But floating rules for a single interface are not possible anymore in the new rules.

No, the LAN rule wasn't floating.  Perhaps it was floating in the old rules and was somehow converted into a non-floating one when it was imported.

The blocking rule was quietly converted from a floating one to a group rule.  So who knows what all was converted and what wasn't.

Why aren't there any warnings being given when rules get converted into other types of rules??

So how do I now make a rule that would allow all traffic from a certain IP address on the LAN interface to everywhere on all interfaces in the interface group that goes before the blocking rule?  I guess it would have to be a floating rule which I can't create ...

I get two possible ways in mind.

• Turn the floating rule into a "last match" rule by removing the Quick check.
• Exclude the certain LAN devices from the source in the block rule. You can achieve this by checking "Invert Source" and entering the IP or alias.

Converting the blocking rule into a 'last match' seems more difficult.  There are quite a few rules, but I think it was the last floating rule anyway because the other rules before it are all rules to allow certain things (which now don't work anymore for unknown reasons).

I'm not sure how inverting the source would help.  I can't very well enter all possible addresses on all the other networks to allow traffic to them from a particular address on the LAN.

Instead --- and that is extremely unintuitive and illogical --- you have to make a group rule by selecting the whole interface group as interface and then limit the rule by selecting the particular address on the LAN as source just because this is the only way to turn a rule which could otherwise be much simpler into a group rule so it can be moved before the blocking rule.  It's overkill to first select --- i. e. allow --- the whole interface group only to limit it to a particular address right away.

Why not select --- i. e. allow --- the LAN interface first, then limit it to the particular address and then move it before the blocking rule?  There is no reason for the simple rule having to be a more complicated group rule in order to be able to move it.

Doing it this way doesn't even seem safe.

[Monviech (Cedrik)]

The easiest way to create a floating rule right now if you really need to for some reason is to create a random loopback interface(Interfaces -> Devices - Loopback), assign it, name it "Floating" for example, and add it as second interface to any rule that should be floating.

But please keep in mind that this is not the intended way to use the ruleset, only a workaround if you really need it for some reason.

-----

Also group and floating are not concepts of PF (the paket engine thats configured) Check out /tmp/rules.debug, there is no floating or group in there. PF only goes by a global order.

There is no special mythical behavior to a floating rule, its just a hardcoded cheat to set it earlier in the global order. Most of the time that is never even needed.

If you unhide the advanced options in a new rule you can see the priority group and the seqence. Ideally, there should only be a global sequence and no priority group (group and floating), but getting to there and resolving the sins of the past is still up in the air.

[defaultuserfoo]

Sure here is the relevant section. Please note group rules are not a new concept, they also exist in the old firewall rules since ages.

Well, I never had to mess with group rules before.  I'm finding interface groups for things like VPNs very strange, if not to say annoying.  What's the point of forcing entirely different, unrelated interfaces into a group?  I have to have specific rules for each connection anyway, and only if I should ever need to have the same set of rules for a bunch of connections (interfaces), only then I should be able to optionally create a group interface.

Floating rules seem fine.  I made some floating rules according to https://forum.opnsense.org/index.php?topic=28447.msg138309#msg138309 because, unless that changed in the meantime, is the only way to deal with IPv6 when you don't have a static prefix.  It required to make an interface group but no group rules.

After this migration it seems that the floating rules need to be converted into group rules because they aren't working anymore.

Floating is an annoying concept that still has to be decided on most likely at some point. For now it is what it is.

Floating is very nice indeed.  Mankind has been dreaming of it for thousands of years before we finally managed floating :)

I think the problem is more with the arbitrary numbering of rules.  What matters is the order of the rules, not their numbers, but with the new rules, things got worse because the arbitrary numbering didn't go away but the distinction between different types of rules did.

I'm not sure if would make sense, but I might suggest to explicitly introduce types of rules, and for each type its own number range.  Each of the number ranges could go from 0 to N to order the rules of each type in a desirable way.  And the overall order of rules would be determined by their types.  It would be more user friendly to have types of rules.

It kinda shows in the GUI with the new rules in that the new rules are in a particular order.  That even begs the question why there are different types of rules.  What's the point of having these types?  It seems the order of the rules matters, not their types ...

All it currently does is preventing me from moving my LAN rule before the blocking rule.  If I could do just that, things would be much easier, and there wouldn't be a need to create overkill rules (unless I'd actually need one).

Also you are not forced to migrate yet. The old rules stick around for 2 more years at least. No rush.

Why didn't it say that?

But I'll have to do it anyway, eventually ...

-----

Since Firewall ‣ Rules [new] and Firewall ‣ Rules implementations exist side by side, there are some additional considerations regarding the processing order of rules.

If a Firewall ‣ Rules [new] filter rule has:

a single interface defined, it is an Interface Rule
a group interface defined, it is a Group Rule
any number of interfaces or one inverted interface defined, it is a Floating Rule

Ah yes, I saw that but couldn't find it when I tried to find it again later.

This makes no sense.  The order of the rules should matter, not what their arbitrary types are.  The GUI to create rules is already the same for all of them.  It only needs to allow us to move our rules to where we want them.

[Monviech (Cedrik)]

Yes that is true, there is already a global sequence and priority groups. Everything is in place for more flexibility. But it has to start somewhere.

Enable advanced mode in a rule, see the sequence and the sort order.

Unhide them inside the Grid, you can do so with the button next to the number of items to be shown.

Please keep your annoyance in check, it's quite hard to go from a historically grown legacy implementation to a new one without having some edge cases. That it seems to overly affect you personally is nothing that could have been anticipated.

https://github.com/opnsense/core/issues/9652

[defaultuserfoo]

The easiest way to create a floating rule right now if you really need to for some reason is to create a random loopback interface(Interfaces -> Devices - Loopback), assign it, name it "Floating" for example, and add it as second interface to any rule that should be floating.

But please keep in mind that this is not the intended way to use the ruleset, only a workaround if you really need it for some reason.

I made floating rules because I needed them, without creating extra interfaces :)  I still need rules that do what the floating ones did, only now floating rules don't work anymore and group rules are required instead.

-----

Also group and floating are not concepts of PF (the paket engine thats configured) Check out /tmp/rules.debug, there is no floating or group in there. PF only goes by a global order.

Mmh, that's what I thought.

If you unhide the advanced options in a new rule you can see the priority group and the seqence. Ideally, there should only be a global sequence and no priority group (group and floating), but getting to there and resolving the sins of the past is still up in the air.

If that's so easy, then why did the GUI prevent me from moving my LAN rule above the blocking rule?  Is that a bug in the GUI?

[defaultuserfoo]

Please keep your annoyance in check, it's quite hard to go from a historically grown legacy implementation to a new one without having some edge cases. That it seems to overly affect you personally is nothing that could have been anticipated.

https://github.com/opnsense/core/issues/9652

Don't be surprised when users are very annoyed.  Having the so-called migration assistant and no more warning than to back up your configuration and/or to take a snapshot makes users think that it's an easy and simple migration that will take maybe 5 minutes, if that, and that in the unlikely case in which one might find an issue, it'll be easily resolved by skimming over the exported rules in ones favourite spreadsheet.

You need to tell users that this is NOT a migration but merely an export and import of existing rules.  You need to tell them that their old rules will inevitably mean something else in the new rule section and that the old rules will only give you some clue as to what was intended with the old rules.  You need to tell them that they will have to set up their whole firewall again as if from scratch.

You also need to tell them that they have at least 2 years before making this change and that they better don't do it but wait until they actually want to rebuild their firewall from scratch and do that with the new rules.

Maybe also tell them that the new rules are a work in progress which is in its early stages; that it will be improved upon and that it might be easier later to create a new firewall with the new rules.

Only if you do that, the users may not be too annoyed but might even be glad that there was an improvement made (or prepared) by switching to the new rules.

[nero355]

Also group and floating are not concepts of PF

But where do they come from then and why were they invented the way they are ?

I know you guys basically forked pfSense but I have never worked enough with pfSense to know all the "ins & outs" about the Firewall part :)

Maybe also tell them that the new rules are a work in progress which is in its early stages; that it will be improved upon and that it might be easier later to create a new firewall with the new rules.

I totally agree with you on this one considering the amount of people on this forum that did not know about the fact that the migration is not mandatory at all and can be ignored for a long time into the future :)

[OPNenthu]

But where do they come from then and why were they invented the way they are ?

I didn't know this coming in to OPNsense but interface groups are a BSD networking construct, not at all related to the firewall: https://man.freebsd.org/cgi/man.cgi?ifconfig. 

It makes sense now why groups are printed in 'ifconfig' output and I can appreciate why developers would want to let us define rules at that level.  It might be messier in the back end ruleset and lead to higher rule counts, but as an administrative abstraction it's quite useful.  Especially paired with non-'quick' rules that can be overridden at interface level (though these can lead to subtle ruleset bugs which are not obvious if they're used sloppily; you have to mind the ordering of pass/block rules and the successive scope of the matches).

The 'floating' concept confused me as well because the pf manual makes reference to 'floating' also, so as a newcomer I conflated them.  Totally separate things: pf talks only about floating states, not rules.  @Monviech's explanation above is much needed context for new users, IMO.