Aliasing completely broken for me recently

[gstuartj]

I'm having an odd and difficult to diagnose problem with both the latest production (OPNsense 19.1.6-amd64) and development branch releases.

• If I create a host alias the corresponding pf table does not populate with the addresses and rules do not work. This is worrying.
• Deleting alias does not delete corresponding pf tables. I have tried killing the tables manually as well as removing related /var/db/aliastables files. The pf tables continue to be recreated despite not existing as aliases or being referenced in any rules, even after reboot. The alias data is not found in config.xml, so I'm not sure where it persists.
• New aliases of any type do not seem to get populated. While my deleted geoip alias continues to be recreated, new geoip alias tables remain blank, breaking my rules.

To try to diagnose this, I have:

• Rebooted
• Tried killing tables, deleting aliases, recreating aliases, running update_tables.py, configctl filter refresh_aliases, etc.
• Reinstalled OPNSense from scratch, after which my existing config aliases work fine. New aliases continue not to work, newly deleted aliases continue to exist as pf tables.
• Checked logs for any useful info, of which there is none.

Nothing has seemed to make aliasing work correctly in the current versions. I cannot find other complaints about this problem. I have not had this issue until recently. What could be going on?

[bewue]

To create the pfTable entries you have to press the apply button.
(Firewall: Aliases -> Apply)

[gstuartj]

Yeah, I'm aware of the apply button. Does anyone have troubleshooting suggestions for this?

[Steven]

My hosts alias and geoip alias are working in 19.1.5_1.

Please check your "Firewall Maximum Table Entries" the default is 200000 which may be too small to hold all the alias IPs, especially if you use a lot of GeoIP Aliases or URL Table (IPs). I had the same issue and had to fix it by increasing the Firewall Maximum Table Entries value. I changed mine to 800000 but you will need to tweak it to what works for you.

Firewall -> Settings -> Advanced -> "Firewall Maximum Table Entries"

After increasing the Firewall Maximum table Entries, Update Bogons (Firewall -> Diagnostics -> pfTables -> "Update bogons" button) which should also update the geoips.

Then check your System Logs to see if the geoip and bogons were updated, or if you have a table-entries limit warnings. (System -> Log Files -> General). See attachment for example.

[gstuartj]

Update: Still having issues.

Thank you, Steven, good suggestion. I've actually removed all of my geoIP and iplist aliases from my configuration and done another fresh install to remove persistent pfTables. So far new aliases are working fine, I'll see what happens when I recreate them. I upped my table entry limit on the new disk image from 400k to 800k.

I run OPNSense in a VM and still have my faulty disk image to investigate further. I did not see any log errors before about reaching my table entry limit, though admittedly I had several large tables.

I still find it odd that I could not kill the tables or aliasdb files corresponding to deleted aliases. They just kept coming back. Since I don't fully understand the backend mechanisms behind how those tables are built yet, I hadn't figured out how to get them to fully go away. This part seems like a possible bug, even if I was hitting table entry limits.

[gstuartj]

Update: More details. Does not seem to have to do with dynamic aliases.

Still having issues. After removing all dynamic aliases, creating a fresh install, reloading my config, and creating a new static host alias successfully, I attempted to create a geoIP alias. The pf table for this alias stayed blank, new static host aliases no longer fill the corresponding pf table (blank), and pf tables are not killed upon deletion of the alias.

Something's very wrong. It's like creating any "dynamic" alias (geoip, url table, etc) poisons OPNsense's alias functionality for me.

Can someone try this on their system for me? Create a new geoip alias for a random region. See if the pf table is filled. Then try deleting it.

[gstuartj]

Interestingly, OPNSense also fails to download bogon info over IPv6. "Prefer IPv4 even when IPv6 is available" must be turned on.

[gstuartj]

Alright. Thanks to the magic of snapshots I can confirm that alias functionality works in 19.1.4 as expected, including alias deletion (and related pftable). However, immediately after upgrading to 19.1.6 all new alias functionality is broken. Creation of static host aliases fails to fill the pf table, deletion of aliases fails to remove the pf table (even after reboot), etc. No "dynamic" aliases have to exist for me to encounter this bug.

I am not familiar with the codebase. Can anyone think of any changes to aliasing or in the upgrade process in 19.1.6 that would cause this?

This is not a small problem so I'm rolling back to my 19.1.4 snapshot for now. I have the fresh 19.1.6 snapshot available to test, as well.

[va176thunderbolt]

I think it's more than dynamic aliases - I have several port aliases setup that quit working also. I've been trying to figure out why, but keep getting distracted.

[gstuartj]

I think it's more than dynamic aliases - I have several port aliases setup that quit working also. I've been trying to figure out why, but keep getting distracted.

Thanks for confirming. Agreed, after retracing my steps and troubleshooting the issue, it does not seem to be related to dynamic aliases. In my case almost all alias functionality is broken by upgrading to the 9.1.6 release, except aliases that were created before the upgrade.

I'll create a github issue in a bit. Hopefully we can get some dev eyes on this, as it has potential to break firewall rules in production, if it's a bug and not an obscure issue with my configuration.

[Steven]

Something's very wrong. It's like creating any "dynamic" alias (geoip, url table, etc) poisons OPNsense's alias functionality for me.

Can someone try this on their system for me? Create a new geoip alias for a random region. See if the pf table is filled. Then try deleting it.

I am able to create geoip alias, see them in the pfTable, and then delete them without issue. The pfTable will still have them listed until I reboot.

I'm on OPNsense 19.5.1_1 though, I haven't upgraded to 19.1.6 yet.

[gstuartj]

Okay, I was looking before at the General logs when I (obviously) should have been looking at the Backend logs. In 9.1.6 when creating any new alias I get the following error:

Code: [Select]

Apr 12 22:03:40 configd.py: [d15a2999-00f5-4a54-9026-5dbec36e63b0] refresh url table aliases
Apr 12 22:03:39 configd.py: [bbf98ba9-1d60-4a76-ad76-59ecd919e3c4] Reloading filter
Apr 12 22:03:38 configd.py: [ece01db5-8606-4134-b5dd-8d01e6d6496f] Inline action failed with OPNsense/Filter OPNsense/Filter/filter_tables.conf label empty or too long at Traceback (most recent call last): File "/usr/local/opnsense/service/modules/processhandler.py", line 509, in execute return ph_inline_actions.execute(self, inline_act_parameters) File "/usr/local/opnsense/service/modules/ph_inline_actions.py", line 51, in execute filenames = tmpl.generate(parameters) File "/usr/local/opnsense/service/modules/template.py", line 332, in generate raise render_exception Exception: OPNsense/Filter OPNsense/Filter/filter_tables.conf label empty or too long
Apr 12 22:03:38 configd.py: generate template container OPNsense/Filter
Apr 12 22:03:38 configd.py: [ece01db5-8606-4134-b5dd-8d01e6d6496f] generate template OPNsense/Filter
I also found issue #3399 on GitHub which may describe the problem.

[camouflageX]

Oh no, I just upgraded to 19.1.6 and have the same problem!

I created a new alias and tried to use it in a new firewall rule, but the rule does not work.

when creating a rule, I get the following message in Backend log:
configd.py: [4b57c046-1239-4414-975f-c686fb8fd54f] Inline action failed with OPNsense/Filter OPNsense/Filter/filter_tables.conf label empty or too long at Traceback (most recent call last): File "/usr/local/opnsense/service/modules/processhandler.py", line 509, in execute return ph_inline_actions.execute(self, inline_act_parameters) File "/usr/local/opnsense/service/modules/ph_inline_actions.py", line 51, in execute filenames = tmpl.generate(parameters) File "/usr/local/opnsense/service/modules/template.py", line 332, in generate raise render_exception Exception: OPNsense/Filter OPNsense/Filter/filter_tables.conf label empty or too long

Update:
I was able to get aliases working again by removing all three "|encode_idna" in file /usr/local/opnsense/service/templates/OPNsense/Filter/filter_tables.conf for now.

[gstuartj]

Update:
I was able to get aliases working again by removing all three "|encode_idna" in file /usr/local/opnsense/service/templates/OPNsense/Filter/filter_tables.conf for now.

Good tip. You may also be able to use opnsense-patch to roll back to before the idna patch a couple months ago. I did not have any luck patching forward to the latest commit. I just rolled my snapshot back to 9.1.4 until I have more time to look at it or until the devs release an official hotfix/update.

[Mks]

Same issue. Old Alias are working, new ones are emtpy.

Error message

Code: [Select]

configd.py: [5c57e160-70ec-44c2-bee6-c84bd10f3acf] Inline action failed with OPNsense/Filter OPNsense/Filter/filter_tables.conf label empty or too long at Traceback (most recent call last): File "/usr/local/opnsense/service/modules/processhandler.py", line 509, in execute return ph_inline_actions.execute(self, inline_act_parameters) File "/usr/local/opnsense/service/modules/ph_inline_actions.py", line 51, in execute filenames = tmpl.generate(parameters) File "/usr/local/opnsense/service/modules/template.py", line 332, in generate raise render_exception Exception: OPNsense/Filter OPNsense/Filter/filter_tables.conf label empty or too long

br