Messages

That is all done and it's in the PR.

Yeah, that dude just copied two lines from a previous post of mine in this thread without context. Can't tell if they're trying to make a point or if it's some kind of spam test, honestly.

My largest IP alias used for blacklisting has well over 100,000 entries and performance of the rule is great. You should be fine.

finding out what is made more difficult by the lack of a command prompt and file functions via the GUI.

I understand where you're coming from, the 19.1.6 upgrade was pretty frustrating for me, too.  That said, the command prompt is always available via SSH and the filesystem is exposed via SFTP, provided you turn on SSH administration. I don't think they should expose that via the web UI as it would involve significant development overhead with very little benefit over the existing native services.

For what it's worth, I also reinstalled from scratch at one point to wipe out some lingering bugs, but generally speaking the config file is abstracted away enough from the rest of the system that you can usually restore it without fear of bringing back the odd glitch. In this case, for example, restored aliases worked fine.

This would be great. OSQuery is only going to get more popular. While I'm not sure if it should be included in core, it would be a very valuable plugin to have.

Update:
I was able to get aliases working again by removing all three "|encode_idna" in file /usr/local/opnsense/service/templates/OPNsense/Filter/filter_tables.conf for now.

Good tip. You may also be able to use opnsense-patch to roll back to before the idna patch a couple months ago. I did not have any luck patching forward to the latest commit. I just rolled my snapshot back to 9.1.4 until I have more time to look at it or until the devs release an official hotfix/update.

Okay, I was looking before at the General logs when I (obviously) should have been looking at the Backend logs. In 9.1.6 when creating any new alias I get the following error:

Code Select Expand

Apr 12 22:03:40 configd.py: [d15a2999-00f5-4a54-9026-5dbec36e63b0] refresh url table aliases
Apr 12 22:03:39 configd.py: [bbf98ba9-1d60-4a76-ad76-59ecd919e3c4] Reloading filter
Apr 12 22:03:38 configd.py: [ece01db5-8606-4134-b5dd-8d01e6d6496f] Inline action failed with OPNsense/Filter OPNsense/Filter/filter_tables.conf label empty or too long at Traceback (most recent call last): File "/usr/local/opnsense/service/modules/processhandler.py", line 509, in execute return ph_inline_actions.execute(self, inline_act_parameters) File "/usr/local/opnsense/service/modules/ph_inline_actions.py", line 51, in execute filenames = tmpl.generate(parameters) File "/usr/local/opnsense/service/modules/template.py", line 332, in generate raise render_exception Exception: OPNsense/Filter OPNsense/Filter/filter_tables.conf label empty or too long
Apr 12 22:03:38 configd.py: generate template container OPNsense/Filter
Apr 12 22:03:38 configd.py: [ece01db5-8606-4134-b5dd-8d01e6d6496f] generate template OPNsense/Filter

I also found issue #3399 on GitHub which may describe the problem.

It's not obvious how DUID-EN would be used with a software router such as OPNsense, except to provide a way to comply with an ISP that requires the DUID-EN format.

Yeah, that was my situation. AT&T service in the U.S. requires the use of a DUID-EN for IPv6. This is fine as the huge majority of customers use an AT&T provided gateway that connects to the ONT for fiber or copper for DSL. I did not want to use the gateway as it has tiny NAT tables, breaks prefix delegation, and causes issues with random address renewals.

As long as it's documented that the DUID-EN can be set manually I don't see it as a super high priority feature.

I think it's more than dynamic aliases - I have several port aliases setup that quit working also. I've been trying to figure out why, but keep getting distracted.

Thanks for confirming. Agreed, after retracing my steps and troubleshooting the issue, it does not seem to be related to dynamic aliases. In my case almost all alias functionality is broken by upgrading to the 9.1.6 release, except aliases that were created before the upgrade.

I'll create a github issue in a bit. Hopefully we can get some dev eyes on this, as it has potential to break firewall rules in production, if it's a bug and not an obscure issue with my configuration.

Alright. Thanks to the magic of snapshots I can confirm that alias functionality works in 19.1.4 as expected, including alias deletion (and related pftable). However, immediately after upgrading to 19.1.6 all new alias functionality is broken. Creation of static host aliases fails to fill the pf table, deletion of aliases fails to remove the pf table (even after reboot), etc. No "dynamic" aliases have to exist for me to encounter this bug.

I am not familiar with the codebase. Can anyone think of any changes to aliasing or in the upgrade process in 19.1.6 that would cause this?

This is not a small problem so I'm rolling back to my 19.1.4 snapshot for now. I have the fresh 19.1.6 snapshot available to test, as well.

Interestingly, OPNSense also fails to download bogon info over IPv6. "Prefer IPv4 even when IPv6 is available" must be turned on.

Update: More details. Does not seem to have to do with dynamic aliases.

Still having issues. After removing all dynamic aliases, creating a fresh install, reloading my config, and creating a new static host alias successfully, I attempted to create a geoIP alias. The pf table for this alias stayed blank, new static host aliases no longer fill the corresponding pf table (blank), and pf tables are not killed upon deletion of the alias.

Something's very wrong. It's like creating any "dynamic" alias (geoip, url table, etc) poisons OPNsense's alias functionality for me.

Can someone try this on their system for me? Create a new geoip alias for a random region. See if the pf table is filled. Then try deleting it.

Update: Still having issues.

Thank you, Steven, good suggestion. I've actually removed all of my geoIP and iplist aliases from my configuration and done another fresh install to remove persistent pfTables. So far new aliases are working fine, I'll see what happens when I recreate them. I upped my table entry limit on the new disk image from 400k to 800k.

I run OPNSense in a VM and still have my faulty disk image to investigate further. I did not see any log errors before about reaching my table entry limit, though admittedly I had several large tables.

I still find it odd that I could not kill the tables or aliasdb files corresponding to deleted aliases. They just kept coming back. Since I don't fully understand the backend mechanisms behind how those tables are built yet, I hadn't figured out how to get them to fully go away. This part seems like a possible bug, even if I was hitting table entry limits.

Yeah, I'm aware of the apply button. Does anyone have troubleshooting suggestions for this?

I'm having an odd and difficult to diagnose problem with both the latest production (OPNsense 19.1.6-amd64) and development branch releases.

• If I create a host alias the corresponding pf table does not populate with the addresses and rules do not work. This is worrying.
• Deleting alias does not delete corresponding pf tables. I have tried killing the tables manually as well as removing related /var/db/aliastables files. The pf tables continue to be recreated despite not existing as aliases or being referenced in any rules, even after reboot. The alias data is not found in config.xml, so I'm not sure where it persists.
• New aliases of any type do not seem to get populated. While my deleted geoip alias continues to be recreated, new geoip alias tables remain blank, breaking my rules.

To try to diagnose this, I have:

• Rebooted
• Tried killing tables, deleting aliases, recreating aliases, running update_tables.py, configctl filter refresh_aliases, etc.
• Reinstalled OPNSense from scratch, after which my existing config aliases work fine. New aliases continue not to work, newly deleted aliases continue to exist as pf tables.
• Checked logs for any useful info, of which there is none.

Nothing has seemed to make aliasing work correctly in the current versions. I cannot find other complaints about this problem. I have not had this issue until recently. What could be going on?

[enterprise number]

Sure, it's not actually difficult, I just wasn't sure that it would accept a manually formatted string. Having it do the conversion in the UI would be helpful.

With DUID-EN you have two pieces of static info: the enterprise number (e.g., 3562) and the identifier (e.g., xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx). These are assigned by the ISP/enterprise.

• First, we have to indicate that this is a type 2 DUID (EN) by prepending two octets: "00:02"
• Then, we have to convert the decimal enterprise number to hex and format it as four octets, so "3562" becomes "00:00:0D:EA". Append it to the type indicator.
  
• Lastly, we have to append the DUID identifier.

So the final DUID-EN string for the DHCPv6 client becomes: "00:02:00:00:0D:EA:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx"