Firewall Aliases No Longer Updating/Reloading

Started by tonys, March 21, 2025, 09:49:17 PM

Previous topic - Next topic
Hi all,

I'm running OPNsense 25.1.3-amd64 FreeBSD 14.2-RELEASE-p2 OpenSSL 3.0.16.

Over the past two days, my firewall alias tables are no longer refreshing or updating on a daily basis. Things were running fine prior to 3/19/25 and as I added new URL Table IP's, they got pulled in and blocked. The latest IP's are no longer being blocked. I read through many posts here as well as the OPNSense docs but none of the suggestions here are working.

I added a cron job to refresh/reload the aliases (see the Cron attachment) every 30 minutes but no change. I checked the alias updates and it's indeed set to Daily, still no luck. I'm out of ideas short of manually hacking the XML file to add new IP's but that probably wouldn't force a reload either.

Is there any way to force a refresh/reload? I can't find this in the docs anywhere. It's clear from the Last Update attachment that only 33 IPs are loaded in the alias table and the two most recent additions (see Alias Table attachment) are not loaded since last update was two days ago prior to me adding these additional IPs. The first 33 IP's are being blocked properly but unfortunately (for me), the last two entries are hitting my DMZ server hard with over 60,000 attempts in the past two days. I really need to get these loaded into the tony_bogons table ASAP.

Ideas? Thanks...

I don't think */30 is a valid entry, or if it is then you got yourself a block from the list owner(s) for hammering their servers. Usually once a day is acceptable for most list providers.

You get any error in the logs, can the firewall rule successfully be loaded?
Deciso DEC740

Quote from: newsense on March 21, 2025, 10:08:10 PMI don't think */30 is a valid entry, or if it is then you got yourself a block from the list owner(s) for hammering their servers. Usually once a day is acceptable for most list providers.

Actually, this is a perfectly valid entry as shown in https://www.codementor.io/@akul08/the-ultimate-crontab-cheatsheet-5op0f7o4r:

"Every 10 Minutes of Every Day
# m h dom mon dow command
  */10 * * * * /home/user/script.sh"

The cron job should run every 30 minutes in my case. Also, there is no list owner, it's my own, 35-IP address table inputed manually to the alias table in OPNSense. And it's my own DMZ'd server being hammered by both the last IP (a Russian bot documented as a criminal organization) and the second to last IP.

Quote from: patient0 on March 21, 2025, 10:25:28 PMYou get any error in the logs, can the firewall rule successfully be loaded?

I'm not sure which firewall log file holds these error messages if they exist. The firewall rules are fine and working as evidenced by the first 33 entries in the alias table being blocked. It's the alias table itself that's not being reloaded/refreshed.

You should probably read the documentation on "URL tables" again. Its content should be a URL that points to a web resource that contains a list of IPs, not the IPs or networks themselves. Your tony_bogons are of type URL table, but the contents seem to be a list of IPs and networks that would fit a "networks" type alias.

Because there are different alias types, it is syntactically possible to enter something like "192.168.0.0/16" into that field, but what is expected for a URL table alias is something like "https://iplists.firehol.org/files/firehol_level2.netset".

The way you use it, it will never work, and that is regardless of updates and specific OpnSense version.
Intel N100, 4 x I226-V, 16 GByte, 256 GByte NVME, ZTE F6005

1100 down / 800 up, Bufferbloat A+

Quote from: meyergru on March 21, 2025, 11:13:28 PMYou should probably read the documentation on "URL tables" again. Its content should be a URL that points to a web resource that contains a list of IPs, not the IPs or networks themselves. Your tony_bogons are of type URL table, but the contents seem to be a list of IPs and networks that would fit a "networks" type alias.

Because there are different alias types, it is syntactically possible to enter something like "192.168.0.0/16" into that field, but what is expected for a URL table alias is something like "https://iplists.firehol.org/files/firehol_level2.netset".

The way you use it, it will never work, and that is regardless of updates and specific OpnSense version.


Based on your recommendation, I changed the tony_bogons type to "Networks", saved it, applied it, and then rebooted OPNSense. Unfortunately, this didn't resolve the reload/refresh issue. The same 33 original IP's are still loading and the remaining 2 IP's are not. The Diagnostics Alias page also shows only the original 33 IP's. Still no updates to the alias tables (both tony_bogons and the Block Regions) since March 19th. The question here is why aren't the tables refreshing/reloading all of a sudden when they were working fine up until two days ago? How do I get these two lists to reload/refresh?

Quote from: tonys on March 21, 2025, 10:48:28 PMI'm not sure which firewall log file holds these error messages if they exist. The firewall rules are fine and working as evidenced by the first 33 entries in the alias table being blocked. It's the alias table itself that's not being reloaded/refreshed.
Check the # of tables entries / max tables entries and the logs
Deciso DEC740

March 22, 2025, 09:04:09 AM #8 Last Edit: March 22, 2025, 09:07:53 AM by troplin
Did you actually hit the ,,Apply" button at the bottom of the Aliases page after adding the new entries?

A ,,Network(s)" alias is completely static, the contents don't change unless you change them yourself (or if the alias references other dynamic aliases).
So it's completely expected that the update date doesn't change as well.

Also, the Cron job seems completely pointless to me. Dynamic aliases (e.g. ,,URL Tables") are already updated automatically without an explicit Cron job, based on the individual update settings of that specific alias.

I recreated the exact list of your networks and did not see this problem, no matter how I tried (by first adding 33 entries, then 2 or using them all at once.

Also, when I first created the alias as URL table, I never saw 33 or 35, but 0 entries (as expected).

If anyone wants to check, here is the list as clear text:

198.199.71.0/24
167.99.49.0/24
198.199.119.0/24
122.189.51.0/24
20.42.92.0/24
162.142.125.0/24
159.65.217.0/24
67.184.162.20
154.212.141.0/24
167.71.254.235
143.198.106.164
144.126.222.41
45.62.170.97
174.138.70.162
23.227.203.0/24
206.189.199.217
165.232.145.190
146.190.158.29
193.41.206.0/24
206.189.198.88
178.212.35.89
134.122.118.203
79.124.49.10
170.39.218.98
212.102.40.218
47.251.53.147
45.92.19.136
142.93.196.180
142.93.205.51
196.251.87.51
77.83.36.160
45.141.0.0/16
218.92.0.0/16
45.55.0.0/16
91.220.163.0/24

There are no overlapping or conflicting ranges. The input field detects exact duplicates and keeps you from even saving non-networks like when you put a blank in like " 1.2.3.4".

If you still suspect a problem, you should be able to explain an exact series of steps on how to make it show and file a bug on github. You can also closely inspect the real content of the alias via the copy and paste buttons to look at the source text.
Intel N100, 4 x I226-V, 16 GByte, 256 GByte NVME, ZTE F6005

1100 down / 800 up, Bufferbloat A+

Here's what I think happened:
  • At one point the alias had the correct type ,,Network(s)" and correct 33 entries.
  • Then you switched it to the incorrect type ,,URL Tables" for reasons unknown to me.
  • Since the ,,URLs" are obviously not valid blocklist URLs, the (resolved) alias content is not updated and still contains those 33 networks.
  • While the type is still ,,URL Tables" you added two networks. But still none of the values are valid blocklist URLs, so the table still contains the last valid list, i.e those 33 networks.
  • Now you have changed the type back to ,,Network(s)" but since the content field hasn't changed and this is not a dynamic type, the (resolved) alias table isn't updated.

I can reproduce the behavior. I guess this wasn't considered a valid use case (understandably so).

Just make some actual changes to the contents field and the alias will be updated.

March 22, 2025, 04:31:38 PM #11 Last Edit: March 22, 2025, 04:55:19 PM by tonys
Quote from: troplin on March 22, 2025, 01:42:38 PMHere's what I think happened:
  • At one point the alias had the correct type ,,Network(s)" and correct 33 entries.
  • Then you switched it to the incorrect type ,,URL Tables" for reasons unknown to me.
  • Since the ,,URLs" are obviously not valid blocklist URLs, the (resolved) alias content is not updated and still contains those 33 networks.
  • While the type is still ,,URL Tables" you added two networks. But still none of the values are valid blocklist URLs, so the table still contains the last valid list, i.e those 33 networks.
  • Now you have changed the type back to ,,Network(s)" but since the content field hasn't changed and this is not a dynamic type, the (resolved) alias table isn't updated.

I can reproduce the behavior. I guess this wasn't considered a valid use case (understandably so).

Just make some actual changes to the contents field and the alias will be updated.

Both you and meyergru were correct in your assumptions. I originally had the alias table typed as "Networks" but did indeed change it to "URL Tables" after reading the docs which are a bit confusing regarding which types update each day. After changing the type back to "Networks", the table updated overnight and I now have all 35 entries showing up. As for duplicate IPs/ranges, you're correct that these can't be entered into the table but it sure would be nice if OPNSense flagged an error when it automatically deletes duplicate entries. I've seen some entries get deleted but didn't realize they were duplicates at the time. Eventually, I figured out why they were disappearing.

Regarding the 30-minute cron job, this was suggested as a solution in another post similar to mine. Based on the time of the update, it appears that the cron job did not update the table 30 minutes later as expected so I've disabled it for the time being. I just added 4 more entries and the table updated immediately.

Thanks to both of you. Newbies can be dangerous :-)