Large IP Blacklists...performance impact?

[labsy]

Hi,

I am thinking about to aggregate all IP blacklists from various web sites (WP, Joomla, custom builds...), which write logs of attacking (brute-force, dictionary attacks...) IP hosts/addresses into database. I have a script in PHP to extract IP's from database for past 7 or 14 days.
Then I have plan to try/test retreive these into BLocked ALIASES list of OPNSense.

Now, since this list will contain hundreds or even thousands of IP addresses, I am wondering how a 1000's of BLOCK ALIASES LIST would affect firewall performance?

[gstuartj]

My largest IP alias used for blacklisting has well over 100,000 entries and performance of the rule is great. You should be fine.

[labsy]

Great news.
Anyone knows how often do these aliases reload from external source? And more important...how can I check, if they are loaded?

[firewall]

check whether or not they've downloaded:
ls -al /var/db/aliastables

the revision date on mine is 3 days ago but i'm not certain how often it kicks off.

[labsy]

Mine was reloaded yesterday, right after I created the list Alias.

I tried to add CRON job to test and check every 5 minutes for "Update and reload firewall aliases"...but after half an hour the directory of aliases tables still shows yesterday's date. So there must be some other settings, which control frequency of Alias Table list refresh and reload.

[labsy]

Actually...how can I check if IP addresses were properly retreived and accepted by OPNSense?
I have them in format:
1.2.3.4
1.2.3.4
1.2.3.4
And filename is list.php, because it is dynamic and it generates fresh list each time file is displayed.
Is this proper format? How to verify?

*** EDIT ***
Solved! Found out myself!
there were 2 glitches:

1.) The called web site with public list is behind NAT and needs to have SplitDNS configured to be reachable from inside. In OPNSense it is under Services --> Unbound DNS --> Overrides --> Host Overrides

2.) There are actually TWO TYPES of ALIAS lists, URL and URL Table. First one is one-time static, and only second one is dynamic with expiration time.
If you select Type of Alias "URL (IPs)", then it seems to load only once, and requested format is unknown to me.
But if you select Alias Type as "URL Table (IPs)", then format is as above and you can set Expiration time, like 1 hour and it will reload once per hour. Tested & working!

If anyone is interested into sharing the list, here's the link:
http://secureit.si/lockouts/list.php
I might keep it alive for quite some time.