NAT issue using aliases

[siga75]

I have a port forwarding defined for let go out only some ip addresses, through squid.

The alias used (https_www_proxied_hosts) does not seems to contains everything defined

See attachments for details

I defined a cron job to run every 5 minutes to reload the aliases

See here the content of the pf table for alias google_networks

[root@myfw ~]# pfctl -t google_networks -T show
   35.190.247.0/24
   35.191.0.0/16
   64.233.160.0/19
   66.102.0.0/20
   66.249.80.0/20
   72.14.192.0/18
   74.125.0.0/16
   108.177.8.0/21
   108.177.96.0/19
   130.211.0.0/22
   172.217.0.0/19
   172.217.32.0/20
   172.217.128.0/19
   172.217.160.0/20
   172.217.192.0/19
   172.253.56.0/21
   172.253.112.0/20
   173.194.0.0/16
   209.85.128.0/17
   216.58.192.0/19
   216.239.32.0/19

That should be included in this table, but completely missing:

[root@myfw ~]# pfctl -t https_www_proxied_hosts -T show
   23.23.73.124
   50.19.218.16
   54.204.26.223
   54.225.71.235
   54.225.169.250
   54.235.203.7
   54.235.220.229
   54.243.147.226
   104.17.172.102
   104.18.48.62
   104.18.49.62
   104.18.206.87
   104.24.110.187
   104.24.111.187
   104.31.90.50
   104.31.91.50
   104.131.209.4
   108.171.202.195
   108.171.202.203
   108.171.202.211
   172.217.168.4
   172.217.168.10
   172.217.168.14
   172.217.168.42
   172.217.168.45
   172.217.168.46
   172.217.168.78
   198.143.164.0/24
   198.143.164.251
   198.143.164.252
   216.58.215.225
   216.58.215.234
   2606:4700::6811:ac66
   2606:4700::6812:ce57
   2606:4700:3033::6818:6fbb
   2606:4700:3035::6812:313e
   2606:4700:3037::6812:303e
   2606:4700:3037::6818:6ebb
   2606:4700:3037::681f:5a32
   2606:4700:3037::681f:5b32
   2607:f2d8:4010:8::2
   2607:f2d8:4010:b::2
   2607:f2d8:4010:c::2
   2a00:1450:400a:800::2001
   2a00:1450:400a:800::200a
   2a00:1450:400a:801::2004
   2a00:1450:400a:801::200e
   2a00:1450:400a:802::200a
   2a00:1450:400a:802::200d
   2a00:1450:400a:802::200e

Any idea what's the issue here? I saw in the forum in the past there was a bug when using aliases for NAT rules, and here I use an external alias (google_networks) that may complicate the stuff. Could this be the issue?

THX

[siga75]

I confirm that's the issue (external table alias), if I create an alias with the same content but "static" it works. Should I open an issue on github?

[hbc]

How do you fill your external alias? As far as I understand your cronjob just reloads the alias (which is the default refresh of aliases anyway).
Your commands just show the alias content, but do not add entries, so what is your mechanism to fill the alias?

Externally managed alias, this only handles the placeholder. Content is set from another source (plugin, api call, etc)

https://docs.opnsense.org/manual/aliases.html

[siga75]

That alias works on others fw rules, there's a script scheduled that run a pfctl replace

[root@myfw ~]# cat google-nets
#!/bin/sh
dig @8.8.8.8 +noall +answer +short +dnssec +tcp _spf.google.com txt | tr ' ' '\n' | awk -F: '$1=="include" {print $2}' | while read blocks
do
  dig @8.8.8.8 +noall +answer +short +dnssec +tcp "$blocks" txt | tr ' ' '\n' | awk -F: '$1=="ip4" {print $2}'
done | xargs pfctl -t google_networks -T replace

[root@myfw ~]# cat /usr/local/etc/cron.d/custom-pf-tables.cron
SHELL=/bin/sh
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
#minute hour    mday    month   wday    who     command
40      4       *       *       *       root    /root/google-nets

I wrote a small post here:
https://www.signorini.ch/content/opnsense-create-pftable-for-google-networks

[hbc]

And your scripts always returns content? No timeouts that may create empty results?

If static aliases work, I would assume either empty script results or problems with merging the aliases into 'https_www_proxied_hosts'

[siga75]

thanks for your reply

And your scripts always returns content? No timeouts that may create empty results?

As far as I know, yes, it always get results, this is confirmed running this command while I have the issue

[root@myfw ~]# pfctl -t google_networks -T show
   35.190.247.0/24
   35.191.0.0/16
   64.233.160.0/19
   66.102.0.0/20
   66.249.80.0/20
   72.14.192.0/18
   74.125.0.0/16
   108.177.8.0/21
   108.177.96.0/19
   130.211.0.0/22
   172.217.0.0/19
   172.217.32.0/20
   172.217.128.0/19
   172.217.160.0/20
   172.217.192.0/19
   172.253.56.0/21
   172.253.112.0/20
   173.194.0.0/16
   209.85.128.0/17
   216.58.192.0/19
   216.239.32.0/19

problems with merging the aliases into 'https_www_proxied_hosts'

I really think this is the issue.

Next thing I will try is creating a dedicated port forwarding with only "google_networks" alias, instead of having nested aliases.