Messages

you're probably right there's a problem with python. Can you list your installed plugins and run an $unbound-checkconf

I am going to restore a duplicate of Opnsense 24.1.6 and go through the 24.1.8 upgrade again.  I'll run unbound check against it.

But for my current 24.1.6:

unbound-checkconf: no errors in /usr/local/etc/unbound/unbound.conf

Plugins
os-acme-client (installed)   4.2   
os-bind (installed)   1.31   
os-c-icap (installed)   1.7_4      
os-cache (installed)   1.0_1   
os-clamav (installed)   1.8   
os-ddclient (installed)   1.21_2   
os-dnscrypt-proxy (installed)   1.15   
os-freeradius (installed)   1.9.22   
os-frr (installed)   1.39_1   
os-haproxy (installed)   4.3   
os-intrusion-detection-content-pt-open (orphaned)   1.0_1   
os-iperf (installed)   1.0_1   
os-lldpd (installed)   1.1_2   
os-net-snmp (installed)   1.5_3   
os-netdata (installed)   1.2_1   
os-nginx (installed)   1.32.2   
os-ntopng-enterprise (installed)   6.1.240515   
os-postfix (installed)   1.23_3   
os-redis (installed)   1.1_2   
os-sensei (installed)   1.17.2   
os-sensei-updater (installed)   1.17   
os-shadowsocks (installed)   1.0_2   
os-sunnyvalley (installed)   1.4_3   
os-theme-cicada (installed)   1.35   
os-vmware (installed)   1.5_1   
os-web-proxy-sso (installed)   2.2_3   
os-zabbix64-agent (installed)   1.13_2   

Why not use TLS-ALPN-01 or HTTP-01 challenge instead? On the OPNsense, os-acme-client and os-caddy can do those for you just fine, with IPv4 and IPv6, so if CGNAT not an issue if you have IPv6 too.

Well I've yet to learn about newer TLS-ALPN-01 method since DNS01 been working.
HTTP-01 I know I need port 80.  None of my NGINX reverse proxy sites are currently public nor http enabled.  Just rules set to allow specific known WAN IPs.  Yes - I suppose I could temporarily open up port 80 to all or identify lets encrypt IPs to allow inbound.  Just thought manually creating txt would be the simplest but seems not as acme.sh insists on using API and doing it itself.

The goal I want to achieve is to allow internet access to VLAN10 and VLAN20 (Private_networks alias) that is unrestricted and can get to any destination on the internet but specific hosts within  VLAN30 I want to restrict which websites it can get to, for argument sake www.microsoft.com.

I don't use URL tables aliases myself within opnsense as I have a transparent Forigate firewall in front of opnsense that handles specific host domain rules, but with what your aiming for here I believe this should work.

URL Tables (IPs)
A table of IP addresses that are fetched on regular intervals.

Create alias host groups of specific IPs in vlan 30.
Create URLs tables with domains.
Create LAN INT block rule referencing source host group + destination URL tables
Create LAN INT VLAN 30 rule below URL tables rule with destination any IP allow.

Zenarmor is also capable of web based / app filtering.  I 've setup different Zenarmor policies to capture an entire vlan subnet though and not specific IPs.

You should rather transfer your domains to a different provider. Why stay with godaddy if they pull stunts like that? When they loose their customers they learn.

Yes 100% will soon be transferring 2 separate go daddy accounts.  They will lose 4 .com domains.

But I'm needing to get temp solution for now as I've got several certificates expiring on the 6th and haven't had time to refresh my memory of certbot / ZeroSSL tools to manually get certs and import .p12 into opnsense + separate Nginx proxy manager.

I googled around briefly yesterday to find if possible syntax with acme.sh to manually do dns01 validation but not seeing anything where the script will generate txt for you to manually create and then proceed to check for txt record.  Script fails and stops the moment it cannot create txt.

I remember I've done the manual DNS validation before but it was years ago.

I've had this same install since ~2019 and its upgraded all these years with an occasional upgrade issue that gets resolved.

Storage is good.  Opnsense is a VMware VM backed by 2 host VSAN.  Its on a mirrored mirror across 2 hosts.  So the VM disk is on 4 separate HDDs all with checksum validations.  Opnsense can run on either host.

I restored the entire VM and it all repeated same.  Unbound won't start with errors in OP.
IMO Its software / dependency issue surrounding python and/or unbound.

IMO their has been issue with how python gets upgraded + cleaned up after in the past.  I had issue in past with Python 37 package remaining on system when python 39 was the current.  Required manually removing some packages.  I'd have to refresh memory for specifics.

Yes I do have a customized blank Opnsense template that I could deploy and restore Opnsense, plug ins, Zenarmor.  But on several occasions with upgrade issues I find others with issue and community helps resolve together.  Or wait for next version and then that upgrade works fine.

I consider the full reinstall the nuclear bomb approach.  I'm sure python / unbound issue can get fixed.

Encountering this exit code 1 error with Godaddy.  So first tried manually adding sub domain txt record suggested here.  No go.

Then discovered Godaddy recently killed API https://www.reddit.com/r/PFSENSE/comments/1cwuwdo/psa_godaddy_has_changed_its_api_access_affects/

Not Ok.  Deal breaker.  Demanding Godaddy restore API.  See where that goes...
Did not receive any notification from Godaddy about this and API keys still in place...

As temp workaround I am attempted to manually create txt record but it appears the script just aborts upon failure of adding txt record and then won't proceed to validate if txt record exists.

I was hoping by setting DNS delay 0 or 600 I could reference the acme log for the txt data value it wanted to create / validate and create the txt record manually and the script would proceed.

Seems it must be done via custom CLI run of /usr/local/sbin/acme.sh script?

On 24.1.6 attempting upgrade to 24.1.8

After upgrade Unbound won't start.

2024-05-31T11:27:54-07:00   Critical   unbound   [8463:0] fatal error: failed to setup modules   
2024-05-31T11:27:54-07:00   Error   unbound   [8463:0] error: module init for module python failed   
2024-05-31T11:27:54-07:00   Error   unbound   [8463:0] error: python exception in Py_InitializeFromConfig: init_fs_encoding: failed to get the Python codec of the filesystem encoding


Additionally getting NGINX errors on certificates and extremely delayed boot.  I believe its just because of Unbound issue and lack of DNS

failed to setup nginx
performing sanity check on nginx configuration
nginx ssl_stapling ignored, host not found in ocsp responder r3.o.lencr.org

The other odd behavior with 24.1.8 is the extracting hung on extracting net-snmp for over 10 minutes.  During which the webui was completely unresponsive on other tabs.
It finally did extract and finished update.

[142/161] Upgrading net-snmp from 5.9.4,1 to 5.9.4_2,1...
===> Creating groups
Creating group 'snmpd' with gid '344'
===> Creating users
Creating user 'snmpd' with uid '344'
[142/161] Extracting net-snmp-5.9.4_2,1: ........

---

Any suggestions what is going on with unbound?  I've attempted upgrade twice, same issue.
I've since restored backup of 24.1.6.
Thanks

Same.  Deleted os-dyndns and webgui functional again.  Thanks for sharing.

Fair enough.  :) - I appreciate your help and attention.

Ok, thanks for checking on that.
So - with the tunables enabling VTI filtering and child always being 0.0.0.0 - is this behavior by design (working correctly) or a bug?

Cause essentially the filtering isn't working how I envisioned.
I would still need to control traffic via VTI outbound rules as all LAN traffic behind opnsense would be processed by ipsec kernel.

See I thought by enabling VTI filtering tunables I wouldn't need those VTI outbound rules because only specific phase 2 defined subnets would get forwarded (filtered) to ipsec kernel.

I'm just trying a proof of concept of filtering multiple networks or host to host like a policy VPN would work

So the lab just simulating that with the two phase 2s.  Only having a single phase 2 up.  The ping should not flow for the other phase 2 tunnel networks .99 or .20 network.

I did the same test attempting to limit hosts on single phase 2 tunnel
Local. 10.99.99.254/32
Remote 10.20.20.25/32
So VPN tunnel between Firewall A + Firewall B LAN INT

I then ping from PC off Firewall A LAN - IP 10.99.99.100 to Firewall B LAN INT IP 10.20.20.25.  It still works.  But Why?

So it appears that even though I have defined subnets in phase 2 it's still using 0.0.0.0.

Edit
Maybe I'm not understanding VTI filtering correctly.
Not sure what command you used to get that routing info but in your example it has:
IPsec Tunnel Child: Local 0.0.0.0/0 Remote 0.0.0.0/0

The way I'm understanding VTI filtering that child should be the defined subnets in the phase 2 and thus filtering on them.
Is that CLI command?  I'll run it on mine and see if it also has tunnel child 0.0.0.0/0

I'll probably just stick with migrating legacy VPN over to new policy VPNs.
But here's a better layout of config and screenshots

Firewall A
WAN:  10.100.100.200/24
LAN:  10.99.99.254/24
LO:  172.16.1.1/32
IPSEC1:  192.168.200.1/30

Firewall B
WAN:  10.100.100.201/24
LAN:  10.20.20.25/24
LO:  172.16.2.1/32
IPSEC1:  192.168.200.2/30








So you can see status only the one loopback phase 2 is enabled.
A PC off Firewall A LAN with IP 10.99.99.100 pings Firewall B LAN INT IP 10.20.20.25
Its works despite the only phase 2 being enabled on both firewalls is for the loopback tunnel.
And with the traffic now flowing inbound via IPSEC1 VTI rather than IPSEC I believe the tunables for VTI filter are enabled.

Firewall B - live view

I considered that last night and made that change.  And just confirmed again its set.  And confirmed no typos in tunables.

Both firewalls IPSEC1 VTI (REQID=1)
Both firewalls have each phase 2 defined as REQID = 1

Still doesn't appear VTI filtering is working.

Since the phase 2 - B consists of loopback INT IP on both firewalls I did a swap.
Disabled Phase 2 -A
Enabled Phase 2 - B

Same.  Traffic still flows for 10.99.99.0/24 + 10.20.20.0/24 when it shouldn't.

If I disable both 2 phases then all VPN traffic stops. (as it should)

Code Select Expand


net.enc.in.ipsec_filter_mask = 0
net.enc.out.ipsec_filter_mask = 0
net.inet.ipsec.filtertunnel = 1
net.inet6.ipsec6.filtertunnel = 1

= Enable Packet Filtering and NAT on if_ipsec, if_gre, if_vxlan and if_gif
= Disable Packet Filtering and NAT on if_enc0 (Shown as IPsec in the GUI)
This is the mode that makes VTI IPsec (and more) filtered, but makes policy based IPsec unfiltered.

Yes this is exactly what I have set on both firewalls yet having behavior mentioned on previous post where IPSEC VTI filtering does not appear to be working.

After enabling these tunables the difference I see that inbound traffic on a destination firewall comes in via IPSEC# VTI INT rather than the traditional IPSEC INT.

But if IPSEC filtering was working then by disabling my phase 2 - b, traffic should not flow for 172.16.1.1 or 172.16.2.1 yet it still does.

Thanks for help and links.
Agreed - not that easy to understand.  I need to re-read more understand the SNAT.  I do not believe NAT has been an issue in my test lab.

I really like Opnsense and am grateful for it.  But the devs should really consider a VPN wizard tool that takes one through a linear step 1,2,3.  Its kinda wild a VPN config covers so many different sections of config.  I count 8 different pages / sections.

Reading through your posts its my understand that by enabling these tunables the phase 2 local / remote address filtering should be working.

In my lab which very simliar of yours in these post I have two phase 2s.

Phase 2 - A
Local:  10.99.99.0/24
Remote:  100.20.20.0/24

Phase 2 - B
Local:  172.16.1.1/32
Remote:  172.16.2.1/32

What I don't understand is why when I disable Phase 2 - B.  VPN Traffic still flows for 172.16.1.1 or 172.16.2.1
Its seems that despite Phase 2 A having defined subnets its still using 0.0.0.0/0 and all traffic is being processed by ipsec kernel.

I thought that enabling the ipsec filtering tunables that I could achieve ipsec filtering just like policy vpns and define the subnets within the phase2?