Messages

For the time being I've fixed the user experience by setting MSS through a policy for traffic to/from IPsec. However, I'd like to ensure I get the issue fixed properly in case a client ignores MSS or uses UDP.

Note the RTT in the following two pings:

Code Select Expand

user@server:~$ ping -M do -s 1400 192.168.104.1
PING 192.168.104.1 (192.168.104.1) 1400(1428) bytes of data.
1408 bytes from 192.168.104.1: icmp_seq=2 ttl=62 time=735 ms
1408 bytes from 192.168.104.1: icmp_seq=4 ttl=62 time=733 ms
^C
--- 192.168.104.1 ping statistics ---
4 packets transmitted, 2 received, 50% packet loss, time 3077ms
rtt min/avg/max/mdev = 732.831/734.164/735.498/1.333 ms

user@server:~$ ping -M do -s 1398 192.168.104.1
PING 192.168.104.1 (192.168.104.1) 1398(1426) bytes of data.
1406 bytes from 192.168.104.1: icmp_seq=1 ttl=62 time=24.7 ms
1406 bytes from 192.168.104.1: icmp_seq=2 ttl=62 time=25.8 ms
^C
--- 192.168.104.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 24.732/25.269/25.807/0.537 ms

I'd rather the 1400-byte ping gets dropped than see this delay. The problem is that TCP windowing won't adjust down with a delay of 700 msec, but it will have a big impact on the speed of transfers. How UDP handles this will be down to the application, but it can't be helpful there either.

What could be causing this jump in latency from 25 to 735 msec?

Setting the LAN interface MTU to 1426 would certainly be one option, but one I'd like to avoid as this would affect traffic between LAN interfaces and unencrypted traffic to/from the WAN.

Sadly OPNsense doesn't expose a way to set the MTU on a simple (non-VTI) IPsec tunnel. Strongswan does have the option of `charon.plugins.kernel-netlink.mtu` but I've not found a way to test this without breaking IPsec.

So, as it turns out. DNS lookups work fine until the VPN comes up. How does a VPN affect the reachability of unbound on a directly connected interface?!

Switching to dnsmasq makes no difference. So it's not unbound or dnsmasq at fault.

Which led me to check my phase 2 details on the VPN. Here I had used a supernet for the remote end and the local network was part of this network. From a routing perspective this is not a problem as more specific routes are preferred. However, when it comes to VPNs the routing tables apparently aren't checked and matching traffic is either dropped or forwarded. Bit of an oversight IMHO as now I'll need to define very specific IP address ranges for the child SAs.

I should add that the LAN interface is a tagged interface (opt1). And a direct IPv6 request against the self-assigned IPv6 address of this interface doesn't work either.

But, alas, switching to an untagged interface makes no difference.

Under Services -> Unbound DNS -> General, is "Network Interfaces" set to "All"?

Yes, it is, so it should respond to both IPv4 and IPv6.

Setting outgoing Network interfaces, I only have the LAN and WAN options, not the VPN. Changing this to WAN doesn't fix anything. From the firewall itself:

Code Select Expand

user@fw01:~ $ host ilse.nl ::1
Using domain server:
Name: ::1
Address: ::1#53
Aliases:

ilse.nl has address 76.76.21.164
ilse.nl has address 76.76.21.61
ilse.nl mail is handled by 1 aspmx.l.google.com.
ilse.nl mail is handled by 10 alt3.aspmx.l.google.com.
ilse.nl mail is handled by 5 alt1.aspmx.l.google.com.
ilse.nl mail is handled by 5 alt2.aspmx.l.google.com.
ilse.nl mail is handled by 10 alt4.aspmx.l.google.com.

user@fw01:~ $ host ilse.nl 127.0.0.1
Using domain server:
Name: 127.0.0.1
Address: 127.0.0.1#53
Aliases:

ilse.nl has address 76.76.21.61
ilse.nl has address 76.76.21.164
ilse.nl mail is handled by 5 alt2.aspmx.l.google.com.
ilse.nl mail is handled by 10 alt4.aspmx.l.google.com.
ilse.nl mail is handled by 1 aspmx.l.google.com.
ilse.nl mail is handled by 10 alt3.aspmx.l.google.com.
ilse.nl mail is handled by 5 alt1.aspmx.l.google.com.

user@fw01:~ $ host ilse.nl 192.168.1.1
;; connection timed out; no servers could be reached

So this proves that unbound can resolve public domains fine via IPv6 and unbound is the only thing listening on port 53:

Code Select Expand

user@fw01:~ $ sudo sockstat -l -46 | grep 53
unbound  unbound    27263 5   udp6   *:53                  *:*
unbound  unbound    27263 6   tcp6   *:53                  *:*
unbound  unbound    27263 7   udp4   *:53                  *:*
unbound  unbound    27263 8   tcp4   *:53                  *:*
unbound  unbound    27263 9   udp6   *:53                  *:*
unbound  unbound    27263 10  tcp6   *:53                  *:*
unbound  unbound    27263 11  udp4   *:53                  *:*
unbound  unbound    27263 12  tcp4   *:53                  *:*
unbound  unbound    27263 13  tcp4   127.0.0.1:953         *:*

The timeout yields no log entries in the unbound logs and there are no firewall rules blocking access:

Code Select Expand

userfw01:~ $ nc -zvu 192.168.1.1 53
Connection to 192.168.1.1 53 port [udp/domain] succeeded!

But no response on TCP, despite seeing the traffic permitted in the firewall logs.

New to OPNsense and struggling to work out a fully working unbound DNS config.

Remote firewall, connected via VPN. It only has a public IPv6 address as that's what's used for the VPN. And so it has IPv6 DNS servers configured and these work fine for DNS resolution. I can resolve from the CLI fine with this.

What doesn't work is getting unbound to respond to IPv4 requests on the LAN interface. Traffic is permitted, but unbound won't respond to IPv4 requests. IPv6 works fine.

So to recap:

Code Select Expand

host google.com                  <<< works
host google.com 127.0.01        <<< works
host google.com ::1              <<< works
host google.com 192.168.1.1      <<< doesn't work
host google.com 2a01:blah::blah  <<< (public IP on the WAN interface, also works)
WWW <IPv6> firewall <IPv4> LAN

I eventually will have an IPv4 address on the WAN, but right now I was hoping to be able to test if an IPv4 client can resolve a public domain.

Equally, I haven't got the forwarders working yet for the internal domain. I don't know if this is due to the same problem or something else. I can resolve via direct query via the VPN, so there's no firewall rules blocking this. How does one debug unbound on OPNsense? The logs in the UI are pretty sparse. The config is pretty default: No ACL (yet), no overrides, internal domain added to `Private Domains`.

I briefly dabbled with Dnsmasq where for the forwarders I could set a source IP address. It seems this isn't possible for unbound, though I don't know if this is needed. Maybe I need to configure the VTI mode so that unbound can find the DNS servers better via the VPN.

I'm connecting to another site of mine and was hoping to avoid running a DNS server on another host at this remote site. The VPN is to a Palo Alto firewall and so far things have been working fine without using route based (VTI). But it appears that unbound requires VTI as per this log entry:

Code Select Expand

Error    unbound    [35197:1] error: udp connect failed: Network is unreachable for 192.168.10.22 port 53 (len 16)


It appears this was user error. Hetzner don't like virtual MAC addresses on their network. The fix is this: https://community.hetzner.com/tutorials/install-and-configure-proxmox_ve

Hi all,

Deployed OPNsense v24.7.10_2 today on Proxmox 8.3.1 on a new server.

Right from the start I noticed that I would lose access to the GUI after a little time (about 20 minutes). Connectivity to Proxmox itself is rock solid and from Proxmox I can ping the firewall VM, both on the LAN and WAN addresses.

I've now switched to OVS instead of the Linux bridge as I saw issues with neighbor solicitation. And then noticed that both ipv4 and ipv6 connectivity is lost after a little time. Turning the firewall back on, on the nic (disabled system-wide for now), restored network connectivity. What the heck is going on here?

Management of Proxmox will ultimately be behind a dedicated firewall, so I'm not too fussed about the Proxmox firewall, but why do I need to enable the firewall on the nic of a VM to restore service?!

Thanks,

[EDIT1]
And then I realise I should really post about the Proxmox firewall interference on a Proxmox forum and not here... Having done that, I still see SSH and the WebGUI inaccessible after a little time. I can reach the TCP ports but not the service listening on them.

[EDIT2]
Getting a VPN up to the firewall I just noticed that the VPN remained up and I could login to the firewall via the VPN, which is highly preferred anyway. But this allowed me to inspect the firewall logs and showed the reason for the block: "a default deny rule" Though I'm a little confused as to how a valid session ends up falling through the rule that permits it and then gets denied.

Shortly after my VPN went down and I lost connectivity again, until I reboot the firewall and can once again connect.