Messages

Got the same widget "Waiting for data..." error today when updating from 25.1.3 to 25.1.4. Clear log had no effect. Not such big deal though, leave it for now...

[dseven]

After a while I see a Solicit message to all All_DHCP_Relay_Agents_and_Servers ff02::1:2 requesting an IA-NA with option-request DNS-server DNS-search-list, which is replied by a server with the same MAC-address as replies from a server in messages on the other router.

The answer is "status-code NoAddrsAvail" so it is not the router, but pointing towards the DHCPv6 server at the ISP that for some reason doesn't hand out any IPv6 address to my WAN-interface.

Thanks a lot dseven for the help!

It was actually quite simple to setup Packet Capture in OPNsense directly in Diagnostics under WAN Interfaces.

When I limit the capture to IPv6 on port 546 and UDP only, filtering out all except DHCPv6 communitation in one direction.

Then I can see that there is no message with IA_NA or IA_ADDR from the ISP's DHCPv6 server on the failing router and only IA_PD. There is two messages in row each 15 minutes. On the router with an IPv6 address on WAN, there is four messages each 15 minutes and both IA_NA, IA_ADDR and IA_PD. I'm not sure whether this is a two or four message conversation, but I believe it should be two for renew.

I think what I am seening is a renewing request and a reply, I can see that there is no request for IA_NA and IA_ADDR, so preliminary it points towards an error in the router.

I will swap to port 547 after a while and filter for outgoing messages from the DHCPv6 client instead, if that looks the same it will isolate the problem.

Okey, thank you!

Now I have something that I can delve into. I have not done this before, it will likely take some time to figure out, but packet capture and Wireshark sounds very useful, and a must to learn.

Looking at the IPv6 WAN Address on the other working router from the same ISP, it definately looks DHCPv6 provided, so I doubt SLAAC.

No it's not selected, "Request prefix only" is not ticked on any of the routers.

How can I do packet captures? Is it difficult to setup?

I have 3 OPNsense routers on different sites where 2 out of 3 get a WAN IPv6 address on igc0. All three are on dual stack with native IPv6 and all are having a working prefix delegation and internet access.

One router is missing WAN IPv6 address on igc0. The router with the missing WAN IPv6 share the same ISP as one of the routers that has a WAN IPv6 address on igc0, but they are in different WAN network/regions.

I can't find any differences in the interface settings for IPv6 on the OPNsense routers that should prevent igc0 on one of them from having a WAN IPv6. The one with missing GUA was previously on HE tunnelbroker, but is since some months native IPv6. We also recently swapped ISP, but the problem prevail with both ISP's.

I suppose the relay agents on internet transfer the Advertise and Confirm messages with the leases relatively untampered from the ISP's DHCPv6-server to the DHCPv6-client in the OPNsense router? So losing WAN IPv6 in transfer doesn't make sense, right?

I have been in contact with the ISP several times, and they can't find any differences between the DHCPv6 settings for the two sites where I am using them that should prevent a WAN IPv6 on one of the sites.

The problematic site has a manually configured link-local address on the gateway fe80::ff, whereas the two working routers have autoconfigured. All three have autoconfigured link-local addresses on igc0 above the GUA.

I don't know if the link-local address on the gateway could be important or if any residuals from the tunnelbroker settings, not shown in the OPNsense GUI could prevent a IPv6 address from attaching to igc0?

Any thoughts or ideas are appreciated!

I reverted all 3 routers back to 24.1.8. I started with just opnsense, which didn't help alone, but after reverting dhcp6c as well, the WAN IPv6 address remains. I used an SSH client and did all three remotely from one site. Both Telia and Obe Network is working now.

# opnsense-revert -r 24.1.8 opnsense
# opnsense-revert -r 24.1.8 dhcp6c

I have not tried using the commands below yet; I am fairly new with OPNsesnse, but I guess they can''t be used in the GUI, perhaps from SSH, or else you need to hook up screen and keyboard directly to the router and use them in a UNIX shell, right?

# opnsense-revert -r 24.1.8 dhcp6c
# opnsense-revert -r 24.1.8 opnsense

My impression is that none of the hotfixes shown below are for correcting a "WAN IPv6 address not renewing after initial dhcp request", so it won't help. I updated directly to 24.1.9 hotfix 4 from 24.1.8.

"A hotfix release was issued as 24.1.9_1:

o firewall: "natreflection" rule attribute missed in MVC/API migration

A hotfix release was issued as 24.1.9_3:

o firewall: typo in "destination" migration for one-to-one NAT
o firewall: one-to-one NAT default reflection setting was ignored

A hotfix release was issued as 24.1.9_4:

o system: proper HA sync for new one-to-one NAT section"

I have the same problem on routers on two different sites with two different ISP, Telia AB and Obe Networks.

Upgraded from 24.1.8 to 24.1.9_4 last saturday and both routers now lose the WAN IPv6 address a while after first request upon boot. Looking in the log and there seem to be periodic requests from the DHCP6 client to the DHCP6 server after about half lease time, but no IPv6 IP-address show up in the dashboard and the VPN-tunnels fails as expected without peers.

This looks almost epidemic now and maybe it requires a hotfix?

Or is a complete revert back to 24.1.8 best practice?

The error message have been there for a long time I discovered in the general firewall log and it just prevents that faulty rule from being loaded. There is several other rules that actually were loaded, so likely is the message not related to my problems, since it worked before, despite this single exception after every boot.

Anyone seen IPv6 WAN Address being dropped after a while?

It didn't happen before version 24.1.9!

VPN tunnels for IPv6 has been working flawlessly for many weeks in 24.1.8.

Problems started when going to 24.1.9_4. No changes made to any IPsec or firewall setting, but the update created a mismatch between address families in the firewall rule for ISAKMP (port 500) for IPv6, thus mixing IPv4 with IPv6 peers, see error message below. IPv6 WAN Address in dashboard is also dropped after a while.

"There were error(s) loading the rules: /tmp/rules.debug:131: no routing address with matching address family found. - The line in question reads [131]: pass in log on igc0 reply-to ( igc0 2.242.xxx.xxx ) proto udp from {2a07:3aa1:xxxx::xxxx} to {any} port {500} keep state label "00eff9b1ada77af37818877b66bca707" # IPsec: Site1_Site2_IPV6"

Anyone seen this too or any idea what could be the root cause?

It seems that this problem now is gone without doing anything to fix it!

Didn't understand why it appeared and what solved it. Could be something done by the Internet Service Provider or by the Communication Operator in the P- or the PE-routers out on the internet, but I guess we will never know. I asked Hurricane Electric, but they haven't done anything recently.

RPC traffic on the DNS-servers now work in all directions when packet size is large enough. MTU 1480 was applied to the Tunnel Broker interface quite early in the process, but didn't increase MTU until after some days. First it only increase payload to internet, but suddenly also through the tunnel after more days. Initially the setting have to reapplied after boot, but now it comes on every time. Strange indeed!

From this morning I can ping between Tunnel Broker and native IPv6 sites to up to 1295 Byte and RPC traffic between DC:s is now working! I don't understand, since I have not done anything, what could have changed?

The problem with MTU to be reapplied after boot persist, but the most serious problem seems gone, at least for now...

I reported the issues and is currently on community support https://github.com/opnsense/core/issues/7451. OPNsense doubt that this is a real world problem and ask me to reach out to the community.

So please any idea what I possibly could have done wrong?

I assume the problem can be split in two pieces that may not be related:


1.  Why don't the entered interface setting for MTU reload after boot. It is very repeatable, MTU increase to 1430 Byte in payload ping after a while around 8-10 minutes after being applied, but it always has to be applied again although still present in the GUI and backup XML before and after boot, so what incorrect settting could cause this? I attach the screens in the thread.
2.  The VPN IPsec ESP tunnel does not benefit from the MTU increase and will thus carry too small packets to allow for DNS server RPC traffic.

I tested a new Windows Server 2022 for a short while to see if it makes any difference, but it missing some hardware and was removed again. The replication and access problem in DNS mainly persisted although Nslookup worked liked described for the client computers.

I contacted Hurricane Electric support and they said these problems typically are related to MTU.

Ping through HE-tunnel over IPv6 with payload empasized the problem. Ping to Internet only worked until 1230 Byte, whereas Ipsec to other site only reach 1095 Byte. Native site to Internet hit 1450 byte and 1310 byte in Ipsec. The MTU for the HE-tunnel is too low for IPv6 to work, particularly for IPsec that seem to take 135-140 Byte. IPv6 requires 1280 as a minimum.

MTU default at Hurricane Electric is 1480 Byte. The GIF interface in OPNsense defaults to 1280 Byte according to other threads and is part of FreeBSD. Setting MTU to 1480 Byte on the TunnelBrokerHE interface increase the Payload ping from 1230 to 1430 Byte, but it had no effect on traffic through IPsec, which still fails over 1095 Byte. The packet size reverts to 1230 after boot although the MTU 1480 setting still exists on the interface!

I have seen other reporting the same thing and suppose this is kind of a bug in OPNsense?