IPv6 connectivity error after update to OPNsense 25.7.10-amd64

[ischilling]

Before I updated to OPNsense 25.7.10 (amd64) everything worked perfect, right after the update and a reboot, the IPv6 problem on the WAN interface appeared on my system as well.. In short words, I get the fixed IPv4 but neither a fixed IPv6 nor my fixed IPv6 /56 network.

I've a fixed IPv6 /56 network and the following settings worked very well before the update, please find my settings in the attached screenshot.

Currently it looks as if the dhcp6c.conf which to my understanding is needed for dhcp6c service isn't existing:

auser@theFirewall:~ # ls -l /usr/local/etc/dhcp6c.conf
ls: /usr/local/etc/dhcp6c.conf: No such file or directory
auser@theFirewall:~ # service dhcp6c onestart
/usr/local/etc/rc.d/dhcp6c: WARNING: /usr/local/etc/dhcp6c.conf is not readable.
/usr/local/etc/rc.d/dhcp6c: WARNING: failed precmd routine for dhcp6c
auser@theFirewall:~ # ps aux | grep dhcp6c
root      824  0.0  0.0  13744    2404  0  S+  00:54    0:00.00 grep dhcp6c
auser@theFirewall:~ # opnsense-version
OPNsense 25.7.10 (amd64)
auser@theFirewall:~ # ls -l /usr/local/opnsense/service/conf/actions.d | grep dhcp
-rw-r--r--  1 root wheel 1052 Dec 18 14:13 actions_dhcpd.conf
-rw-r--r--  1 root wheel 1090 Dec 18 14:13 actions_dhcpd6.conf

As said - everything worked perfect before the update.

I now installed OPNsense 25.7-amd64 - with NO patch.

Everything works smooth - which from my point of view indicates that something is wrong with the current OPNsense 25.7.10 (amd64) update.

Interestingly - when using a configuration backup I made with OPNsense 25.7.10 (amd64) in OPNsense 25.7-amd64 - the IPv6 issue reappears....

Note:

• igb0 is the WAN interface on my system
• the following is a fresh install - no further settings, except for correct settings for IPv6 on the WAN interface and track interface (0) on the LAN interface
• no Packages are installed in addition

ifconfig igb0 results in on OPNsense 25.7-amd64 (before the update)

Code Select Expand

igb0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: WAN (wan)
        options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
        ether 01:23:45:67:89:00
        inet 000.000.000.000 netmask 0xfffffffc broadcast 000.000.000.000
        inet6 fe80::a236:9fff:fea0:7d54%igb0 prefixlen 64 scopeid 0x3
        inet6 2a02:8109:8000:6a::144b prefixlen 128 pltime 86400 vltime 86400
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>

ibg0 is configured (right after the update) to OPNsense 25.7.10 (amd64)

Code Select Expand

igb0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: WAN (wan)
        options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
        ether 01:23:45:67:89:00
        inet 000.000.000.000 netmask 0xfffffffc broadcast 000.000.000.000
        inet6 fe80::0000:0000:0000:0000%igb0 prefixlen 64 scopeid 0x3
        inet6 2a02:0000:0000:00::144b prefixlen 128 pltime 86400 vltime 86400
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>

And without further configuration everything is fine and works.

However, as soon as I restore the full configuration from a backup before the update, a well working configuration though, the following happens:

Code Select Expand

igb0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: ZONE0_0_WAN_KD (wan)
        options=48520b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,HWSTATS,MEXTPG>
        ether 01:23:45:67:89:00
        inet 000.000.000.000 netmask 0xfffffffc broadcast 90.187.76.171
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,[b]IFDISABLED[/b],AUTO_LINKLOCAL>

As yo ucan see the IPv6 part of WAN is disabled - with a before well working configuration setting....

Checking the packages (which have been used with this configuration) I had to resolve the missing ones - which I did and I rebooted the system, just in case.

Code Select Expand

igb0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: ZONE0_0_WAN_KD (wan)
        options=48520b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,HWSTATS,MEXTPG>
        ether 01:23:45:67:89:00
        inet 000.000.000.000 netmask 0xfffffffc broadcast 000.000.000.000
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,[b]IFDISABLED[/b],AUTO_LINKLOCAL>

So it seems, one of the installed packages on my system is in combination with OPNsense 25.7.10-amd64, the problem.

I'll update this here as soon as I found out which...

... and we've two candidates: os-crowdsec and os-maltrail or both.

I am not sure which combination is the problem, however, if I install the one or the other and reboot (without doing any configuration) I come back to the problem as described here. So for my situation, I am happy to not use the one or the other and have a working OPNsense system running :)

Last and least (for now) I want to state: I had both installed and everything worked before the update to OPNsense 25.7.10-amd64.

To find out the plugins, I stripped all PlugIns from the configuration file I had and used it as new configuration for a blank, fully updated, OPNsense 25.7.10-amd64. Then I added step by step all PlugIns I was sure they won't hurt and ended up with these two.

Since I've no time for further testing I've the following findings:

• the ISP got the DHCPv6 requests and delivered the information back to OPNsense - this could be seen in the logfiles and via tcpdump etc...
• it seems dhcp6c never got the result
• Rules for the ports (even unsecure ones) on the WAN interface don't fix this as well
• I am definitely not deep enough into firewalling with OPNsense or FreeBSD to find out the reason for my problem nor to fix it

So I hope someone here hat ideas - and a solution.

[dmurphy]

I have a very similar issue.

Restored my configuration from scratch, and now I cannot get dhcp6c to start automatically in any way.

If I run it by hand from the CLI, I have no problem pulling an IPv6 address.

Code Select Expand

/usr/local/sbin/dhcp6c -D -n -c /var/etc/dhcp6c.conf
/usr/local/sbin/configctl interface reconfigure wan

works fine, brings up the IPv6 addresses, delegates to my various VLANs and I'm off to the races.

Excited to see what changes are coming in 26.1 - for now I'll roll with the manual method rather than try to debug further ... no sense with major dhcp6c changes on tap.

[franco]

I don't think there are structural issues that would warrant this behaviour. Very rarely I see that dhcp6c is not starting, but that's because e.g. the port it's being run over is not plugged in for example.

There are logs and context missing here. 26.1 would not change what is described here. We're relying on the same mechanism to start the dhcp6c service which works IMO.


Cheers,
Franco

[dmurphy]

I don't think there are structural issues that would warrant this behaviour. Very rarely I see that dhcp6c is not starting, but that's because e.g. the port it's being run over is not plugged in for example.

There are logs and context missing here. 26.1 would not change what is described here. We're relying on the same mechanism to start the dhcp6c service which works IMO.


Cheers,
Franco

Then I'll do some more digging and see if I can identify why dhcp6c isn't starting.  My WAN connection (Verizon Fios) is dual-stacked and comes up fine on IPv4; has no issue grabbing and holding its IP.

As usual I'm sure it's a misconfiguration on my part somewhere; just not entirely sure what I'd misconfigure as the DHCPv6 configuration is very basic on the WAN interface:

Generic Configuration
  - IPv6 Configuration Type: DHCPv6
DHCPv6 client configuration
  - Use VLAN priority: Disabled
  - Configuration mode: Basic
  - Prefix delegation size: 56
  - Request Prefix Only: Yes
  - Send prefix hint: Yes

And, that's about it.

I just rebooted the firewall and it still isn't firing - starting dhcp6c and then a "configctl interface newip wan" brought up the V6 connectivity.

[franco]

What I don't understand is that there are actually two ways dhcp6c is started: once immediately and once per rtsold. 25.7.11 decouples the starts of both so they don't race to the finish line together, but OTOH that was never really the case and only the side effect that the second one would release the IP of the first one and try again (and sometimes fail because the ISP says no to the second request).

You can just start via "/var/etc/rtsold_wan_script.sh igb1" normally which is how the system does it.  "wan" and "igb1" may be different for you.

The other thing is to check /var/etc/dhcp6c.conf if the configuration for "igb1" is actually there or not. In the worst case it's not there when it decides to start which means it may exit as soon as it was started and then it looks like it wasn't started. If not some code path decided that it's simply not ready (or misconfigured).


Cheers,
Franco

[franco]

FWIW, here is a similar report.

https://github.com/opnsense/core/issues/9153

If we can find the bug where ever it is that would be great.


Cheers,
Franco

[dmurphy]

What I don't understand is that there are actually two ways dhcp6c is started: once immediately and once per rtsold. 25.7.11 decouples the starts of both so they don't race to the finish line together, but OTOH that was never really the case and only the side effect that the second one would release the IP of the first one and try again (and sometimes fail because the ISP says no to the second request).

You can just start via "/var/etc/rtsold_wan_script.sh igb1" normally which is how the system does it.  "wan" and "igb1" may be different for you.

The other thing is to check /var/etc/dhcp6c.conf if the configuration for "igb1" is actually there or not. In the worst case it's not there when it decides to start which means it may exit as soon as it was started and then it looks like it wasn't started. If not some code path decided that it's simply not ready (or misconfigured).


Cheers,
Franco

So - I don't seem to have that script (at least, not with an interface assignment name in it -- see the two scripts below):

Code Select Expand

[root@dmurphy-gw /var/etc]# ls -l *sh
-rwxr-xr-x  1 root wheel 1355 Jan 19 23:35 dhcp6c_wan_script.sh
-rwxr-xr-x  1 root wheel 1041 Jan 19 23:35 rtsold_script.sh
[root@dmurphy-gw /var/etc]#

Contents of /var/etc/dhcp6c.conf here - all of which look at least nominally correct to me.

Code Select Expand

root@dmurphy-gw:~ # cat /var/etc/dhcp6c.conf
interface igc0 {
  send ia-pd 0; # request prefix delegation
  request domain-name-servers;
  request domain-name;
  script "/var/etc/dhcp6c_wan_script.sh"; # we'd like some nameservers please
};
id-assoc pd 0 {
  prefix ::/56 infinity;
  prefix-interface lagg0_vlan77 {
    sla-id 2;
    sla-len 8;
  };
  prefix-interface lagg0 {
    sla-id 1;
    sla-len 8;
  };
  prefix-interface lagg0_vlan91 {
    sla-id 3;
    sla-len 8;
  };
};

Now, this may be where things are getting interesting.

I've migrated 100% over to dnsmasq - not running ISC or KEA, and (perhaps crucially) - not radvd.

That said, for whatever reason, "Router Advertisements" was unchecked in my dnsmasq configuration, too.

That was very much not intentional - I could've sworn that was enabled.  Not sure if it matters one way or another, but I turned on the RA setting in dnsmasq, and then did want to verify that /var/etc/radvd.conf exists - it does, and it also appears (nominally) correct:

Code Select Expand

root@dmurphy-gw:~ # cat /var/etc/radvd.conf
# Automatically generated, do not edit
# Generated RADVD config for dhcp6 assignment from wan on lan
interface igc0 {
        AdvSendAdvert on;
        AdvLinkMTU 1500;
        AdvManagedFlag on;
        AdvOtherConfigFlag on;
        DNSSL localdomain { };
};

So after a fresh reboot, I see that rtsold is running, looks like it's hooked the correct scripts, but dhcp6c never starts.

Code Select Expand

root@dmurphy-gw:~ # ps auxwww|grep -i rtsol
root    78062   0.0  0.0   13924   2552  -  SCs  23:51   0:00.13 /usr/sbin/rtsold -aiu -p /var/run/rtsold.pid -A /var/etc/rtsold_script.sh -R /usr/local/opnsense/scripts/interfaces/rtsold_resolvconf.sh -D
root    79643   0.0  0.0   13924   2504  -  Is   23:51   0:00.03 rtsold: rtsold.llflags (rtsold)
root    80526   0.0  0.0   13924   2504  -  Is   23:51   0:00.00 rtsold: rtsold.script (rtsold)
root    81395   0.0  0.0   13924   2496  -  Is   23:51   0:00.00 rtsold: rtsold.sendmsg (rtsold)
root    81902   0.0  0.0   13924   2616  -  Ss   23:51   0:00.01 rtsold: system.syslog (rtsold)
root     7004   0.0  0.0   13744   2416 u2  S+   23:53   0:00.00 grep -i rtsol
root@dmurphy-gw:~ #
root@dmurphy-gw:~ #
root@dmurphy-gw:~ #
root@dmurphy-gw:~ #
root@dmurphy-gw:~ # ps auxwww|grep -i dhcp
root     7612   0.0  0.2   29944  18272  -  Ss   23:51   0:00.57 /usr/local/bin/python3 /usr/local/opnsense/scripts/dhcp/unbound_watcher.py --domain home (python3.11)
root    47467   0.0  0.0   14504   2656  -  Is   23:52   0:00.01 /usr/local/bin/dpinger -f -S -r 0 -i WAN_DHCP -B 98.111.111.111 -p /var/run/dpinger_WAN_DHCP.pid -u /var/run/dpinger_WAN_DHCP.sock -s 1s -l 4s -t 60s -d 1 1.1.1.1
_dhcp   76103   0.0  0.0   14080   2816  -  SCs  23:51   0:00.00 dhclient: igc0 (dhclient)
root    12022   0.0  0.0   13744   2416 u2  S+   23:53   0:00.00 grep -i dhcp

I still have to run it by hand.  Trying to run "/var/etc/rtsold_script.sh igc0" just results in "Rejecting own configuration" as it finds igc0 in /var/etc/radvd.conf (see logic in the rtsold_script.sh)

So without deeper knowledge, I guess the question is --- should /var/etc/radvd.conf contain an igc0 stanza for WAN? And is it correct?  Or should I remove it altogether (what generates it if radvd is disabled?)

[dmurphy]

OK, so I went ahead and tried removing /var/etc/radvd.conf altogether.

Guess what?  Works perfectly now.  dhcp6c starts right up on boot, prefixes are allocated to the various VLANs correctly, all looks perfect.

I think, perhaps, I see how we ended up with a rogue radvd.conf.

Here's what I did and how I think we got there:

Step 1 - Wipe DEC2752
Step 2 - Fresh install 25.7
Step 3 - Restore A10-gen3-config.xml to get proper tunables etc.
    *** This has igc0 assigned as the LAN interface. ***
Step 4 - Restore my DEC2752 backup on top of the A10-gen3-config.xml
   --> I have radvd disabled, but I believe the radvd.conf was ALREADY generated by the A10-gen3-config.xml, and wasn't "cleaned up" when igc0 was deassigned.
Step 5 - Orphaned radvd.conf was in /var/etc, but since radvd is disabled, it was never overwritten with a 'clean' copy.

So I think that's how we got a /var/etc/radvd.conf that referenced igc0, which it shouldn't as that's now the WAN interface.

Odd scenario for sure.   Just wondering as more of us move from ISC/KEA+radvd to straight dnsmasq, if there's a way to 'scrub' the radvd config?

[franco]

Ok, right, radvd is for LAN connections. Running it on a WAN has the risk of DHCPv6 client picking up its own configuration and that's why the sanity check was put in place.

https://github.com/opnsense/core/commit/572ae8a66575

Although that was done for SLAAC tracking which no longer exists the same is true for DHCPv6.

We can try to make sure that radvd.conf is cleared and the log message is easier to find from the GUI.


Cheers,
Franco

[franco]

https://github.com/opnsense/core/commit/5da971f2c67 should help here.  It currently does not apply to 25.7.x as a lot of changes are inbound for 26.1 already.


Cheers,
Franco

[TDroenner]

Is there any timeframe for the release of this fix?

[nero355]

Is there any timeframe for the release of this fix?

The release of OPNsense 26.1 is planned for the end of January according to a message in another thread and according to @franco's reply the fix should be in that release so I guess you will have to wait another 8 to 11 days :)

[TDroenner]

Thanks, nero355. Not a problem at all. I was just curious.

[franco]

The patch doesn't really fix anything, just do "rm /var/etc/radvd.conf" if you think you have the problem but I doubt it given the data you have provided.


Cheers,
Franco