Hi all
I have set up IPv6 and CARP and have a strange problem on the CARP backup instance. When I want to perform an update, the backup OpnSense tries to update the packages over IPv6 first, which times out, so I always get:
opnsense-update -V -k
...
(long time til timeout)
Fetching kernel-25.1.1-amd64.txz: .+ mkdir -p /var/cache/opnsense-update/81289
+ opnsense-fetch -T 30 -q -o /var/cache/opnsense-update/81289/kernel-25.1.1-amd64.txz.sig https://mirrors.dotsrc.org/opnsense/FreeBSD:14:amd64/25.1/sets/kernel-25.1.1-amd64.txz.sig
..............................[fetch: transfer timed out
fetch: /var/cache/opnsense-update/81289/kernel-25.1.1-amd64.txz.sig appears to be truncated: 0/1332 bytes]+ exit_msg ' failed, no signature found'
The 'truncated' message appears every time. I can manually download and install the files (sig and kernel), but it works only when done over IPv4.
The funny thing is, when I temporarily run pfctl -d on the (!) master OpnSense, everything works over IPv6 on the backup OpnSense. I have checked three times, there are no rules that would prohibit IPv6 on the backup OpnSense. Otherwise, everything works normally on the master and also on the hosts behind it (IPv4 and IPv6).
Have I forgotten any extra 'switch'? Does anyone know this problem?
Thx in advance for any tips!
Greetz
here the routes on both systems...
CARP Master:
root@cerber:~ # netstat -6 -r
Routing tables
Internet6:
Destination Gateway Flags Netif Expire
default fe80::52e6:36ff:fe UGS vlan0.0004
localhost link#6 UHS lo0
dns.google fe80::52e6:36ff:fe UGHS vlan0.0004
2a02:8106:39:e600: link#11 U vlan0.0004
2a02:8106:39:e600: link#6 UHS lo0
2a02:8106:39:e602: link#3 U vtnet0
cerber link#6 UHS lo0
2a02:8106:39:e604: link#11 U vlan0.0004
2a02:8106:39:e604: link#6 UHS lo0
2a02:8106:39:e605: link#12 U vlan0.0005
cerber link#6 UHS lo0
2a02:8106:39:e615: link#13 U vlan0.0015
2a02:8106:39:e615: link#6 UHS lo0
2a02:8106:39:e624: link#15 U vlan0.1024
2a02:8106:39:e624: link#6 UHS lo0
2a02:8106:39:e648: link#16 U vlan0.2048
2a02:8106:39:e648: link#6 UHS lo0
fe80::%vtnet0/64 link#3 U vtnet0
fe80::be24:11ff:fe link#6 UHS lo0
fe80::%lo0/64 link#6 U lo0
fe80::1%lo0 link#6 UHS lo0
fe80::%lagg0/64 link#10 U lagg0
fe80::a236:9fff:fe link#6 UHS lo0
fe80::%vlan0.0004/ link#11 U vlan0.0004
fe80::192:168:40:2 link#6 UHS lo0
fe80::a236:9fff:fe link#6 UHS lo0
fe80::%vlan0.0005/ link#12 U vlan0.0005
fe80::192:168:50:2 link#6 UHS lo0
fe80::a236:9fff:fe link#6 UHS lo0
fe80::%vlan0.0015/ link#13 U vlan0.0015
fe80::192:168:150: link#6 UHS lo0
fe80::a236:9fff:fe link#6 UHS lo0
fe80::%vlan0.1024/ link#15 U vlan0.1024
fe80::192:168:24:2 link#6 UHS lo0
fe80::a236:9fff:fe link#6 UHS lo0
fe80::%vlan0.2048/ link#16 U vlan0.2048
fe80::192:168:48:2 link#6 UHS lo0
fe80::a236:9fff:fe link#6 UHS lo0
CARP Backup:
root@fenrir:~ # netstat -6 -r
Routing tables
Internet6:
Destination Gateway Flags Netif Expire
default fe80::52e6:36ff:fe UGS vlan0.0004
localhost link#4 UHS lo0
dns.google fe80::52e6:36ff:fe UGHS vlan0.0004
2a02:8106:39:e600: link#9 U vlan0.0004
2a02:8106:39:e602: link#1 U igb0
fenrir link#4 UHS lo0
2a02:8106:39:e604: link#9 U vlan0.0004
2a02:8106:39:e604: link#4 UHS lo0
2a02:8106:39:e605: link#10 U vlan0.0005
fenrir link#4 UHS lo0
2a02:8106:39:e615: link#11 U vlan0.0015
2a02:8106:39:e615: link#4 UHS lo0
2a02:8106:39:e624: link#12 U vlan0.1024
2a02:8106:39:e624: link#4 UHS lo0
2a02:8106:39:e648: link#13 U vlan0.2048
2a02:8106:39:e648: link#4 UHS lo0
fe80::%igb0/64 link#1 U igb0
fe80::20d:b9ff:fe4 link#4 UHS lo0
fe80::%lo0/64 link#4 U lo0
fe80::1%lo0 link#4 UHS lo0
fe80::%lagg0/64 link#8 U lagg0
fe80::20d:b9ff:fe4 link#4 UHS lo0
fe80::%vlan0.0004/ link#9 U vlan0.0004
fe80::20d:b9ff:fe4 link#4 UHS lo0
fe80::%vlan0.0005/ link#10 U vlan0.0005
fe80::20d:b9ff:fe4 link#4 UHS lo0
fe80::%vlan0.0015/ link#11 U vlan0.0015
fe80::20d:b9ff:fe4 link#4 UHS lo0
fe80::%vlan0.1024/ link#12 U vlan0.1024
fe80::20d:b9ff:fe4 link#4 UHS lo0
fe80::%vlan0.2048/ link#13 U vlan0.2048
fe80::20d:b9ff:fe4 link#4 UHS lo0
When I start the update on the CARP Backup OpnSense, I see this in the log of the CARP Master OpnSense:
>134>1 2025-02-20T15:23:34+01:00 cerber.mgmt.chao5.net filterlog 82363 - [meta sequenceId="24691"] 30,,,02f4bab031b57d1e30553ce08e0ec131,vlan0.0004,match,block,in,6,0x00,0x8a540,120,tcp,6,40,2001:4860:4860::8888,2a02:8106:39:e604::4,853,63203,0,SA,2018633911,114422878,65535,,mss;sackOK;TS;nop;wscale
The rule triggers even though I have an IPv6 rule on the DMZ interface (vlan0.0004) that allows everything...
grep 02f4bab031b57d1e30553ce08e0ec131 /tmp/rules.debug
block in log inet from {any} to {any} label "02f4bab031b57d1e30553ce08e0ec131" # Default deny / state violation rule
block in log inet6 from {any} to {any} label "02f4bab031b57d1e30553ce08e0ec131" # Default deny / state violation rule
Why is the automatic rule being triggered? Shouldn't my explicit IPv6 allow any to any rule take precedence? Moreover, why is it triggering on the CARP Master OpnSense? When I ping out to the internet (ICMPv6), the communication does not go through the Master CARP OpnSense but directly to the router.