Routing issues on failover WAN with 24.7.6

[hdholm]

Being in the path of a recent hurricane I lost my primary Internet connection for a while.  To mitigate this I (long story short) managed to tether an old iPhone via USB to my OPNsense box as an alternate path out.  After some setup pain, that worked pretty well. When my primary (cable) provider returned, I configured the gateways to be priority 1 for IPv4 on WAN (igc0), priority 250 for WAN IPv6, priority 254 for IPv4 BackupWAN(ue0) and disabled IPv6 on the BackupWAN interface since it doesn't support IPv6 delegation. My cablle provider (Xfinity) seems to not  provide a proper gateway address, so I've set the monitor address to 8.8.4.4.  That appropriately sets a route for 8.8.4.4 to the WAN interface. There is a "gateway" detected on icg4 because there are two routes configured to systems on the other side of that interface beyond another NAT. Things seem to mostly work failing back and forth appropriately when disconnected from either network. But whenever the OPNsense box reboots by the time I log in it  sees all the gateways as up and running but has the default route set to the BackupWAN (BackupWAN is, as you'd expect, both more expensive and much slower. So that's not ideal to say the least.)  Nothing in the logs seems to point to anything "interesting".  General log reproduced below:

Code: [Select]

2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (execute task : wireguard_sync())
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (execute task : webgui_configure_do(,wan,opt4,opt2,opt10,opt7,lan,opt5,opt3,opt14,opt1))
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (execute task : vxlan_configure_do())
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (execute task : unbound_configure_do(,wan,opt4,opt2,opt10,opt7,lan,opt5,opt3,opt14,opt1))
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (execute task : openssh_configure_do(,wan,opt4,opt2,opt10,opt7,lan,opt5,opt3,opt14,opt1))
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (execute task : opendns_configure_do())
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (execute task : ntpd_configure_do())
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (execute task : dnsmasq_configure_do())
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (execute task : dhcrelay_configure_if(,wan,opt4,opt2,opt10,opt7,lan,opt5,opt3,opt14,opt1,inet6))
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (,wan,opt4,opt2,opt10,opt7,lan,opt5,opt3,opt14,opt1,inet6)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,opt1)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,opt1)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,opt14)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,opt14)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,opt3)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,opt3)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,opt5)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,opt5)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,lan)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,lan)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,opt7)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,opt7)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,opt10)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,opt10)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,opt2)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,opt2)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,opt4)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,opt4)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,wan)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,wan)
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn_map (execute task : wireguard_configure_do())
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn_map (execute task : openvpn_configure_do(,wan,opt4,opt2,opt10,opt7,lan,opt5,opt3,opt14,opt1))
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn_map (execute task : ipsec_configure_do(,wan,opt4,opt2,opt10,opt7,lan,opt5,opt3,opt14,opt1))
2024-10-18T16:29:29 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn_map (,wan,opt4,opt2,opt10,opt7,lan,opt5,opt3,opt14,opt1,inet6)
2024-10-18T16:29:28 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure newwanip_map:rfc2136 (,wan)
2024-10-18T16:29:27 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure monitor (execute task : dpinger_configure_do(,[COMCAST_DHCP6]))
2024-10-18T16:29:27 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure monitor (,[COMCAST_DHCP6])
2024-10-18T16:29:27 Notice opnsense /usr/local/etc/rc.newwanipv6: ROUTING: ignoring down gateways: COMCAST_DHCP6
2024-10-18T16:29:27 Notice opnsense /usr/local/etc/rc.newwanipv6: ROUTING: entering configure using wan, opt4, opt2, opt10, opt7, lan, opt5, opt3, opt14, opt1
2024-10-18T16:29:27 Warning radvd exiting, 1 sigterm(s) received
2024-10-18T16:29:26 Error opnsense /usr/local/etc/rc.newwanipv6: The command '/bin/kill -'TERM' '26025''(pid:/var/dhcpd/var/run/dhcpdv6.pid) returned exit code '1', the output was 'kill: 26025: No such process'
2024-10-18T16:29:26 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure dhcp (execute task : dhcpd_dhcp_configure(,inet6))
2024-10-18T16:29:26 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure dhcp (,inet6)
2024-10-18T16:29:26 Notice opnsense /usr/local/etc/rc.newwanipv6: IP renewal starting (address: 2001:558:6021:2:6d08:c494:a872:291, interface: wan, device: igc0)
2024-10-18T16:29:23 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : unbound_configure_do())
2024-10-18T16:29:23 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : dnsmasq_configure_do())
2024-10-18T16:29:23 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns ()
2024-10-18T16:29:23 Warning radvd exiting, 1 sigterm(s) received
2024-10-18T16:29:23 Warning opnsense /usr/local/etc/rc.linkup: dhcpd_radvd_configure(auto) found no suitable IPv6 address on opt1(igc3)
2024-10-18T16:29:23 Warning opnsense /usr/local/etc/rc.linkup: dhcpd_radvd_configure(auto) found no suitable IPv6 address on opt14(bridge0)
2024-10-18T16:29:23 Warning opnsense /usr/local/etc/rc.linkup: dhcpd_radvd_configure(auto) found no suitable IPv6 address on opt7(igc5_vlan4)
2024-10-18T16:29:23 Warning opnsense /usr/local/etc/rc.linkup: dhcpd_radvd_configure(auto) found no suitable IPv6 address on opt4(igc5)
2024-10-18T16:29:23 Warning opnsense /usr/local/etc/rc.linkup: dhcpd_radvd_configure(manual) found no suitable IPv6 address on opt10(vlan0.6)
2024-10-18T16:29:23 Notice opnsense /usr/local/etc/rc.newwanip: plugins_configure newwanip_map (execute task : wireguard_sync())
2024-10-18T16:29:23 Notice opnsense /usr/local/etc/rc.newwanip: plugins_configure newwanip_map (execute task : webgui_configure_do(,wan))
2024-10-18T16:29:23 Notice opnsense /usr/local/etc/rc.newwanip: plugins_configure newwanip_map (execute task : vxlan_configure_do())
2024-10-18T16:29:23 Notice opnsense /usr/local/etc/rc.newwanip: plugins_configure newwanip_map (execute task : unbound_configure_do(,wan))
2024-10-18T16:29:23 Warning opnsense /usr/local/etc/rc.linkup: dhcpd_radvd_configure(manual) found no suitable IPv6 address on opt5(igc5_vlan2)
2024-10-18T16:29:23 Notice opnsense /usr/local/etc/rc.newwanip: plugins_configure newwanip_map (execute task : openssh_configure_do(,wan))
2024-10-18T16:29:23 Notice opnsense /usr/local/etc/rc.newwanip: plugins_configure newwanip_map (execute task : opendns_configure_do())
2024-10-18T16:29:23 Notice opnsense /usr/local/etc/rc.newwanip: plugins_configure newwanip_map (execute task : ntpd_configure_do())
2024-10-18T16:29:23 Notice opnsense /usr/local/etc/rc.newwanip: plugins_configure newwanip_map (execute task : dnsmasq_configure_do())
2024-10-18T16:29:23 Notice opnsense /usr/local/etc/rc.newwanip: plugins_configure newwanip_map (execute task : dhcrelay_configure_if(,wan,inet))
2024-10-18T16:29:23 Notice opnsense /usr/local/etc/rc.newwanip: plugins_configure newwanip_map (,wan,inet)
2024-10-18T16:29:23 Notice opnsense /usr/local/etc/rc.newwanip: plugins_configure newwanip (,wan)
2024-10-18T16:29:23 Notice opnsense /usr/local/etc/rc.newwanip: plugins_configure vpn (,wan)
2024-10-18T16:29:22 Warning opnsense /usr/local/etc/rc.linkup: dhcpd_radvd_configure(manual) found no suitable IPv6 address on opt2(igc4)
2024-10-18T16:29:22 Notice opnsense /usr/local/etc/rc.newwanip: plugins_configure vpn_map (execute task : wireguard_configure_do())
2024-10-18T16:29:22 Notice opnsense /usr/local/etc/rc.newwanip: plugins_configure vpn_map (execute task : openvpn_configure_do(,wan))
2024-10-18T16:29:22 Notice opnsense /usr/local/etc/rc.newwanip: plugins_configure vpn_map (execute task : ipsec_configure_do(,wan))
2024-10-18T16:29:22 Notice opnsense /usr/local/etc/rc.newwanip: plugins_configure vpn_map (,wan,inet)
2024-10-18T16:29:22 Notice dhcp6c dhcp6c_script: REQUEST on igc0 renewal
2024-10-18T16:29:22 Notice dhcp6c dhcp6c_script: REQUEST on igc0 executing
2024-10-18T16:29:22 Warning opnsense /usr/local/etc/rc.linkup: dhcpd_radvd_configure(manual) found no suitable IPv6 address on opt3(vlan0.5)
2024-10-18T16:29:22 Warning opnsense /usr/local/etc/rc.linkup: dhcpd_radvd_configure(manual) found no suitable IPv6 address on lan(vlan0.4093)
2024-10-18T16:29:22 Warning opnsense /usr/local/etc/rc.linkup: dhcpd_dhcp6_configure() found no suitable IPv6 address on opt10(vlan0.6)
2024-10-18T16:29:22 Warning opnsense /usr/local/etc/rc.linkup: dhcpd_dhcp6_configure() found no suitable IPv6 address on opt5(igc5_vlan2)
2024-10-18T16:29:22 Warning opnsense /usr/local/etc/rc.linkup: dhcpd_dhcp6_configure() found no suitable IPv6 address on opt2(igc4)
2024-10-18T16:29:22 Warning opnsense /usr/local/etc/rc.linkup: dhcpd_dhcp6_configure() found no suitable IPv6 address on opt3(vlan0.5)
2024-10-18T16:29:22 Warning opnsense /usr/local/etc/rc.linkup: dhcpd_dhcp6_configure() found no suitable IPv6 address on lan(vlan0.4093)
2024-10-18T16:29:21 Notice opnsense /usr/local/etc/rc.newwanip: plugins_configure monitor (execute task : dpinger_configure_do(,[COMCAST_DHCP]))
2024-10-18T16:29:21 Notice opnsense /usr/local/etc/rc.newwanip: plugins_configure monitor (,[COMCAST_DHCP])
2024-10-18T16:29:21 Notice opnsense /usr/local/etc/rc.newwanip: ROUTING: setting inet default route to 172.20.10.1
2024-10-18T16:29:21 Notice opnsense /usr/local/etc/rc.newwanip: ROUTING: configuring inet default gateway on opt8
2024-10-18T16:29:21 Notice opnsense /usr/local/etc/rc.newwanip: ROUTING: ignoring down gateways: COMCAST_DHCP, COMCAST_DHCP6
2024-10-18T16:29:21 Notice opnsense /usr/local/etc/rc.newwanip: ROUTING: entering configure using wan
2024-10-18T16:29:21 Notice opnsense /usr/local/etc/rc.newwanip: IP renewal starting (new: 73.20.153.26, old: 73.20.153.26, interface: wan, device: igc0, force: yes)
2024-10-18T16:29:21 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp (execute task : dhcpd_dhcp_configure())
2024-10-18T16:29:21 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp ()
2024-10-18T16:29:21 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (execute task : ipsec_configure_do(,wan))
2024-10-18T16:29:21 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (,wan)
2024-10-18T16:29:21 Warning opnsense /usr/local/etc/rc.linkup: The required COMCAST_DHCP6 IPv6 interface address could not be found, skipping.
2024-10-18T16:29:21 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure monitor (execute task : dpinger_configure_do(,[COMCAST_DHCP,COMCAST_DHCP6]))
2024-10-18T16:29:21 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure monitor (,[COMCAST_DHCP,COMCAST_DHCP6])
2024-10-18T16:29:21 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: keeping inet6 default route to fe80::dead:beef:2:1%igc0
2024-10-18T16:29:21 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: configuring inet6 default gateway on wan
2024-10-18T16:29:21 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: setting inet default route to 73.20.152.1
2024-10-18T16:29:21 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: configuring inet default gateway on wan
2024-10-18T16:29:21 Notice dhcp6c RTSOLD script - Sending SIGHUP to dhcp6c
2024-10-18T16:29:21 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: entering configure using wan
2024-10-18T16:29:21 Notice dhcp6c RTSOLD script - Sending SIGHUP to dhcp6c
2024-10-18T16:29:21 Notice dhclient dhclient-script: Creating resolv.conf
2024-10-18T16:29:21 Notice dhclient dhclient-script: New Routers (igc0): 73.20.152.1
2024-10-18T16:29:21 Notice dhclient dhclient-script: New Broadcast Address (igc0): 255.255.255.255
2024-10-18T16:29:21 Notice dhclient dhclient-script: New Subnet Mask (igc0): 255.255.254.0
2024-10-18T16:29:21 Notice dhclient dhclient-script: New IP Address (igc0): 73.20.153.26
2024-10-18T16:29:21 Notice dhclient dhclient-script: New Hostname (igc0): gatekeeper
2024-10-18T16:29:21 Notice dhclient dhclient-script: Reason REBOOT on igc0 executing
2024-10-18T16:29:19 Notice kernel <6>igc0: permanently promiscuous mode disabled
2024-10-18T16:29:19 Notice dhclient dhclient-script: Reason PREINIT on igc0 executing
2024-10-18T16:29:19 Notice opnsense /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for wan(igc0)
2024-10-18T16:29:19 Notice kernel <6>igc0: link state changed to UP
2024-10-18T16:29:16 Critical dhclient exiting.
2024-10-18T16:29:16 Error dhclient connection closed
2024-10-18T16:29:16 Error opnsense /usr/local/etc/rc.linkup: The command '/sbin/ifconfig 'igc0' inet6 '2001:558:6021:2:6d08:c494:a872:291' delete' returned exit code '1', the output was 'ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address'
2024-10-18T16:29:16 Error opnsense /usr/local/etc/rc.linkup: The command '/usr/sbin/daemon -f -p '/var/run/dhcpleases6.pid' '/usr/local/opnsense/scripts/dhcp/prefixes.sh'' returned exit code '3', the output was 'daemon: process already running, pid: 41363'
2024-10-18T16:29:15 Notice kernel 955.599153 [ 319] generic_netmap_register Emulated adapter for igc5_vlan2 activated
2024-10-18T16:29:15 Notice kernel 955.589193 [1167] generic_netmap_attach Emulated adapter for igc5_vlan2 created (prev was NULL)
2024-10-18T16:29:15 Notice kernel <6>igc5_vlan2: permanently promiscuous mode enabled
2024-10-18T16:29:15 Notice kernel 955.578263 [1072] generic_netmap_dtor Emulated netmap adapter for igc5_vlan2 destroyed
2024-10-18T16:29:15 Notice kernel 955.568297 [1167] generic_netmap_attach Emulated adapter for igc5_vlan2 created (prev was NULL)
2024-10-18T16:29:15 Notice kernel 955.558440 [1072] generic_netmap_dtor Emulated netmap adapter for igc5_vlan2 destroyed
2024-10-18T16:29:15 Notice kernel 955.548438 [1167] generic_netmap_attach Emulated adapter for igc5_vlan2 created (prev was NULL)
2024-10-18T16:29:15 Notice kernel 955.539086 [1072] generic_netmap_dtor Emulated netmap adapter for igc5_vlan2 destroyed
2024-10-18T16:29:15 Notice kernel 955.529168 [1167] generic_netmap_attach Emulated adapter for igc5_vlan2 created (prev was NULL)
2024-10-18T16:29:15 Notice kernel 955.513793 [ 319] generic_netmap_register Emulated adapter for igc5_vlan4 activated
2024-10-18T16:29:15 Notice kernel 955.503822 [1167] generic_netmap_attach Emulated adapter for igc5_vlan4 created (prev was NULL)
2024-10-18T16:29:15 Notice kernel 955.492866 [1072] generic_netmap_dtor Emulated netmap adapter for igc5_vlan4 destroyed
2024-10-18T16:29:15 Notice kernel 955.482153 [1167] generic_netmap_attach Emulated adapter for igc5_vlan4 created (prev was NULL)
2024-10-18T16:29:15 Notice kernel 955.472827 [1072] generic_netmap_dtor Emulated netmap adapter for igc5_vlan4 destroyed
2024-10-18T16:29:15 Notice kernel 955.462867 [1167] generic_netmap_attach Emulated adapter for igc5_vlan4 created (prev was NULL)
2024-10-18T16:29:15 Notice kernel 955.453563 [1072] generic_netmap_dtor Emulated netmap adapter for igc5_vlan4 destroyed
2024-10-18T16:29:15 Notice kernel 955.443634 [1167] generic_netmap_attach Emulated adapter for igc5_vlan4 created (prev was NULL)
2024-10-18T16:29:15 Notice kernel 955.213407 [ 850] iflib_netmap_config txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
2024-10-18T16:29:15 Notice kernel 955.204776 [ 850] iflib_netmap_config txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
2024-10-18T16:29:15 Notice kernel 955.196222 [ 850] iflib_netmap_config txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
2024-10-18T16:29:15 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp (execute task : dhcpd_dhcp_configure(,inet6,[opt4,opt2,opt10,opt7,lan,opt5,opt3,opt14,opt1]))
2024-10-18T16:29:15 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp (,inet6,[opt4,opt2,opt10,opt7,lan,opt5,opt3,opt14,opt1])
2024-10-18T16:29:15 Notice kernel <6>igc0: link state changed to DOWN
2024-10-18T16:29:15 Notice kernel 954.966635 [ 850] iflib_netmap_config txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
2024-10-18T16:29:15 Notice kernel <6>igc0: permanently promiscuous mode enabled
2024-10-18T16:29:15 Notice opnsense /usr/local/etc/rc.linkup: DEVD: Ethernet detached event for wan(igc0)
2024-10-18T16:29:14 Notice kernel 954.956666 [ 850] iflib_netmap_config txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
2024-10-18T16:29:14 Notice kernel 954.948057 [ 850] iflib_netmap_config txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
2024-10-18T16:29:14 Notice kernel 954.939534 [ 850] iflib_netmap_config txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
2024-10-18T16:29:14 Notice kernel 954.925014 [ 319] generic_netmap_register Emulated adapter for igc5_vlan3 activated
2024-10-18T16:29:14 Notice kernel 954.870986 [1167] generic_netmap_attach Emulated adapter for igc5_vlan3 created (prev was NULL)
2024-10-18T16:29:14 Notice kernel <6>igc5_vlan3: permanently promiscuous mode enabled
2024-10-18T16:29:14 Notice kernel 954.860226 [1072] generic_netmap_dtor Emulated netmap adapter for igc5_vlan3 destroyed
2024-10-18T16:29:14 Notice kernel 954.849239 [1167] generic_netmap_attach Emulated adapter for igc5_vlan3 created (prev was NULL)
2024-10-18T16:29:14 Notice kernel 954.839935 [1072] generic_netmap_dtor Emulated netmap adapter for igc5_vlan3 destroyed
2024-10-18T16:29:14 Notice kernel 954.829954 [1167] generic_netmap_attach Emulated adapter for igc5_vlan3 created (prev was NULL)
2024-10-18T16:29:14 Notice kernel 954.820624 [1072] generic_netmap_dtor Emulated netmap adapter for igc5_vlan3 destroyed
2024-10-18T16:29:14 Notice kernel 954.810649 [1167] generic_netmap_attach Emulated adapter for igc5_vlan3 created (prev was NULL)
2024-10-18T16:28:54 Error flowd_aggregate.py flowd aggregate died with message Traceback (most recent call last): File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 160, in run aggregate_flowd(self.config, do_vacuum) File "/usr/local/opnsense/scripts/netflow/flowd_aggregate.py", line 80, in aggregate_flowd stream_agg_object.add(copy.copy(flow_record)) File "/usr/local/opnsense/scripts/netflow/lib/aggregates/ports.py", line 69, in add super(FlowDstPortTotals, self).add(flow) File "/usr/local/opnsense/scripts/netflow/lib/aggregates/__init__.py", line 187, in add self._update_cur.execute(self._insert_stmt, flow) sqlite3.DatabaseError: database disk image is malformed
2024-10-18T16:28:53 Notice flowd_aggregate.py start watching flowd
2024-10-18T16:28:53 Notice opnsense /usr/local/etc/rc.routing_configure: plugins_configure monitor (execute task : dpinger_configure_do(1,[]))
2024-10-18T16:28:53 Notice opnsense /usr/local/etc/rc.routing_configure: plugins_configure monitor (1,[])
2024-10-18T16:28:53 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: keeping inet6 default route to fe80::dead:beef:2:1%igc0
2024-10-18T16:28:53 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: configuring inet6 default gateway on wan
2024-10-18T16:28:53 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: setting inet default route to 73.20.152.1
2024-10-18T16:28:53 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: configuring inet default gateway on wan
2024-10-18T16:28:53 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: entering configure using defaults
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (execute task : wireguard_sync())
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (execute task : webgui_configure_do(,wan,opt4,opt2,opt10,opt7,lan,opt5,opt3,opt14,opt1))
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (execute task : vxlan_configure_do())
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (execute task : unbound_configure_do(,wan,opt4,opt2,opt10,opt7,lan,opt5,opt3,opt14,opt1))
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (execute task : openssh_configure_do(,wan,opt4,opt2,opt10,opt7,lan,opt5,opt3,opt14,opt1))
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (execute task : opendns_configure_do())
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (execute task : ntpd_configure_do())
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (execute task : dnsmasq_configure_do())
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (execute task : dhcrelay_configure_if(,wan,opt4,opt2,opt10,opt7,lan,opt5,opt3,opt14,opt1,inet6))
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip_map (,wan,opt4,opt2,opt10,opt7,lan,opt5,opt3,opt14,opt1,inet6)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,opt1)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,opt1)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,opt14)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,opt14)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,opt3)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,opt3)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,opt5)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,opt5)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,lan)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,lan)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,opt7)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,opt7)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,opt10)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,opt10)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,opt2)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,opt2)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,opt4)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,opt4)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure newwanip (,wan)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn (,wan)
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn_map (execute task : wireguard_configure_do())
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn_map (execute task : openvpn_configure_do(,wan,opt4,opt2,opt10,opt7,lan,opt5,opt3,opt14,opt1))
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn_map (execute task : ipsec_configure_do(,wan,opt4,opt2,opt10,opt7,lan,opt5,opt3,opt14,opt1))
2024-10-18T16:28:44 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure vpn_map (,wan,opt4,opt2,opt10,opt7,lan,opt5,opt3,opt14,opt1,inet6)
2024-10-18T16:28:41 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure monitor (execute task : dpinger_configure_do(,[COMCAST_DHCP6]))
2024-10-18T16:28:41 Notice opnsense /usr/local/etc/rc.newwanipv6: plugins_configure monitor (,[COMCAST_DHCP6])
2024-10-18T16:28:41 Notice opnsense /usr/local/etc/rc.newwanipv6: ROUTING: keeping inet6 default route to fe80::dead:beef:2:1%igc0
2024-10-18T16:28:41 Notice opnsense /usr/local/etc/rc.newwanipv6: ROUTING: configuring inet6 default gateway on wan
2024-10-18T16:28:41 Notice opnsense /usr/local/etc/rc.newwanipv6: ROUTING: entering configure using wan, opt4, opt2, opt10, opt7, lan, opt5, opt3, opt14, opt1
2024-10-18T16:28:41 Warning radvd exiting, 1 sigterm(s) received
2024-10-18T16:28:41 Notice kernel <118> SSH: SHA256 [...] (RSA)
2024-10-18T16:28:41 Notice kernel <118> SSH: SHA256 [...] (ED25519)
2024-10-18T16:28:41 Notice kernel <118> SSH: SHA256 [...] (ECDSA)
2024-10-18T16:28:41 Notice kernel <118> [...]
2024-10-18T16:28:41 Notice kernel <118> HTTPS: sha256 [...]
2024-10-18T16:28:40 Notice kernel <118>
2024-10-18T16:28:40 Notice kernel <118> v6/t6: 2601:100:8c00:dca7:227c:14ff:fea2:d51f/64
2024-10-18T16:28:40 Notice kernel <118> eufyHomeBase (igc3) -> v4: 192.168.128.1/24
2024-10-18T16:28:40 Notice kernel <118> v6/t6: 2601:100:8c00:dca8:5a9c:fcff:fe10:8162/64
2024-10-18T16:28:40 Notice kernel <118> euffyBridge (bridge0) -> v4: 192.168.129.1/24
2024-10-18T16:28:40 Notice kernel <118> Wireguard (wg0) -> v4: 10.5.8.1/24
2024-10-18T16:28:40 Notice kernel <118> v6: 2600:1005:b0b8:e92a:14bd:61ff:fe91:d791/64
2024-10-18T16:28:40 Notice kernel <118> WANBackup (ue0) -> v4/DHCP4: 172.20.10.3/28
2024-10-18T16:28:40 Notice kernel <118> VoIP (igc6) -> v4: 10.60.0.1/24
2024-10-18T16:28:40 Notice kernel <118> v6/t6: 2601:100:8c00:dca4:227c:14ff:fea2:d520/64
2024-10-18T16:28:40 Notice kernel <118> ProdLAN (vlan0.5) -> v4: 10.5.7.1/24
2024-10-18T16:28:40 Notice kernel <118> v6/t6: 2601:100:8c00:dca3:227c:14ff:fea2:d521/64
2024-10-18T16:28:40 Notice kernel <118> NetHK (igc5_vlan2) -> v4: 10.5.6.1/24
2024-10-18T16:28:40 Notice kernel <118> v6/t6: 2601:100:8c00:dca0:227c:14ff:fea2:d520/64
2024-10-18T16:28:40 Notice kernel <118> LAN (vlan0.4093) -> v4: 10.5.5.1/24
2024-10-18T16:28:40 Notice kernel <118> KaliLAN (vlan0.7) -> v4: 10.254.0.1/24
2024-10-18T16:28:40 Notice kernel <118> v6/t6: 2601:100:8c00:dca6:227c:14ff:fea2:d521/64
2024-10-18T16:28:40 Notice kernel <118> KWFHVLAN (igc5_vlan4) -> v4: 10.5.254.1/24
2024-10-18T16:28:40 Notice kernel <118> IoTNet (igc5_vlan127) -> v4: 192.168.127.1/24
2024-10-18T16:28:40 Notice kernel <118> v6/t6: 2601:100:8c00:dca5:227c:14ff:fea2:d520/64
2024-10-18T16:28:40 Notice kernel <118> DevLAN (vlan0.6) -> v4: 10.1.0.1/24
2024-10-18T16:28:40 Notice kernel <118> v6/DHCP6: 2001:558:6021:2:6d08:c494:a872:291/128
2024-10-18T16:28:40 Notice kernel <118> Comcast (igc0) -> v4/DHCP4: 73.20.153.26/23
2024-10-18T16:28:40 Notice kernel <118> v6/t6: 2601:100:8c00:dcae:227c:14ff:fea2:d520/64
2024-10-18T16:28:40 Notice kernel <118> AdminLAN (igc4) -> v4: 10.5.1.1/24
2024-10-18T16:28:40 Notice kernel <118> v6/t6: 2601:100:8c00:dcaf:227c:14ff:fea2:d521/64
2024-10-18T16:28:40 Notice kernel <118> APMgtVLAN (igc5) -> v4: 10.5.2.1/24
2024-10-18T16:28:40 Notice kernel <118> 973Guest (igc5_vlan3) -> v4: 192.168.25.1/24
2024-10-18T16:28:40 Notice kernel <118>
2024-10-18T16:28:40 Notice kernel <118>*** gatekeeper.hdholm.net: OPNsense 24.7.6 ***
2024-10-18T16:28:40 Notice kernel <118>
2024-10-18T16:28:40 Notice kernel <118>Fri Oct 18 16:28:40 UTC 2024
2024-10-18T16:28:40 Notice kernel <118>Root file system: zroot/ROOT/known-good-20241014231644


[hdholm]

I guess before someone asks, the gateways look like this:

Code: [Select]

COMCAST_DHCP6 (active) Comcast IPv6 254 (upstream) fe80::dead:beef:2:1 2001:4860:4860::8844 20.3 ms 2.9 ms 0.0 % Interface COMCAST_DHCP6 Gateway
DevLANGW DevLAN IPv4 255 10.1.0.254 ~ ~ ~ Dev LAN Gateway
COMCAST_DHCP (active) Comcast IPv4 1 (upstream) 73.20.152.1 8.8.4.4 20.5 ms 2.1 ms 0.0 % Interface COMCAST_DHCP Gateway
WANBackup_DHCP WANBackup IPv4 250 (upstream) 172.20.10.1 172.20.10.1 0.7 ms 0.1 ms 0.0 %