Messages

Ok but does it behave? ;)

So far I can tell the hopping is gone; it stays stable on the Telekom gateway as expected.
I'm only not very happy that we need to disable the multipath support to "fix" the issue.
But I don't know if there is an another way to handle the issue.

Anyways, thx Franco for helping and find the cause of the problems.

Looks way better now:

Code Select Expand


# sysctl -a | grep net.route.multipath
net.route.multipath: 0


Code Select Expand


# netstat -4 -r -n
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            62.155.242.170     UGS      pppoe0
1.0.0.1            178.27.68.254      UGHS       igb2
1.1.1.1            62.155.242.170     UGHS     pppoe0
...

Certainly odd it has both default routes. Do you have default gateway switching on or off?

Default gateway switching is active.

We're going to prevent that from happening for now using:

# sysctl net.route.multipath=0


Cheers,
Franco

I will test this in a minute; thank you franco.

Wie sieht deine Config für das Interface aus?
Vor allem für das IPv6?

OK, small update from my side:
I "solved" this issue for now with a workaround: I setuped the gateway groups for IPv4 (Telekom and Vodafone) and IPv6 (only Telekom) and send every traffic from the LAN net to the GW Groups.
For now it seems that this is working; the traffic goes mostly via Telekom.

Unfortunately, when checking the situation directly on the firewall, the traffic goes nearly the complete time via Vodafone.

One thing I want to add is, that the route table on the FW looks like this:

Code Select Expand

$ netstat -4 -r -n
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            178.27.64.254      UGS        igb2
default            62.155.242.170     UGS      pppoe0
...

For me, with my understanding of routing etc, this routing table can be the cause that everything goes via Vodafone.
But I'm not a expert how this stuff should look like / work under FreeBSD / Opnsense.

If your MAC address changed, make sure to reboot the Cable modem once.

Thanks for the hint; but normally the NICs are not changing on the firewall side.

---

After enabling the vodafone interface again, the apply is running for ages (still loading currently), but the interface is up and have a IP.

General log:

Code Select Expand

2022-01-13T16:21:11 Error opnsense /usr/local/etc/rc.filter_configure: ROUTING: keeping current default gateway 'fe80::9ecc:83ff:fec8:789f%pppoe0'
2022-01-13T16:21:11 Error opnsense /usr/local/etc/rc.filter_configure: ROUTING: keeping current default gateway '62.155.242.170'
2022-01-13T16:21:11 Error opnsense /system_gateways.php: Adding static route for monitor 1.0.0.1 via 178.27.64.254
2022-01-13T16:21:11 Error opnsense /system_gateways.php: Removing static route for monitor 1.0.0.1 via 178.27.64.254
2022-01-13T16:21:11 Error opnsense /system_gateways.php: Adding static route for monitor 1.1.1.1 via 62.155.242.170
2022-01-13T16:21:11 Error opnsense /system_gateways.php: Removing static route for monitor 1.1.1.1 via 62.155.242.170
2022-01-13T16:21:11 Error opnsense /system_gateways.php: Choose to bind TELEKOM_PPPOE on 91.9.129.160 since we could not find a proper match.
2022-01-13T16:21:11 Error opnsense /system_gateways.php: Adding static route for monitor 2606:4700:4700::1111 via fe80::9ecc:83ff:fec8:789f%pppoe0
2022-01-13T16:21:11 Error opnsense /system_gateways.php: Removing static route for monitor 2606:4700:4700::1111 via fe80::9ecc:83ff:fec8:789f%pppoe0
2022-01-13T16:21:11 Error opnsense /system_gateways.php: ROUTING: keeping current default gateway 'fe80::9ecc:83ff:fec8:789f%pppoe0'
2022-01-13T16:21:11 Error opnsense /system_gateways.php: ROUTING: setting IPv6 default route to fe80::9ecc:83ff:fec8:789f
2022-01-13T16:21:11 Error opnsense /system_gateways.php: ROUTING: IPv6 default gateway set to opt2
2022-01-13T16:21:11 Error opnsense /system_gateways.php: ROUTING: keeping current default gateway '62.155.242.170'
2022-01-13T16:21:11 Error opnsense /system_gateways.php: ROUTING: setting IPv4 default route to 62.155.242.170
2022-01-13T16:21:11 Error opnsense /system_gateways.php: ROUTING: IPv4 default gateway set to opt2
2022-01-13T16:21:11 Error opnsense /system_gateways.php: ROUTING: entering configure using defaults

Ifconfig:

Code Select Expand

$ ifconfig
igb0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: LAN
options=4e0272b<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,TSO4,TSO6,LRO,WOL_MAGIC,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
ether 20:7c:14:a0:64:4e
inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
inet6 fe80::227c:14ff:fea0:644e%igb0 prefixlen 64 scopeid 0x1
inet6 2003:ef:bf0e:2a00:227c:14ff:fea0:644e prefixlen 64
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
igb1: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
ether 20:7c:14:a0:64:4f
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
igb2: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: Vodafone
options=4e0272b<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,TSO4,TSO6,LRO,WOL_MAGIC,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
ether 20:7c:14:a0:64:50
inet 178.27.64.56 netmask 0xffffff00 broadcast 178.27.64.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
igb3: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: TelekomStatic
options=4e0272b<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,TSO4,TSO6,LRO,WOL_MAGIC,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
ether 20:7c:14:a0:64:51
inet 192.168.100.5 netmask 0xffffff00 broadcast 192.168.100.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
enc0: flags=0<> metric 0 mtu 1536
groups: enc
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=20100<PROMISC,PPROMISC> metric 0 mtu 33160
groups: pflog
pfsync0: flags=0<> metric 0 mtu 1500
syncpeer: 0.0.0.0 maxupd: 128 defer: off
syncok: 1
groups: pfsync
pppoe0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1492
description: Telekom
inet6 fe80::227c:14ff:fea0:644e%pppoe0 prefixlen 64 scopeid 0x9
inet6 2003:ef:bfff:e3f:227c:14ff:fea0:644e prefixlen 64 autoconf
inet 91.9.129.160 --> 62.155.242.170 netmask 0xffffffff
nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>

Not sure why it doesn't return an address. What does ifconfig say to VODAFONE_DHCP interface? Any IP at all? Coming and going?

We will see, I disabled the Vodafone interface to have a stable connection but will enable it again.

This is the log after I disabled the gateway monitoring:

Code Select Expand

2022-01-13T15:04:08 Error opnsense /usr/local/etc/rc.newwanipv6: Resyncing OpenVPN instances for interface Telekom.
2022-01-13T15:04:08 Error opnsense /usr/local/etc/rc.newwanipv6: ROUTING: keeping current default gateway 'fe80::9ecc:83ff:fec8:789f%pppoe0'
2022-01-13T15:04:07 Error opnsense /usr/local/etc/rc.newwanipv6: ROUTING: keeping current default gateway '62.155.242.170'
2022-01-13T15:04:07 Error opnsense /usr/local/etc/rc.newwanipv6: The VODAFONE_DHCP IPv4 gateway address is invalid, skipping.
2022-01-13T15:04:07 Error opnsense /usr/local/etc/rc.newwanipv6: Choose to bind VODAFONE_DHCP on since we could not find a proper match.
2022-01-13T15:04:07 Error opnsense /usr/local/etc/rc.newwanipv6: Removing static route for monitor 1.1.1.1 via 62.155.242.170
2022-01-13T15:04:07 Error opnsense /usr/local/etc/rc.newwanipv6: Choose to bind TELEKOM_PPPOE on 91.9.129.160 since we could not find a proper match.
2022-01-13T15:04:07 Error opnsense /usr/local/etc/rc.newwanipv6: Removing static route for monitor 2606:4700:4700::1111 via fe80::9ecc:83ff:fec8:789f%pppoe0
2022-01-13T15:04:07 Error opnsense /usr/local/etc/rc.newwanipv6: ROUTING: keeping current default gateway 'fe80::9ecc:83ff:fec8:789f%pppoe0'
2022-01-13T15:04:07 Error opnsense /usr/local/etc/rc.newwanipv6: ROUTING: setting IPv6 default route to fe80::9ecc:83ff:fec8:789f
2022-01-13T15:04:07 Error opnsense /usr/local/etc/rc.newwanipv6: ROUTING: IPv6 default gateway set to opt2
2022-01-13T15:04:07 Error opnsense /usr/local/etc/rc.newwanipv6: ROUTING: keeping current default gateway '62.155.242.170'
2022-01-13T15:04:07 Error opnsense /usr/local/etc/rc.newwanipv6: ROUTING: setting IPv4 default route to 62.155.242.170
2022-01-13T15:04:07 Error opnsense /usr/local/etc/rc.newwanipv6: ROUTING: IPv4 default gateway set to opt2
2022-01-13T15:04:07 Error opnsense /usr/local/etc/rc.newwanipv6: ROUTING: entering configure using 'opt2'
2022-01-13T15:04:07 Error opnsense /usr/local/etc/rc.newwanipv6: On (IP address: 2003:ef:bfff:e3f:227c:14ff:fea0:644e) (interface: Telekom[opt2]) (real interface: pppoe0).
2022-01-13T15:04:07 Error opnsense /usr/local/etc/rc.newwanipv6: IPv6 renewal is starting on 'pppoe0'
2022-01-13T15:04:06 Error opnsense /usr/local/etc/rc.filter_configure: ROUTING: keeping current default gateway 'fe80::9ecc:83ff:fec8:789f%pppoe0'
2022-01-13T15:04:06 Error opnsense /usr/local/etc/rc.filter_configure: ROUTING: keeping current default gateway '62.155.242.170'
2022-01-13T15:04:06 Error opnsense /interfaces.php: The VODAFONE_DHCP IPv4 gateway address is invalid, skipping.
2022-01-13T15:04:06 Error opnsense /interfaces.php: Choose to bind VODAFONE_DHCP on since we could not find a proper match.
2022-01-13T15:04:06 Error opnsense /interfaces.php: Removing static route for monitor 1.1.1.1 via 62.155.242.170
2022-01-13T15:04:06 Error opnsense /interfaces.php: Choose to bind TELEKOM_PPPOE on 91.9.129.160 since we could not find a proper match.
2022-01-13T15:04:06 Error opnsense /interfaces.php: The TELEKOM_DHCP6 IPv6 gateway address could not be found, skipping.
2022-01-13T15:04:06 Error opnsense /interfaces.php: ROUTING: keeping current default gateway 'fe80::9ecc:83ff:fec8:789f%pppoe0'
2022-01-13T15:04:06 Error opnsense /interfaces.php: ROUTING: setting IPv6 default route to fe80::9ecc:83ff:fec8:789f
2022-01-13T15:04:06 Error opnsense /interfaces.php: ROUTING: IPv6 default gateway set to opt2
2022-01-13T15:04:06 Error opnsense /interfaces.php: ROUTING: keeping current default gateway '62.155.242.170'
2022-01-13T15:04:06 Error opnsense /interfaces.php: ROUTING: setting IPv4 default route to 62.155.242.170
2022-01-13T15:04:06 Error opnsense /interfaces.php: ROUTING: IPv4 default gateway set to opt2
2022-01-13T15:04:06 Error opnsense /interfaces.php: ROUTING: entering configure using defaults
2022-01-13T15:04:06 Error opnsense /interfaces.php: Clearing states for stale opt1 route on igb2
2022-01-13T15:04:06 Critical dhclient exiting.
2022-01-13T15:04:06 Error dhclient connection closed
2022-01-13T15:04:06 Error opnsense /usr/local/etc/rc.newwanip: Resyncing OpenVPN instances for interface Telekom.
2022-01-13T15:04:06 Error opnsense /usr/local/etc/rc.newwanip: IP address change detected, killing states of old ip 91.9.129.136
2022-01-13T15:04:05 Error opnsense /usr/local/etc/rc.newwanip: ROUTING: keeping current default gateway 'fe80::9ecc:83ff:fec8:789f%pppoe0'
2022-01-13T15:04:05 Error opnsense /usr/local/etc/rc.newwanip: ROUTING: keeping current default gateway '62.155.242.170'
2022-01-13T15:04:05 Error opnsense /usr/local/etc/rc.newwanip: Choose to bind VODAFONE_DHCP on 178.27.64.56 since we could not find a proper match.
2022-01-13T15:04:05 Error opnsense /usr/local/etc/rc.newwanip: Removing static route for monitor 1.1.1.1 via 62.155.242.170
2022-01-13T15:04:05 Error opnsense /usr/local/etc/rc.newwanip: Choose to bind TELEKOM_PPPOE on 91.9.129.160 since we could not find a proper match.
2022-01-13T15:04:05 Error opnsense /usr/local/etc/rc.newwanip: The TELEKOM_DHCP6 IPv6 gateway address could not be found, skipping.
2022-01-13T15:04:05 Error opnsense /usr/local/etc/rc.newwanipv6: Failed to detect IP for Telekom[opt2]
2022-01-13T15:04:05 Error opnsense /usr/local/etc/rc.newwanipv6: On (IP address: ) (interface: Telekom[opt2]) (real interface: pppoe0).
2022-01-13T15:04:05 Error opnsense /usr/local/etc/rc.newwanipv6: IPv6 renewal is starting on 'pppoe0'
2022-01-13T15:04:05 Error opnsense /usr/local/etc/rc.newwanip: ROUTING: keeping current default gateway 'fe80::9ecc:83ff:fec8:789f%pppoe0'
2022-01-13T15:04:05 Error opnsense /usr/local/etc/rc.newwanip: ROUTING: setting IPv6 default route to fe80::9ecc:83ff:fec8:789f
2022-01-13T15:04:05 Error opnsense /usr/local/etc/rc.newwanip: ROUTING: IPv6 default gateway set to opt2
2022-01-13T15:04:05 Error opnsense /usr/local/etc/rc.newwanip: ROUTING: keeping current default gateway '62.155.242.170'
2022-01-13T15:04:05 Error opnsense /usr/local/etc/rc.newwanip: ROUTING: setting IPv4 default route to 62.155.242.170
2022-01-13T15:04:05 Error opnsense /usr/local/etc/rc.newwanip: ROUTING: IPv4 default gateway set to opt2
2022-01-13T15:04:05 Error opnsense /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'opt2'
2022-01-13T15:04:05 Error opnsense /usr/local/etc/rc.newwanip: On (IP address: 91.9.129.160) (interface: Telekom[opt2]) (real interface: pppoe0).
2022-01-13T15:04:05 Error opnsense /usr/local/etc/rc.newwanip: IPv4 renewal is starting on 'pppoe0'
2022-01-13T15:04:05 Error opnsense /interfaces.php: Warning! dhcpd_radvd_configure(auto) found no suitable IPv6 address on igb0
2022-01-13T15:04:05 Error opnsense /interfaces.php: ROUTING: skipping IPv6 default route
2022-01-13T15:04:05 Error opnsense /interfaces.php: ROUTING: IPv6 default gateway set to opt2
2022-01-13T15:04:05 Error opnsense /interfaces.php: ROUTING: entering configure using 'opt2'
2022-01-13T15:04:05 Error opnsense /interfaces.php: Accept router advertisements on interface pppoe0
2022-01-13T15:04:03 Error dhcp6c transmit failed: Can't assign requested address

And the gateway logfile:

Code Select Expand

2022-01-13T14:50:50 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 1.0.0.1 bind_addr 178.27.64.56 identifier "VODAFONE_DHCP "
2022-01-13T14:50:50 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 1.1.1.1 bind_addr 91.9.129.136 identifier "TELEKOM_PPPOE "
2022-01-13T14:50:50 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 2606:4700:4700::1111 bind_addr 2003:ef:bfff:e1b:227c:14ff:fea0:644e identifier "TELEKOM_DHCP6 "
2022-01-13T14:50:49 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 1.0.0.1 bind_addr 178.27.64.56 identifier "VODAFONE_DHCP "
2022-01-13T14:50:49 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 1.1.1.1 bind_addr 91.9.129.136 identifier "TELEKOM_PPPOE "
2022-01-13T14:50:48 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 1.0.0.1 bind_addr 178.27.64.56 identifier "VODAFONE_DHCP "
2022-01-13T14:50:48 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 1.1.1.1 bind_addr 91.9.129.136 identifier "TELEKOM_PPPOE "
2022-01-13T14:50:48 Warning dpinger TELEKOM_DHCP6 2606:4700:4700::1111: sendto error: 65
2022-01-13T14:50:47 Warning dpinger TELEKOM_PPPOE 1.1.1.1: sendto error: 50
2022-01-13T14:50:47 Warning dpinger TELEKOM_DHCP6 2606:4700:4700::1111: sendto error: 50
2022-01-13T14:50:46 Warning dpinger TELEKOM_PPPOE 1.1.1.1: sendto error: 50
2022-01-13T14:50:46 Warning dpinger TELEKOM_DHCP6 2606:4700:4700::1111: sendto error: 50
2022-01-13T14:50:45 Warning dpinger TELEKOM_PPPOE 1.1.1.1: sendto error: 50
2022-01-13T14:49:09 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 1.0.0.1 bind_addr 178.27.64.56 identifier "VODAFONE_DHCP "
2022-01-13T14:49:09 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 1.1.1.1 bind_addr 91.9.129.208 identifier "TELEKOM_PPPOE "
2022-01-13T14:49:09 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 2606:4700:4700::1111 bind_addr 2003:ef:bfff:e2a:227c:14ff:fea0:644e identifier "TELEKOM_DHCP6 "
2022-01-13T14:48:42 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 1.1.1.1 bind_addr 91.9.129.208 identifier "TELEKOM_PPPOE "
2022-01-13T14:48:42 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 2606:4700:4700::1111 bind_addr 2003:ef:bfff:e2a:227c:14ff:fea0:644e identifier "TELEKOM_DHCP6 "
2022-01-13T14:47:59 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 1.1.1.1 bind_addr 91.9.129.208 identifier "TELEKOM_PPPOE "
2022-01-13T14:47:59 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 2606:4700:4700::1111 bind_addr 2003:ef:bfff:e2a:227c:14ff:fea0:644e identifier "TELEKOM_DHCP6 "
2022-01-13T14:45:14 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 1.1.1.1 bind_addr 91.9.129.208 identifier "TELEKOM_PPPOE "
2022-01-13T14:45:14 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 2606:4700:4700::1111 bind_addr 2003:ef:bfff:e2a:227c:14ff:fea0:644e identifier "TELEKOM_DHCP6 "
2022-01-13T14:43:58 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 1.0.0.1 bind_addr 178.27.64.56 identifier "VODAFONE_DHCP "
2022-01-13T14:43:58 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 1.1.1.1 bind_addr 91.9.129.208 identifier "TELEKOM_PPPOE "
2022-01-13T14:43:58 Warning dpinger send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 2606:4700:4700::1111 bind_addr 2003:ef:bfff:e2a:227c:14ff:fea0:644e identifier "TELEKOM_DHCP6 "

Since the switching still happen after disabling the monitoring, for me it seems to be not related to the monitoring itself.
On the other side, the system log don't look healthy...

VG;
Alex

Which log files would help to debug this?

I disabled the monitoring of the gateways already; but it still is happening.

Hello,

after upgrading to 22.1 R1 from 21.7 I saw a strange "gateway hopping".

Background:
To my firewall there are two internet connections connected:

• Telekom via PPPoE
• Vodafone via DHCP (ISP Provider in bridge mode)

In the past the gateways were configured that both gateways from Telekom (PPPoE IPv4 & DHCPv6 IPv6) have the priority of 253, the gateway for Vodafone have 254.
That was working as expected, as normally the Telekom connections were used mainly and only if there was a issue, the vodafone gateway were used.

Now after the switch to 22.1 I noticed that often the connection is switching between Telekom and Vodafone, which  cause issues with VOIP and other services which needs a stable connection.

The only tempfix now was to disable the vodafone interface; which is of course not ideal.
I know the "normal" way to handle a failover connection is to use the gateway group; but out of KISS reasons I tested the above solution and it worked for month without any major issue.

Was there a change in the routing behavior in 22.1?

Thanks,
Alex

Thank you franco, sometimes the problem sits in front of the screen.
Upgrade worked well from 21.7 --> 22.1 Beta --> 22.1 r.1

Hello,

maybe I'm to stupid to upgrade from 21.7.7 to 22.1.r1:

Code Select Expand

# opnsense-update -ur 22.1.r1
Fetching packages-22.1.r1-OpenSSL-amd64.tar: .. failed, no signature found

If I activate the verbosed log, I can see the issue:

Code Select Expand

# opnsense-update -V -ur 22.1.r1
+ [ '-r 22.1.r1' '=' -R ]
+ [ -n '-r 22.1.r1' ]
+ RELEASE=22.1.r1
+ [ -n '' ]
+ [ -n '' ]
+ [ ! -f /usr/local/etc/pkg/repos/OPNsense.conf ]
+ [ -n '' ]
+ [ -n '' ]
+ [ -n '' ]
+ [ -n '' ]
+ [ -n '' ]
+ [ -n '' ]
+ grep -q '^[[:space:]]*signature_type:[[:space:]]*"fingerprints"' /usr/local/etc/pkg/repos/OPNsense.conf
+ [ -n '' ]
+ [ -z '' ]
+ [ -n -u -o -n '' ]
+ DO_KERNEL=-k
+ DO_BASE=-b
+ DO_PKGS=-p
+ [ -n '' ]
+ [ -n '' ]
+ [ -z '' ]
+ [ -n -k -a -n '' -a -z -u ]
+ [ -n -b -a -n '' -a -z -u ]
+ [ -n -p -a -n '' -a -z -u ]
+ [ -n '' ]
+ [ -b '=' -B ]
+ [ -p '=' -P ]
+ [ -n '' ]
+ [ -p '=' -p -a -z -u ]
+ [ -n '' ]
+ FLAVOUR=Base
+ [ -n '' ]
+ [ -f /usr/local/bin/openssl ]
+ /usr/local/bin/openssl version
+ awk '{ print $1 }'
+ FLAVOUR=OpenSSL
+ DEVICE=''
+ [ -n '' ]
+ PACKAGESSET=packages-22.1.r1-OpenSSL-amd64.tar
+ KERNELSET=kernel-22.1.r1-amd64.txz
+ BASESET=base-22.1.r1-amd64.txz
+ mirror_abi
+ local 'DIR=\2'
+ [ -n '' ]
+ opnsense-verify -a
+ ABI=FreeBSD:12:amd64
+ [ -n '' ]
+ sed -n 's/^[[:space:]]*url:[[:space:]]*\"pkg\+\(.*\/${ABI}\/\)\([^\/]*\)\/.*/\1\2/p' /usr/local/etc/pkg/repos/OPNsense.conf
+ MIRROR='https://mirror.dns-root.de/opnsense/${ABI}/21.7'
+ [ -z 'https://mirror.dns-root.de/opnsense/${ABI}/21.7' ]
+ eval 'MIRROR=https://mirror.dns-root.de/opnsense/${ABI}/21.7'
+ MIRROR=https://mirror.dns-root.de/opnsense/FreeBSD:12:amd64/21.7
+ echo https://mirror.dns-root.de/opnsense/FreeBSD:12:amd64/21.7
+ MIRROR=https://mirror.dns-root.de/opnsense/FreeBSD:12:amd64/21.7/sets
+ [ -n '' ]
+ [ -z '' ]
+ [ 22.1.r1 '=' 21.7.7 -a -n -k ]
+ [ 22.1.r1 '=' 21.7.7 -a -n -b ]
+ [ -z -k-b-p ]
+ [ -p '=' -p ]
+ [ '-r 22.1.r1' '=' -R -a 22.1.r1 '=' unknown ]
+ [ -z '' -o -n -u ]
+ rm -f /usr/local/opnsense/version/core.lock
+ fetch_set packages-22.1.r1-OpenSSL-amd64.tar
+ STAGE1='opnsense-fetch -a -T 30 -q -o /var/cache/opnsense-update/9672/packages-22.1.r1-OpenSSL-amd64.tar.sig https://mirror.dns-root.de/opnsense/FreeBSD:12:amd64/21.7/sets/packages-22.1.r1-OpenSSL-amd64.tar.sig'
+ STAGE2='opnsense-fetch -a -T 30 -q -o /var/cache/opnsense-update/9672/packages-22.1.r1-OpenSSL-amd64.tar https://mirror.dns-root.de/opnsense/FreeBSD:12:amd64/21.7/sets/packages-22.1.r1-OpenSSL-amd64.tar'
+ STAGE3='opnsense-verify -q /var/cache/opnsense-update/9672/packages-22.1.r1-OpenSSL-amd64.tar'
+ [ -n '' ]
+ [ -n '' ]
+ echo -n 'Fetching packages-22.1.r1-OpenSSL-amd64.tar: .'
Fetching packages-22.1.r1-OpenSSL-amd64.tar: .+ mkdir -p /var/cache/opnsense-update/9672
+ opnsense-fetch -a -T 30 -q -o /var/cache/opnsense-update/9672/packages-22.1.r1-OpenSSL-amd64.tar.sig https://mirror.dns-root.de/opnsense/FreeBSD:12:amd64/21.7/sets/packages-22.1.r1-OpenSSL-amd64.tar.sig
.+ exit_msg ' failed, no signature found'
+ [ -n ' failed, no signature found' ]
+ echo ' failed, no signature found'
failed, no signature found
+ exit 1

It try to get the packages.tar from the 21.7 folder on the mirror; where of course the set is not stored.
What did I maybe missed?

Thanks a lot,
Alex

Hallo,

auch wenn dieser Thread schon "etwas" älter ist, wollte ich fragen, ob du eine finale Lösung für das Problem gefunden hast?

Vielen Dank,
Alex

Hello,

I use the internal CA of the OpnSense Firewall to create SSL certs for internal devices.
In the past, that was working without any problem, but when I tried to issue a new cert today, the entered subjectAltName's are not part of the issued cert.

I douple checked all my settings and the issued cert, but it seems that something is going wrong here.

Can someone reproduce this issue or is able to help me in this topic?

Thanks,
Alex

Hello,

after the upgrade to 20.7 I have a very special problem:
I normally use the IPv4 and IPv6 addresses of my external DNS resolvers as Health Endpoints for the Gateway monitoring.
After the upgrade, the IPv6 address of my resolver is not pingable anymore from the firewall.
The strange thing is: The IP is still accessable via ICMP from my client or any other computer, only the firewall is not able to reach the DNS server via IPv6.
DNS requests are no problem; only ICMP is affected.

Do anyone have a idea for this issue?

Thanks,

Alex