System:Gateways:Single and dpinger not working

[darp12345]

I'm having a problem on one of my internet connections (I have two) when I enable gateway monitoring.
When I enable gateway monitoring, I get either 100% loss or periods of few min with a loss and few min with no loss. I figured out that the problem seems to be related to the host being monitored added to the routing table. I see this row added when I enable gateway monitoring (the host being pinged is 4.2.2.3).

Code Select Expand

Routing tables
Internet:
Destination        Gateway            Flags     Netif Expire
...
4.2.2.3            98.51.132.1        UGHS       cxl1
...

When that entry is in the routing table, pinging the host doesn't work

Code Select Expand


ping 4.2.2.3
PING 4.2.2.3 (4.2.2.3): 56 data bytes
^C
--- 4.2.2.3 ping statistics ---
5 packets transmitted, 0 packets received, 100.0% packet loss

I also tried tcpdump to see if the ping is working and apparently each reply is duplicated when the route to the host being monitoried is in the routing table

Code Select Expand


tcpdump -n -i cxl1 icmp
...
20:02:51.431991 IP 98.51.135.187 > 4.2.2.3: ICMP echo request, id 18212, seq 963, length 8
20:02:51.460338 IP 4.2.2.3 > 98.51.135.187: ICMP echo reply, id 18212, seq 963, length 8
20:02:51.460349 IP 4.2.2.3 > 98.51.135.187: ICMP echo reply, id 18212, seq 963, length 8

This is what dpinger looks like for this interface

Code Select Expand


/usr/local/bin/dpinger -f -S -r 0 -i WAN_DHCP -B 98.51.135.187 -p /var/run/dpinger_WAN_DHCP.pid -u /var/run/dpinger_WAN_DHCP.sock -C /usr/local/etc/rc.syshook monitor -s 1s -l 2s -t 60s -A 1s -D 500 -L 20 -d 0 4.2.2.3


Given that dpinger is using explicit interface, it isn't clear why that route has to be added. I looked at the code (/usr/local/etc/inc/plugins.inc.d/dpinger.inc)and found this:

Code Select Expand


            /*
             * If the gateway is the same as the monitor we do not add a
             * route as this will break the routing table.
             * Add static routes for each gateway with their monitor IP
             * not strictly necessary but is an added level of protection.
             */
            if (is_ipaddrv4($gateway['gateway']) && $gateway['monitor'] != $gateway['gateway']) {
                log_error("Adding static route for monitor {$gateway['monitor']} via {$gateway['gateway']}");
                system_host_route($gateway['monitor'], $gateway['gateway'], false, true);
            }

The comment seems to indicate as well that adding the route is redundant. I commented the section and tried again.
This seems to have solved the problem. tcpdump shows again single reply for each request

Code Select Expand

tcpdump -n -i cxl1 icmp
...
20:09:37.074603 IP 98.51.135.187 > 4.2.2.3: ICMP echo request, id 8970, seq 12, length 8
20:09:37.111082 IP 4.2.2.3 > 98.51.135.187: ICMP echo reply, id 8970, seq 12, length 8

pinging the host being monitored also works. It does require however to specify source address given that there isn't a route. When the host being monitored is in the routing table, ping with a source address doesn't work as well as ping without it.

Code Select Expand


ping -S 98.51.135.187 4.2.2.3
PING 4.2.2.3 (4.2.2.3) from 98.51.135.187: 56 data bytes
64 bytes from 4.2.2.3: icmp_seq=0 ttl=57 time=18.534 ms
64 bytes from 4.2.2.3: icmp_seq=1 ttl=57 time=17.202 ms


So, I guess the question here is - can we remove that code that adds static route given that dpinger uses explicit interface?
I can't say why that is a problem. I'm just observing that with the static route it doesn't work in my case and without it - it does.

[tracerrx]

Hmm maybe this is whats causing the gateway flapping to occur... Interesting find..

[tracerrx]

So a reply from @franco in another thread has helped me solve this.  Apparently you cannot have the same ip's set in settings-->general DNS servers and as a gateway monitoring target.  Changing one to the alternate dns ip (such as 8.8.4.4) has solved the issue for me.