High ping glitches

[Learning]

Hi folks.

Running the latest OPNsense (all updated) on an i3 2nd gen.  CPU & RAM use always low on this device as it just runs a couple of VPN connections for routing traffic.

Over the last 3 weeks or so, I've been getting problems with glitches which I can see when I run a ping test.

If you check out the ping from Google below, you'll notice average times in the 36ms range.  This would be expected.  But you'll also see multiple high latency returns, right up to over 1000ms.

Code Select Expand

64 bytes from sea09s29-in-f14.1e100.net (142.250.217.78): icmp_seq=20 ttl=117 time=35.7 ms
64 bytes from sea09s29-in-f14.1e100.net (142.250.217.78): icmp_seq=21 ttl=117 time=33.8 ms
64 bytes from sea09s29-in-f14.1e100.net (142.250.217.78): icmp_seq=22 ttl=117 time=36.1 ms
64 bytes from sea09s29-in-f14.1e100.net (142.250.217.78): icmp_seq=23 ttl=117 time=36.0 ms
64 bytes from sea09s29-in-f14.1e100.net (142.250.217.78): icmp_seq=24 ttl=117 time=1141 ms
64 bytes from sea09s29-in-f14.1e100.net (142.250.217.78): icmp_seq=25 ttl=117 time=87.0 ms
64 bytes from sea09s29-in-f14.1e100.net (142.250.217.78): icmp_seq=26 ttl=117 time=41.6 ms
64 bytes from sea09s29-in-f14.1e100.net (142.250.217.78): icmp_seq=27 ttl=117 time=34.5 ms
64 bytes from sea09s29-in-f14.1e100.net (142.250.217.78): icmp_seq=28 ttl=117 time=523 ms


This is an issue that only recently started, and is affecting all traffic, even pure direct-to-WAN (VPN bypass).

This recurring and persistent problem is affecting VoIP calls and my kid's internet gaming is apparently suffering considerably.

I hooked a PC direct to modem, and the issue disappeared(45 minute test), so I know it is happening through OPNsense (or possibly my D-Link switch).  This is with devices that are hardwired.

Any thoughts on how I could begin to troubleshoot?
Thanks!

[verasense]

Hi,

I am just going to think out loud some ideas to find out more about the issue... Apologies if you have already done all this testing.

Does the issue happen almost consistently when pinging different servers?

Have you tried connecting the PC directly to OPNsense to see if the issue persist? (since you said there is a switch in the middle).

Do you experience the same from the router itself? Try to do a ping (with a large count) from OPNsense "Interfaces - Diagnostics - Ping" and see what you get.

I would also try to do Traceroutes from the computer and from OPNsense (also in Interfaces - Diagnostics).

Ping Plotter is a tool I used in the past to check latency issues too (they have a free trial).

I remember a jitter issue I found via Wireshark and Smart TV and Plex were to be blamed, but your case seems different. The fact that this happens in more than one computer and that it is a ICMP ECHO request make me think that it is not related to PC processor congestion or TCP windows. Router congestion seems like a possible reason but you already said it is unlikely.

[Learning]

Hi,

I am just going to think out loud some ideas to find out more about the issue... Apologies if you have already done all this testing.

Does the issue happen almost consistently when pinging different servers?

Yes, it happens regardless of server.

Have you tried connecting the PC directly to OPNsense to see if the issue persist? (since you said there is a switch in the middle).

Do you experience the same from the router itself? Try to do a ping (with a large count) from OPNsense "Interfaces - Diagnostics - Ping" and see what you get.

Great idea.  Hadn't tried this, and appreciate you providing the instructions on how to do it.  I ran 10 consecutive tests each with 10 pings.  No degradation at all.

Your thoughts then prompted me to perform some other tests.
I hardwired a PC direct to the OPNsense box.  Ping was just fine, for 3 minutes.
Then I ran Speedtest.net at the same time as ping, and the problem started appearing again.
Seems the issue is just when under load!

So, hooked the PC direct to modem.  Ran the same speedtest & simultaneous ping test.  Got the same result.  A few 34ms pings, then BAM, 1427ms.  When I had previously hooked the PC direct to modem, there was no download activity.

That suggests to me it is either the modem, or the the ISP and not OPNsense at all.  Which makes sense since I hadn't changed the config in OPNsense prior to the issue commencing.

Thanks for your pointers verasense!

[verasense]

Glad I could help!

If it is related to the load I think you can also use OPNsense to investigate if there is a computer using a lot of bandwidth so, for example, someone/something at home is doing a lot of connections and degradating the network  user experience. There is even an option (I recall it was at the beginning) that shows you a graph.

Similarly, you can also put devices in different VLANs, limit traffic from some devices so they do not overload the modem, and give priority to e.g. the gaming computers so they do not suffer lag if other devices want to overload the system.

But I will be cautious since sometimes the source of problems is not as clear as we initially think.