Throughput on WAN took a noose dive back in Dec

[nullspace]

So my WAN down and up started to tank sometime in december and all I can think is an update caused it. Today I had some time and tweaked the MTU a little with middling to no change:

ersion   Date   
26.1.5 (installed)   2026-03-24   
26.1.4   2026-03-11   
26.1.3   2026-03-04   
26.1.2   2026-02-12   
26.1.1   2026-02-04   
26.1   2026-01-28   
25.7.11   2026-01-15   
25.7.10   2025-12-18   
25.7.9   2025-12-04   
25.7.8   2025-11-26   
25.7.7   2025-11-06   
25.7.6   2025-10-22   
25.7.5   2025-10-08   
25.7.4   2025-09-30   
25.7.3   2025-09-09


Timestamp (GMT)   Server id   Server name   Download   Upload   Latency
2026-03-26T16:40:47   69076   Ripplefiber, Detroit, MI   154.27   119.56   34.63
2026-03-26T16:39:46   53778   123NET, Southfield, MI   321.78   162.60   23.07
2026-03-26T16:39:26   53778   123NET, Southfield, MI   427.84   150.26   18.03
2026-03-26T16:30:47   53778   123NET, Southfield, MI   303.88   158.49   23.59
2026-03-26T16:30:18   69076   Ripplefiber, Detroit, MI   175.65   118.15   35.37
2026-03-26T15:55:38   69076   Ripplefiber, Detroit, MI   282.70   66.56   46.06
2026-03-26T15:54:56   53778   123NET, Southfield, MI   309.22   197.95   22.45
2026-03-26T15:18:56   53778   123NET, Southfield, MI   298.12   178.34   31.09
2026-03-26T15:12:58   15342   Jefferson County Cable, Toronto, OH   532.03   87.85   81.72
2026-03-26T15:09:05   69076   Ripplefiber, Detroit, MI   337.01   80.93   33.71
2026-03-26T15:08:41   69076   Ripplefiber, Detroit, MI   269.19   96.78   36.84
2026-03-26T14:56:52   69076   Ripplefiber, Detroit, MI   147.98   103.90   31.88
2026-03-26T14:56:14   69076   Ripplefiber, Detroit, MI   258.65   82.40   33.43
2026-03-26T14:48:14   69076   Ripplefiber, Detroit, MI   132.60   117.18   39.13
2026-03-26T14:12:39   73050   Omni Fiber, Detroit, MI   135.47   69.68   62.72
2026-03-26T13:59:46   73050   Omni Fiber, Detroit, MI   156.37   80.17   50.09
2026-03-26T13:56:54   69076   Ripplefiber, Detroit, MI   207.24   98.28   45.51
2026-03-26T13:53:49   25491   Cable Co-op, Oberlin, OH   43.46   5.73   1035.91
2026-03-25T11:43:25   16973   Spectrum, Livonia, MI   193.45   98.21   46.64
2026-03-25T11:42:35   74288   EZEE Fiber, Detroit, MI   144.72   96.05   61.52
2026-03-25T11:42:05   5709   Merit Network, Inc, Ann Arbor, MI   288.97   93.82   70.66
2026-03-25T11:41:17   71374   Wyandotte Municipal Services, Wyandotte, MI   310.37   120.34   49.01
2025-12-28T14:45:18   6124   Merit Network, Inc, Southfield, MI   306.94   94.94   64.64
2025-12-13T14:27:32   1778   Comcast, Detroit, MI   517.04   191.64   43.55
2025-12-13T14:26:55   64415   Surf Internet, Southfield, MI   144.70   100.25   55.62
2025-07-12T20:47:45   1778   Comcast, Detroit, MI   1846.12   260.60   13.40
2025-06-28T09:38:00   53778   123NET, Southfield, MI   1472.07   346.66   14.02
2025-06-28T09:36:58   1778   Comcast, Detroit, MI   2310.24   351.93   13.62
2025-06-20T14:30:56   1778   Comcast, Detroit, MI   2304.69   349.62   13.60
2025-06-06T16:46:48   69076   Ripplefiber, Detroit, MI   2144.83   317.85   31.49
2025-06-06T16:45:58   6124   Merit Network, Inc, Southfield, MI   2204.34   347.66   26.89
2025-06-06T16:45:21   1778   Comcast, Detroit, MI   2279.34   352.00   13.46
2025-05-28T14:58:20   1778   Comcast, Detroit, MI   2311.01   339.32   13.22
2025-04-20T20:51:52   53778   123NET, Southfield, MI   1252.25   314.12   9.47
2025-04-20T20:47:59   1778   Comcast, Detroit, MI   2017.36   351.83   14

[pfry]

The difference is significant, but not on an obvious (bandwidth) boundary. A rather asymmetric service... DOCSIS? Comcast? I'd expect a (more) symmetric offering these days.

Not much to go on. Start by characterizing performance through the firewall (that is not clear here), then (as necessary) without it, watching for errors and/or packet loss.

[meyergru]

Did you actually try to keep the server ID fixed? Or try to use one of the fast Comcast servers now? Obviously, those were never used in 2026 for whatever reasons, thus the numbers are not comparable as such.

[nullspace]

This is what I have for history. In December I got really busy with work and it's only now started to cool off and I can start to look at some issues.

Yes this is on a comcast DOCSIS line.

The system is an Intel Atom C3758R with intel i226V .  Opensense is running as a VM in proxmox currently by itself.

The throughput has drastically changed. I tested directly on the modem with a laptop and I'm able to get around 800Mbps D / 300 Mbps U using fast.com or speedtest.net . Yes this is not the speed I use to get but the upload is really what I rely on. The throughput of opensense has taken a dive and I'm not even sure where to begin.

[nullspace]

I'm trying to test the LAN side with iperf and I'm not sure what I'm doing wrong here but I can't connect to the client on on opensense, firewall rule is in place to allow any lan source to talk to the firewall on any port

[Greg_E]

Do you have any IDS/IPS or other filtering installed? Possible one of those changed.

Do you have ZFS and snapshots before and after the December updates? You could roll back to a previous and see if the speeds go back up and work forward from that point.

Glad you did a direct test on your modem, Spectrum was horrible when we had it, more down and than when we finally cancelled. Dealing with T-Mobile Home internet now (5g) and it's OK enough.

[pfry]

[...]The throughput has drastically changed. I tested directly on the modem with a laptop and I'm able to get around 800Mbps D / 300 Mbps U using fast.com or speedtest.net . Yes this is not the speed I use to get but the upload is really what I rely on. The throughput of opensense has taken a dive and I'm not even sure where to begin.

Ugh. That doesn't narrow it down much, unfortunately. Could even be something like a modem firmware push. Or multiple things, just to be even more irritating. I'd try to get a line test (upstream of the modem, assuming you haven't already). In addition to any other troubleshooting. Oh, and this might be barking up the wrong tree, but I've had to deal with a few silent, erroneous service downgrades in the past. You might want to verify your service (as much as possible, from an administrative standpoint). (I worked for my provider, so I had internal visibility and contacts, and it was still a pain in the ass.)