Messages

Completely bypassed my bgw320-500, now everything works well. It was a problem with the gateway.

Completely disabled monitoring, still same problem persists.

Here is today, before and after restarting the modem.

Will look at the logs in the AT&T modem now. Also will try to selectively disable a few things I'm running on my network. Maybe AT&T doesn't like certain kinds of traffic.

Still same problem, with default gateway monitoring. Latency and loss increase exponentially over time. Gonna change the re-try timing to every 10 seconds and reboot gateway now...

Delays and packet losses are accumulating again, even after those changes.

By the way, very interesting pattern:

WAN_DHCP6 - *only* packet loss becomes apparent, not delay(!). With WAN_DHCP both packet loss and delay start to grow after some time. The ipv6 address for monitor IP I was using for WAN_DHCP6 was 2001:4860:4860::8888, also google's public DNS.

I will try one more thing: remove the monitor IPs from the configuration, and let OPNsense figure out a monitor IP by itself. Rebooting modem now...

- What IP is OPNsense selecting for the WAN gateway monitor? Is it directly on the ATT provided gateway, or is this an IP further upstream on their network?

It's not selecting anything, I chose this one: 8.8.4.4 (this is google DNS I believe)

- If it's an IP upstream, can you duplicate the higher response times when you manually ping the same IP from a client on your LAN?

yes

- In there set your data length to a small number, usually 3 to 5.
Ok, changed to 4

- change the monitor IP to a known stable WAN IP
Ok, changed to 8.8.8.8


Rebooting router and opnsense, and seeing what will happen now...

Replaced my ISP device, AT&T sent me a new modem, problem still persists. Delays are starting to grow again. Maybe it's really just the Ethernet cable between my router and modem??? That would be so weird. I'll try replacing that next. Maybe I'll also then try to look at the modem logs, but that will require disabling IP passthrough... ugh. Any other things I should try?

I chatted to AT&T support agent a few days ago, they said they fixed something on their end, but problem persists. I had to again manually restart my modem today. Might try to call the ISP again. But is there a way to figure out what's going on from OPNsense logs? Could my OPNsense router be misconfigured in some way? Thanks!

I don't believe this is linked to updating OPNsense. I updated on 2024-06-23 (to 24.1.9), and the problems started on 2024-07-02. Next time I updated OPNsense (to 24.1.10) was today.

Hi, I have some quality problems, my Quality chart looks like the attached. Also, my RTT and RTTd values start increasing over time. What solves it is restarting the ATT fiber modem. What is the root cause? How do I diagnose the issue? Thank you!

I changed three things:
- Switched my WAN to igb0 (previously it was re0)
- Switched my LAN to re0 (previously it was igb0)
- installed the os-realtek-re plugin
- reset my AT&T modem, and configured it again for passthrough

let's see if any of this helps...

Interesting. I also have igb0 and igb1. Should I just switch over to one of those?

Looking at Reporting->Health->System->States

Not sure what this means. At 17:46 internet connection was lost, at 18:43 I restarted it and it has been working well since then. What does this chart mean?


Here are logs from the time of the failure:

Code Select Expand


2024-04-09T17:48:00-05:00 Notice dhcp6c RTSOLD script - Sending SIGHUP to dhcp6c
2024-04-09T17:48:00-05:00 Notice dhclient dhclient-script: Reason FAIL on re0 executing
2024-04-09T17:47:58-05:00 Notice dhclient dhclient-script: New Routers (re0): 123.123.123.123
2024-04-09T17:47:57-05:00 Notice dhclient dhclient-script: New Routers (re0): 123.123.123.123
2024-04-09T17:47:57-05:00 Notice dhclient dhclient-script: New Broadcast Address (re0): 123.123.123.123
2024-04-09T17:47:57-05:00 Notice dhclient dhclient-script: New Subnet Mask (re0): 255.255.254.0
2024-04-09T17:47:57-05:00 Notice dhclient dhclient-script: New IP Address (re0): 123.123.123.123
2024-04-09T17:47:57-05:00 Notice dhclient dhclient-script: Reason TIMEOUT on re0 executing
2024-04-09T17:46:37-05:00 Notice kernel <6>re0: link state changed to UP
2024-04-09T17:46:33-05:00 Notice kernel <6>re0: link state changed to DOWN
2024-04-09T17:46:33-05:00 Notice kernel <6>re0: watchdog timeout
2024-04-09T17:46:28-05:00 Notice kernel <6>re0: link state changed to UP
2024-04-09T17:46:24-05:00 Notice kernel <6>re0: link state changed to DOWN
2024-04-09T17:46:24-05:00 Notice kernel <6>re0: watchdog timeout
2024-04-09T17:46:20-05:00 Error dhclient connection closed
2024-04-09T17:46:19-05:00 Notice kernel <6>re0: link state changed to UP
2024-04-09T17:46:16-05:00 Notice kernel <6>re0: link state changed to DOWN
2024-04-09T17:46:16-05:00 Notice kernel <6>re0: watchdog timeout
2024-04-09T17:46:09-05:00 Notice kernel <6>re0: link state changed to UP
2024-04-09T17:46:09-05:00 Error dhclient send_packet: No buffer space available
2024-04-09T17:46:05-05:00 Notice kernel <6>re0: link state changed to DOWN
2024-04-09T17:46:05-05:00 Notice kernel <6>re0: watchdog timeout
2024-04-09T17:46:05-05:00 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp (execute task : dhcpd_dhcp_configure(,inet6,[lan]))
2024-04-09T17:46:05-05:00 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp (,inet6,[lan])
2024-04-09T17:46:05-05:00 Notice opnsense /usr/local/etc/rc.linkup: DEVD: Ethernet detached event for wan(re0)


Normally, on the dashboard, in the Gateways section I have

Code Select Expand

WAN_DHCP6
1234::1234:1234:1234:1234
~ ~ ~ Online

and

Code Select Expand

WAN_DHCP
123.123.123.123
~ ~ ~ Online

with my real WAN ips.

Every few days internet completely stops working, and I just see

Code Select Expand

WAN_DHCP6 - ~ ~ ~ Online

in the gateways, meaning no WAN_DHCP and no ip address.

Rebooting the router fixes the problem for a few days. How do I start diagnosing and fixing this issue? Thank you!

I have AT&T fiber, not sure if that's relevant. Connection was stable for about a year, this issue started to manifest itself about two weeks ago.