Logging for "Connection Refused" Issues

[crionics]

Currently running OPNsense 25.7.3_7-amd64 on a Qotom miniPC (Intel Atom 8C / 8T, 16MB RAM, 1TB HDD, (5) 2.5G LAN ports).

Recently, when the router restarts or a device restarts / loses connection, there is a period of time where the device will not connect to the internet. I first installed OPNsense v25.1.2 on 02/28/2025, but I don't have a specific start time due to taking a long time to understand the symptoms. However, they did tart before I installed 25.7.

Note the following occurs whenever the router restarts (update or power outage), and it doesn't happen to all devices at the same time. There are times everything is connected, only one device having issues or multiple devices having issues.

I can also replicate this on my main PC. On device restart,
- Can ping router (192.168.1.1)
- Can ping local DNS (pihole - 192.168.1.XX)
- Can ping other local service IP's on LAN (192.168.1.XX)
- Cannot ping external domains
- Cannot ping 1.1.1.1

Connection to WebUI just states Connection Refused.

After a period of time (Could be 30 minutes or could be several hours), connections will be re-established.

Since I am still looking for the cause of these issues, my question is which logs should I be reviewing. I intentionally disconnected the ethernet cable on my desktop, and upon reconnection I downloaded the following logs for review, but none of which showed any message of blocking or even showing the connection attempts:
-   audit.log
-   boot.log
-   Configd.log
-   Filter.log
-   Lighttpd.log
-   System.log
-   Dhcpd.log
-   Suricata.log
-   Each additional log under Services

What's especially curious to me is audit.log obviously shows webgui logins and "action allowed" messages, but it has no indication of the blocked / refused connections that I am seeing. None of the connections are making it to the firewall logs (filter.log), and DHCP logs show ip address renewals are occurring.

Additionally, I am noticing this intermittently across vlans, but seems to mainly affect my trusted LAN network. I did an experiment once where I moved everything from LAN to a new trusted 192.168.5.0/24 network, and the same things occurred.

Again, just looking for information on where else I should be looking for logging that I may have missed while combing the documentation.

No forum or web searches seemed to be the same issue, and tracking down information specifically on the causes of "refused connections" seemed impossible. The pages is the standard Edge error page that states "192.168.1.1 refused to connect."

Thanks.

[pfry]

[...]After a period of time (Could be 30 minutes or could be several hours), connections will be re-established.[...]

That's a bit long to be an OPNsense issue. The first thing I'd do is check your client machines for correct ARP - specifically, not just ARP, but correct ARP. Especially if they run Windows or FreeBSD, which seem to be more vulnerable to ARP proxies than Linux. The Qotom boxes seem to have video out, so I'd hook up a display and keyboard and check it as well. If ARP checks out, you can test other connectivity - look for connections from your clients when you attempt to access the GUI, for instance. And go from there...

There's the "log_in_vain" sysctl - I use it to check servers behind my firewall and their own firewalls... sometimes.

[crionics]

ARP tables all appear to be correct.

I agree it may not be an opnsense issue, but if I am getting a refused connection by the router and can ping the router, I figured something should be logged somewhere.

[pfry]

[...]I am getting a refused connection[...]

To get that from OPNsense - particularly inconsistently - you'd have to have a pretty specific set of configuration elements (pf reject rule, or pass rule against a closed port with "net.inet.tcp.blackhole=0") and circumstance. It would be tough to do accidentally.

How about checking on the firewall console? Any connection to or through the firewall should be visible in pftop. It might be inconvenient to read if it's torn down rapidly (I don't know offhand if pftop would show it in closed/wait or similar); you could always check the logs when you get into the GUI (assuming rule logging is enabled).

Getting away from the browser, how about ssh access? (I don't like using browsers for diagnostics as their UIs are junk and http is often proxied at multiple levels.)