Random clients denied access to Internet

[Callahan]

I'm having real issues with clients randomly being prevented from accessing the Internet. I'm forced to reboot the clients. This fixes the issue most of the time (sometimes multiple reboots are required) for anywhere between an hour and 1 day the I'm forced to rebott all my servers/clients to get them access to the Internet again.

Environment

• 2 Windows 2012 Domain Controllers each running DNS & DHCP.
• Windows/Mac clients.
• Clients are a mix of hard wired Ethernet connected devices as well as wifi devices (the problem affects both).
• DHCP served by a pair of load balanced Windows DHCP servers.
• DNS forwarder on the DNS servers is pointing to a Pihole running on docker and secondary forwarder is the OPNSense firewall.
• Default gateway is my OPNSense box running on an Intel NUC with additional NIC feed from the internal mini pci-e port. All has worked fine for about 8/9 months or more.
• Running the latest stable version of OPNSense

Problem
Random selection of clients being denied access to the Internet

I can log onto the client machine and it picks up an address from the DHCP server along with the correct DNS servers/default gateway and correct subnet mask. I get no indication that there is any issue with the connection. I then try to browse the Internet and can't resolve any pages. I can ping and resolve every other host on the internal network and other hosts on the same subnet can get to the Internent.

• Next, I try and ping google.com from the problem host. No reply. So I assume it's DNS and ping 8.8.8.8. No reply.
• Top rule in FW LAN rules is to allow all LAN traffic out

The only change I have made of any significance in the last month is that I have set up a VPN to IPVanish via OpenVPN set up on OPNSense that I use with an Alias list to route certain hosts over the IPVanish OpenVPN/OPNSense gateway. In order to ensure that OPNSense doesn't just push traffic down the default gateway when the VPN to IPVanish is down, I enabled:

Firewall/Settings/Advanced
Gateway Monitoring
Skip rules when gateway is down = disable

Additionally, I have a route-based VPN to my Azure estate that has not given me any issues since I set it up months ago.

Oddly, (and this might be the most telling), the web GUI of OPNSense during this down time is not accessable from the problem hosts, is unaccessable until you wait out the 5/10 mins for the connection to be suddenly be restored. I make no changes to the FW to make this happen during this time. If I restart OPNSense during the time the newly booted hosts can't get Internet access, it fixes the issue (until the next machine fires up and needs Internet access). All other machines that have a working connection can access both the Internet and the OPNSense web GUI.

So from what I've seen so far:

• It can't be firewall rules as the devices all sit on the same subnet
• It can't be general routing issues as its specific to machines while others on same LAN are fine.
• It can't be the connection to the Internet (suggesting modem issues) as the rest of the machines have active connections while the problem machines can't get out the Internet.
• I am monitoring the default gateway and the connection does not drop
• This isn't specific to wired or wifi connections as it happens to both.

I am at a loss to figure out why OPNSense is preventing access. I made the mode from pfsense on a whim and I'm thinking I might live to regret it.

Is there anywhere in the logs I should be looking that I haven't already checked?

If anyone has any idea as to what could be causing this, I'd really appreciate some pointers.

Thanks.