NUC running OPNSense 25.1.11 drops connections when using VPN or d/loading files

[Old_Rager]

Hey Guys,

This thread is a continuation of this thread from about 4-5 months ago (https://forum.opnsense.org/index.php?topic=44356.msg231128#msg231128).

My setup is as follows:
- ISP connection is 750Mbps down and 100Mbps up.
- My ISP has given me an Adtran 422G GFast NTU to connect to the internet.
- I have a DreamQuest Pro N5105 NUC with 2 x 1 Gbps ethernet ports that utilises an Intel 3165NGw network interface card (NIC) as a bare metal OPNsense device running version 25.7.2 before providing access to the rest of my home network.  The NUC that runs OPNSense runs all firewall, DHCP, and DNS activities for my home network.

What happens is that generally (not all the time) whenever I connect to a VPN or download files, the connection to the internet drops.  The unfortunate thing is it generally drops a few times a day when my wife works from home which is fast becoming a problem for us.  I have noticed that I can still access the OPNSense system on the NUC but have no idea what I'd be doing in this regard.  Some people have said that I need to have a look at certain access logs to see what is going on, but seriously, I'd need very specific instructions on how to do this as technology is really starting get beyond me these days even taking into account that I was once an MSCP in MS Windows 3.1... 🤣

In the previous post, someone suggested that I ascertain which NIC my NUC was using as I might need a dedicated NIC driver to alleviate the problem.  I notice that there is a post RE: the Intel 3165NGw NIC on the OPNSense forums, but this was ~8 years ago now (https://forum.opnsense.org/index.php?topic=6452.0).  Additionally, this post made the following statement but am not sure what in fact it actually means - "But I think that only client mode will be supported by the driver! So you can't use this wifi card to offer an AP on OPNsense."

If anybody could help me get to the bottom of what is going on here, it would be very much appreciated indeed.  Why this keeps happening to my setup I don't know, but I hear of others using OPNSense having no problems whatsoever...  😭😞

[Greg_E]

If you were using a USB NIC, I'd say it was overheating. Not sure what is going on. Are you certain it isn't the Adtran device having a problem? Do you have a consumer router you can hook up to do some testing?

If all of that works, I'd be looking at a new mini PC with a different Intel NIC set. The 5105 processor should be fine for a lot of what you are doing, but going up to an n150 might be slightly faster and probably save a few watts of power.

If you decide to either reload the OPNsense OS, or move to a newer mini PC, make sure you save your config file to make it easier to get back up to running. Only thing you might need to edit would be the ethernet interfaces.

[cookiemonster]

Intel 3165NGw is a wireless card it seems, so how is this OPN device setup for WAN and LAN ? And it will be interesting to know if this NIC is in AP mode.

[Old_Rager]

To Greg:

RE: your question about the Adtran device having a problem.  Prior to incorporating the NUC into my home network, the Adtran NTU in conjunction with my ISP provided TP-Link AX-5400 WI-FI 6 modem/router worked seamlessly and without fault.  Before incorporating the NUC into the equation, I upgraded my router to a TP-Link Archer BE800 BE19000 Tri-Band Wi-Fi 7 Router which honestly, worked like a dream.  As my PCs have both 10 and 2.5 Gbps ethernet ports, this router worked very well.  It is because of the costs associated every year by running Nortons over my entire home network had blown out so badly, that I decided to incorporate the NUC running OPNSense into the home network, and ever since I did this, I have had no end of troubles.


To Cookie Monster:

I have checked the NUC's BIOS and all references to WIFI were all disabled (there was nothing WAP related anywhere in the BIOS).


To Any and Everybody:

Does anyone know how I can access pertinent logs via the VGA CLI as I am unable to access OPNSense from a browser when all network connectivity is lost?  What would also be nice is for someone also to tell me how I might copy these logs (assumed files) from the OPNSense CLI to a USB drive?  And lastly, to whom can I send these logs for them to possibly work out what is going on here?

Seriously, this problem is fast becoming a problem for us with my wife not being happy with the network going down as she is sometimes on phone shifts when working from home.  Anything people can provide here - even the simplest of things that might rectify what is going on, that would be greatly appreciated!!!  😉🙏

[meyergru]

I am quite sure that your hardware is sub-optimal:

The Dreamquest NUC seems to be a cheap N5105 machine. The cheapest ones make use of Realtek NICs instead of Intel, which you can verify by looking at the interface names (reX for Realtek vs. igbX for Intel). I have not found any documentation that says if one or the other is used in that machine - which rises only more concern that it might be Realtek.

The dedicated chinese firewall boxes at Aliexpress mostly use Intel NICs for good reasons. Your machine is more of a desktop type that was designed to run Windows. That is also the reason why it has a wireless adapter.

The wireless card is an Intel card, but wireless is flaky at best under FreeBSD and should not be trusted (see this, #7). If you have use that, you may be out of luck.

For Realtek NIC adapters, there are vendor drivers which may work better than the FreeBSD provided ones. Look in the forum to find how they can be installed, like here, #6.

If your NICs are indeed Intel types, follow the advice here, #23.

[Greg_E]

system --> firmware --> plugins and install the os-realtek-re plugin to help make the realtek stuff work better. My only experience with this is on one of my firewalls, the built in NIC is Realtek and I only use this for a "management" connection when all else fails. It seems to be solid after installing the plugin but I'm not using it for more than a few hours while setting up the device.

[cookiemonster]

I suspected Realtek from your previous thread https://forum.opnsense.org/index.php?topic=44356.msg231128#msg231128 but we never got to confirm it. So as to add to what's been said so far in order to have a view if it is, ssh to it or drop to a shell with VGA connected when it is working fine and do:

Code Select Expand

$ifconfig and paste the text results here in code brackets.

Does anyone know how I can access pertinent logs via the VGA CLI as I am unable to access OPNSense from a browser when all network connectivity is lost?  What would also be nice is for someone also to tell me how I might copy these logs (assumed files) from the OPNSense CLI to a USB drive? 

Code Select Expand

$sudo dmesg will give you your latest system logs. Note that in unix-like systems not all logs go to a single file. They live in /var/logs/ where you will find different logs for different services but dmesg shall be helpful to begin with.
To save them to a USB stick, you'll need to create a mount point, plug it in, mount the filesystem onto it and then you can copy them using the "cp" command. BUT the formatting of the usb stick needs to exist so the right "mount" incantation can be issued.
So start by issuing

Code Select Expand

$gpart show after plugging it in, so we can see the filesystem (if any) on it.

[Old_Rager]

Hey Guys,

Firstly, let me thank you all for the very helpful information that you've been able to provide me - especially those of Greg_E, Cookie Monster, and Meyegru!

Secondly, after reconnecting with the DreamQuest support team (or maybe it was you Cookie Monster?!?), they sent me an email yesterday advising me to run 'pciconf -lv | grep -A3 -B1 ethernet' from the OPNSense shell which would let me see the make and model of my ethernet NIC, which ended up being a Realtek NIC (right on the money here Meyergru) - more specifically an 'RTL8111/8168/8211/8411 PCI Express Gigabit Ethernet Controller'.  As such, the following video (https://www.youtube.com/watch?v=jeXQueTipH0) showed me how to install the Realtek 'os-realtek-re' driver (which was also covered by Greg_E here).  After doing this as well as ensuring that Interface Settings for Hardware CRC, TSO, and LRO were all set to disabled, I rebooted to the NUC and decided to fire up ExpressVPN on my main PC and tried downloading a large file (26GB), and currently, it has completed over 5GB of the file download.  Normally, even just firing up ExpressVPN would generally cause OPNSense to drop all network connections, but downloading any files with the VPN going would result in a 100% drop in network connections with me not even getting 200MB downloaded.

My wife is about to begin a WFH shift, so I guess we will get to see if this work around continues to work, but I must say, so far, so very good indeed.

You have no idea how grateful I am to everybody helping me get over this hurdle.  I can only hope that it continues to work as intended as this will mean I now have a long-term Nortons replacement that will serve us perfectly here at home both now and into the future.

Again Guys, thank you so very much indeed for all of your help!!!  😜🙏

[cookiemonster]

I am glad that you have made it better but keep this thought with you: Realtek NICs even with the vendor's driver, is a suboptimal device for a freebsd-based networking appliance. There will be a point when the stress on it will cause it to falter. It might not be a 26 GB download with someone doing video calls for work; it might be on the 30 GB download. Who knows.
Not to put a downer on it just reality of the hardware. When and if it comes to that as your usage might not trigger it, you'll need to consider a different NIC.

[Old_Rager]

If I have to replace the NUC, I will ensure next time that I ask the forum for examples of good bare metal OPNSense hardware options - ones that are proven to be very reliable even if they cost a bit more.  I guess I may have kind of got lucky this time via a workaround, but I don't want this to ever happen again!!!  🤞

[Jyling]

- ISP connection is 750Mbps down and 100Mbps up.

Sounds like cable. If your modem is Intel Puma 6 chipset (some say even 7 is affected), then its UDP throughput sucks ballz. Block QUICK (UDP port 443) from the LAN and see if it improves things.