realtek failures

[thoth]

I have a few OPNsense HA installs on very cheap hardware to test HA on my homelab  (some beelinks and other small cheap computers with at minimum two network ports).  At some point on all of them, some more often than others, they go null route or something peculiar.  If I log in to the machine directly with a keyboard and monitor the thing seems to be just fine but the networking is dead.  The one thing that seems to be common amongst them all is the realtek hardware. Plus I have not seen this on more expensive hardware, though I would be interested if anyone has seen these sort of issues on any other hardware.  To mitigate this I have made a small script that tries to ping out to 3 different ips and reboot the machine if all three fail three times in a row.

Code Select Expand

#!/usr/bin/env bash
ips='ip1 ip2 ip3'

test () {
  ping -t 5 -c 1 $1
  if [[ $? -eq 0 ]]; then
    echo good
    # any successful ping means networking is still working
    exit 0
  else
    ((++count))
  fi
}

count=0
while [[ $count -lt 3 ]]; do
  for i in $ips
  do
    test $i
  done
done

# only reboot if all pings fail
reboot


Ugly I know, I'd be happy to hear about more elegant solutions.  For now I paired this with an action:

Code Select Expand

# cat /usr/local/opnsense/service/conf/actions.d/actions_testnet.conf
[check]
command:/root/testnet.sh
parameters:
type:script
message:testing network
description:network test


And have that running as a cronjob every 5 minutes.

[thoth]

Any of you ever think to yourselves I am going to write this simple shell script to do one task, and then bazorkin skynet evolves

https://github.com/joshuacox/testnet

you know, its just a ping why would that be complicated....

[thoth]

To complicate matters even more, for some odd reason I thought I might virtualize the OPNsense instance on the very hardware it was crashing on?!?  Why would I do this?  This does not sound like a good idea.....

Well for some reason this works great.

1. Install debian bookworm
2. bridge the network connections
3. install kvm etc, install OPNsense as a VM
4. give the OPNsense VM any bridge you want, ensuring debian has at least created the bridge.

I have even given such a VM access to my direct fiber connection, in this case you'll want to assign static IP addresses to all the debian interfaces. 

Strangely debian seems to handle the realtek interfaces better than OPNsense and I have not had any failures so far.

[bimbar]

Realtek used to be terrible. They say it has gotten better. I'm not sure :D .

[thoth]

I think I can confirm that realtek is terrible at present under any BSD kernel.  Particularly from the beelink G35 series, as these do seem to be older hardware sold at discount prices (i.e. cheap).  But I have seen this behavior, under many different manufacturers many of the models are very new.   This also happens under PFsense (though I would never use their crap under production, I do test it), and generic FreeBSD.  However, when putting debian in charge of the hardware and then handing opnsense virtio networking everything seems to have stabilized.  I will note that it is best to make debian use static IP addresses.  If you want to avoid these complications overall I suggest finding hardware that does not have realtek networking hardware, intel hardware in particular seems to work better than anything else, I have heard from friends in the biz that they pay for all the patents of various things.  In particular, this is drastic amongst wifi hardware.  I have recently taken to replacing the wifi chips in any laptops that I can, and I have noticed dramatic improvements in not only general reception, but now I can connect in busy public wifi spots that I have never had decent reception before.  Here is an example of a modestly priced computer that runs OPNsense great and has intel hardware:   https://amzn.to/3PBh0wB

and here is an example of a replacement wifi chip for laptops:
https://amzn.to/44PPCze

TLDR realtek is terrible and should be avoided at all costs, otherwise put debian or some other linux on it and then give an opnsense VM kvm's virtio networking.