Interface assignment occasionally being assigned to a different port upon reboot

[Apoc]

I have a strange problem with my OPNsense instance that I've not been able to deliberately reproduce. I've also searched the internet (and this forum) for similar issues, but I wasn't able to find anything conclusive.

I have OPNsense installed on a Qotom box with 4 SFP+ ports. Two ports are in use and therefore assigned interfaces in OPNsense. The first port works flawlessly all the time. The second port not so much.

Occasionally upon reboot OPNsense seems to switch the assignment of the second SFP+ port to the third SFP+ port. Since half of my network is attached to this port this causes major problems, as you can imagine. The only (temporary) fix is to plug the network cable into the other plug until it switches back again.

Even though after much trial-and-error and research I haven't been able to even reproduce when exactly this happens. I've found posts of people having their assignments switched around, but all of these cases did deploy OPNsense as a VM guest and they changed network interfaces being passed-through to OPNsense right before they encountered these problems. As everything is soldered to one mainboard I don't even have the option of changing something related to the network hardware, so that definitely can't be the cause.

Do you have any suggestions on how to troubleshoot this?

[patient0]

Prefix: totally new to me and I did some asking of the big 'ol internet.

The search 'freebsd change nic order' returned a FreeBSD thread "Re-numbering network interfaces" in which the author of a script mentioned that he/she/it created it to rename NIC at boot. Three years later (2020) it got included in FreeBSD.

If the MAC addresses stay the same then the package called 'ethname' may be of use (it is available on OPNsense to install). The repo is on github (for a config example):

FreeBSD rc.d script to map ethernet device names by MAC address.

On my test VM it did work, I created a file /etc/rc.conf.d/ethname to swapped two of the interfaces:

Code Select Expand

ethname_enable="YES"
ethname_vtnet0_mac="MAC of second card"
ethname_vtnet1_mac="MAC of first card"

After that the naming was as defined in the file.

If there can be a possible issue in OPNsense, no idea. Maybe @franco can comment on that.

Addition: I just saw that there is a thread here, Persistent NIC ordering/naming basedon MAC address(es). Recommend the same package but using rc.syshook (see thread for details and the commend from @franco)

[franco]

I would recommend using the lock setting in the interfaces if you want to prevent reassignment, but this won't fix the fact that on boot the port will sometimes not be seen by the system in your particular case which causes this issue and will keep breaking your connectivity. I suspect either the network driver or the SFP+ module on that particular port causing this.


Cheers,
Franco