Adding new network interface from ESXI reorders everything!

[francisaugusto]

Hi,

I have four interfaces/network adapters on an opnsense running on ESXi/vCenter.

When I add a new network adapter to opnsense and reboot the VM, all my previous adapters (em0, em1, em2, em3) are assigned to different Mac addresses. So my previous wan is assigned to em0, and it is still so, but em0 points to the macaddress of the newly added card.

So the mac address my em0 ended with aa, and now with ad, which is the newly card.

This creates me a great deal of trouble, as I have to assign everything again, and I'm afraid of loosing my configurations.

Is there a way to add a new interface from ESXi without loosing my previous relations between the interfaces and the macaddresses?

[bartjsmit]

Change the MAC addresses of the vNIC's from automatic to manual and restore them to what they were before you added the new adapter.

Bart...

[francisaugusto]

Hi Bart,

Do you mean on the VM on the ESXI config? They are already manual.

What I found confusing is that, let's consider this assignments:


Before adding the new interface:
interface  mac  network   
em0         :aa   wan
em1         :ab   lan1

After adding the new interface on ESXi and rebooting, I loose access and get this:

interface    mac network
em0:       :ac    wan
em1:       :ab    lan1
em2:       :aa    lan2


So I end up loosing access to my opnsense and must be physically connected to restore. But if that's the only way to do it, how do I assign - from the console - em0 to :aa, for example? Is it possible? Or will I need to change my network (eg. WAN) to em2?

[bartjsmit]

If they are set to manual you can edit them. Shut down the VM and:

set em2 to :ad
set em0 to :aa
set em1 to :ab
set em2 to :ac (or leave it if you don't care about the sequence)

Bart...

[francisaugusto]

I see.
But isn't there a way to prevent OpnSense to change the interface name when adding another one? IOW, having em0 bound to mac address :aa instead of having to change the order of the adapters at ESXi...

[Supermule]

It shouldnt be a problem at all.

I can reboot a million times without ESXi or pfsense changing anything.

Its a problem... especially when youre using OPNsense in a datacenter....

[cmanzur]

@francisaugusto

I have the same problem. I have assigned manually the MAC in each VM interface. (ESXi - adapter type: E1000E)
Then I assigned the interfaces in OPNSENSE GUI, also writing the specific MAC address. It works.

When I reboot the VM, the configuration keeps exactly in ESXi but in opnsense the interfaces are switched randomly.

If I ssh into opnsense and ifconfig, I can see some stuff like this:

Code: [Select]

em1: ....
  ether 00:11:22:33:44:55
  hwaddr 11:00:22:33:44:66


[liv3010m]

Hi guys,

Just registered to let you know that I'm using https://github.com/eborisch/ethname by eborisch to solve this same problem. I'm running it on 2 SmartOS hypervisors with 4 PFsense VMs but with the same underlying problem applies: FreeBSD reorders NICs when VM hardware is gets changed.
Hope it works for you too.

Best,
Olivia

[franco]

# pkg install ethname

Has been there for a while now...


Cheers,
Franco

[liv3010m]

indeed franco, just giving a heads-up since no one mentioned it.

Olivia

[franco]

Hi Olivia,

Fair point, I just want to make sure the first follow up question is not "how to get it".


Thanks,
Franco

[marcdeb]

I registered just to let all know that it doesn't seem to work with ethname. I got things working correctly with 2 NICs, which persisted after reboots and shutdowns. But after adding the third NIC, OPNsense lost all previously made assignments, to the point that repairing the assignments worked for that moment but didn't surface the next reboot/shutdown. I gave up on the matter. Tried it using OPNsense 22.1.10_4. Still pretty weird that a (albeit virtual) router cannot have those MAC-addresses tightly attached to WAN and LAN to start with...?

Regarding OPNsense in ESXi: eventually I fixed it by manually editing the VMX after adding the third NIC and before starting the VM and made sure that the third NIC [1] is of exactly the same type as the other two (WAN and LAN) and [2] it comes AFTER the first two in terms of the PCI device enumeration. In my case, I had to use this line in VMX:

Code: [Select]

ethernet2.pciSlotNumber = "256"
for the third NIC, as the first NIC (ethernet0) has number 128 and the second NIC (ethernet1) has number 192.

YMMV.

I notified the author (https://github.com/eborisch/ethname/issues/2) but I guess it is not their problem, as he already explained that OPNsense != FreeBSD (see https://github.com/opnsense/tools/issues/202).

[EBorisch]

(author of ethname here)

I wasn't saying it's not my problem, per se, I said as I'm not an OPNsense user, I don't have the tools to decipher what you need. 

Some other users seem to have it working; does this fit your needs? https://forum.opnsense.org/index.php?topic=27023.msg145910#msg145910