Messages

Ok - Have some testing to do but believe may have got it.  VM VMX edit of pcislotnumbers.

Following this link gave me hint where to begin.  https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=198406

If you are adding more than 8 interfaces the mapping should look like this
PciSlot# --------------------Interfaces ----------Hex values for Pcislot#
160 1184 2208   ===>   vmx0   vmx1   vmx2   ===>   A0   4A0   8A0
192 1216 2240 ===>   vmx3   vmx4   vmx5   ===>   C0   4C0   8C0
224 1248 2272 ===>   vmx6   vmx7   vmx8   ===>   E0   4E0   8E0
256 1280 2304   ===>   vmx9   vmx10   vmx11   ===>   100   400   900

However after much trial and error I discovered that scsi0.pciSlotNumber = "160" was interfering with suggested address sequence.  VMX were still outta order...

Reviewing scsi0.pciSlotNumber I found other values for =161 which is meant for 2nd SCSI controller on VM.  Tried it but power on overwrote it to new -  scsi0.pciSlotNumber = "1280"

Have more testing and reading to do but this VMX config has all 10 VMXNET3 in proper order + test VM did boot with modified SCSI.

ESXi 7
VM Version = 15
Guest OS = FreeBSD 12 or later versions (64-bit)

scsi0.pciSlotNumber = "1280"
ethernet0.pciSlotNumber = "160"
ethernet1.pciSlotNumber = "1184"
ethernet2.pciSlotNumber = "2208"
ethernet3.pciSlotNumber = "192"
ethernet4.pciSlotNumber = "1216"
ethernet5.pciSlotNumber = "2240"
ethernet6.pciSlotNumber = "224"
ethernet7.pciSlotNumber = "1248"
ethernet8.pciSlotNumber = "2272"
ethernet9.pciSlotNumber = "256"

I read enough to be dangerous - try at own risk.

Making changes to environment for WIFI client bridge / Cell phone hotspot failover.

Adding additional VMXNET3 nics on existing VM throws entire VMXNET3 to VMX# out of whack.  Breaks entire FW.  One needs to re-assign going through manual pairing referencing MAC on VM to VMX MAC shown in Opnsense.  Assign port groups appropriately.  It works but kinda dirty - especially if not documented for the next guy.

So - my approach in past 20.1 has been OPNsense must just always have all 10 NICs added - so if future changes are required, then VMXNET3 NICs already present, thus no re-ordering on next boot.  I just use NULL Vmware Portgroup with no uplinks + not connected for unused NICs.

Just tried a a fresh 20.7 install with all 10 VMXNET3 from the start.  Same behavior of random ordering.
Reading up a little bit - appears to be an ongoing issue with more than 4 VMXNET3 that throws ordering out of whack.

Anybody have a trick that I can manually adjust for a 1:1 assignment.  I want Network adapter 1 in VM to = VMX0 , Network adapter 2 in VM = VMX1 , ect.

Thanks

I too am waiting for netmap patch to upgrade a particular site that cannot afford to have potential issues.

However I did upgrade my own few weeks back.  ESXi7 hosts.  If you like to try.
Full VM backup + snapshot.
Disable Sensei from auto boot.  Disable Suricata entirely.
Upgrade VM via ISO.  Update Opnsense / plugins post upgrade.  Reboot if necessary.
Run "opnsense-update -kr 20.7.3-netmap" as stated here - https://www.sunnyvalley.io/post/opnsense-kernel-netmap-status/
Reboot.  Upgrade Sensei.  Start sensei leaving auto boot off.  Test - if VM goes into kernel panic / reboots.
If good can enable sensei auto boot & try enabling Suricata and test.

I will note that I also migrated my POST upgrade 20.7 config to a fresh 20.7 VM with large vdisk.  Not sure if it was the config migration or the 20.7 update itself but my GEOIP Aliases WAN rules did not work.  I simply had to re-create them and re-apply to existing rules.  Also had to re-disable hardware offloads.  Besides those - using the modified Netmap kernel w/ Sensei in native mode has been stable w/o crashes.  Also have Suricata in inspect hyperscan.

Share few things I'm noticing. 

The GEOIP aliases do appear to have a glitch.  I'm using them via few inbound rules.  Had traffic getting blocked.  Tried updating / applying rules, moving rules order for refresh, re-running apply on GEOIP.
Quick search landed me here:  https://forum.opnsense.org/index.php?topic=18350.0
Which suggested to just create new alias / reapply to rule.  Now working.

Also share that Sensei 1.6 restore doesn't appear to restore license info properly.  Had to reapply key - This maybe excepted behavior as it was a new Opnsense.  Regardless - after re-activation all my config / policies / whitelist intact.

Just to clarify I did not do an import of config install to 20.7.2.  I upgraded my old Opnsense to 20.7.2  (+ 20.7.2-netmap) + , got  backup, and restore to new VM 20.7.2  (+ 20.7.2-netmap).

I'll explore more and share any other findings.  TTFN

Having played / used Opnsense for ~4 months I just stopping by to share how awesome I think it is...
Coming from having an environment with Cisco FMC / FTD is night and day.  The irony.

Recently I did upgrade from 20.1.9 to to 20.7.2.  Using VMware I do have vmx interfaces and did run into known netmap issue with both Sensei + Suricata.  Following https://www.sunnyvalley.io/post/opnsense-kernel-netmap-status I was able to fix.  (running:  opnsense-update -kr 20.7.2-netmap)

When I first tested it out I was foolish and only gave opnsense a 20GB disk.  So part of this upgrade to 20.7 was also to migrate to a fresh install.  This is what got me even happier with opnsense today to make this post.  Doing a full backup / restore to new VM with 60GB disk worked flawlessly.  Everything restored except for disable hardware offloads from what I can tell.  And I am using quite few plugins.  Even the mismatching Vmware interfaces restored just in the order I wanted. Just pure awesome.

Also briefly share that on my lab environment which is 2x Dell PER710s with Dual X5650s running vmware 2 node VSAN cluster the Vmotion / failover also works flawlessly.  Quite pleased with Opnsense.

Thank you Devs!