Install Issues when adding interfaces

[Cognoquest]

OPNsense 22.7.11_1-amd64
FreeBSD 13.1-RELEASE-p5
ESXi 7.0 Update 3

Hello All,

I am evaluating OPNSense to replace my home Centos 7 Gateway. I have installed a Test OPNsense server. I have given it 2 cores hence 2 thread, 4 Gig of Memory and 10 Gig of disk space on zfs.

The good:
This is a pretty standard installation (for now) and it started with three interface WAN, LAN & Guest. The OPNsense gateway only sees Untagged frames at the interfaces. The setup uses ipv4, no VLAN configuration involved.  All appears to work as expected.

The bad:
I added an IoT interface and that is when my troubles started. Could only communicate with the IoT net via the Gateway. The LAN net was given full access but could not communicate via ICMP to the IoT net. The OPNSense gateway portal also became sluggish, took one minute to respond to many of my requests. Though the WAN and LAN network requests seem to communicate as expected. Now what makes things more confusing is the problem went away when I made a few changes and maybe more that I am not aware.

• I enabled the DHCPv4 service on the IoT interface
• Added a test Alma Linux server on the IoT interface network to request an IP from the above service that worked
• Added a DHCP Static Mappings to the above interface for the above test server.

The good:
The problems on the IoT interface went away and all seem to be functioning as expected. I equated the issue to my lack of expertise.

The bad:
I added two more interfaces: Mgt and DMZ. I am back with what I believe the same problems.

The hypothesis:
I can not be the first that have added interfaces with OPNsense. The problems that I have would have been flagged a long time ago. So what do I have that is different? The only thing that comes to mind as possibly being slightly unusual is I run a PPPoe interface for my WAN.

It seems to me that the routing to the new interfaces are broken when I add interfaces and I do not know how to verify this. I am going  to make a backup of this configuration(ESXi) and try do redo the same above steps as for the IoT interface... and see where that brings me? Thank you for reading about my woes...

P.

[Cognoquest]

Hello All,

In regards to the following:

I added two more interfaces: Mgt and DMZ. I am back with what I believe the same problems.

I applied the same steps as described in my original post that I believe fixes the access to the interfaces and that worked again for this scenario. Not the most elegant solution since I can not explain the why. Thank you for listening.

P.

[Patrick M. Hausen]

When you add interfaces to an OPNsense VM in ESXi there is a high probability they get reordered.

Compare the MAC addresses as seen inside OPNsense with what you think the assignments should be on the outside - interface --> port group.

Then reassign inside so it matches again.

[Cognoquest]

Hello pmhausen,

Thank you for the reply. Yes I am aware of the reordering issue. From my experience it is not unique to OPNsense distro's.

What is a first for me is that I was not able to do the reassignment as you suggested in OPNsense without hosing my PPPoe WAN access permanently and I tried multiple times.

I took a different approach to the problem. Did an installation with two interfaces WAN(PPPoe) & LAN. Got that working and after I added the extra interfaces, currently a total of seven. Every time I added an interface, I let OPNsense do its reordering and made the reordering changes on the ESXi side instead. But as I mentioned above that created new problems. Even if these problems seem to be resolved this has peaked my interest? hence this post.

Regards,
Philippe