Show Posts
1
General Discussion / Packet passing stops on a bridge when I add a 3rd interface.
« on: July 16, 2020, 06:03:03 pm »
TLDR: Traffic passes (or is blocked) on a transparent bridge with 2 interfaces but completely breaks when I add a 3rd?
Background: I have a need for a transparent bridging firewall for a small number of low-traffic machines. Specifically a few elderly VMs running an EOL OS which, for various reasons, cannot be upgraded/reinstalled (server application will not work with newer OS/x64; application cannot be reinstalled due to vendor no longer existing, so cannot obtain either installation or license code). Trying to phase them out completely but in the meantime they need to be protected as they no longer receive security fixes. For further complications, they need to remain on their existing IP (or at least their existing DNS name, which cannot be made to resolve to an RFC1918 address), hence they need to be on the same network as they currently are. Therefore a bridging firewall between them and the rest of the network to very tightly constrain what on the outside is able to talk to the servers on the inside, limited to specific ports and IPs.
I have created an OpnSense VM on one of my VMware servers with 3 interfaces - one for management of OpnSense itself, and an 'inside' and 'outside' interface. The inside interface is connected to an internal VMWare vSwitch which test servers are attached to and this is working - the servers can make outbound connections, and approved external clients can talk to specific ports on specific servers, everything else is blocked - all on the same network subnet on both sides of the bridge.
To further, further complicate things, not all of the 'insecure' VMs are on the same physical host as the firewall VM, and with vMotion, could move between hosts. Therfore cannot be directly connected to that 'inside' interface. So, I created an 'insecure' port group on the hosts, carried on a VLAN. I added another interface to the OpnSense VM, in that portgroup (which should then tag/untag traffic leaving/entering at a VMware level). The test servers are then put in that port group which is common across all hosts.
As soon as I add the 3rd interface to the bridge, it all stops. The traffic that was already passing across the two interface bridge stops. I can see on the inside interface, the hosts are trying to send ARP requests to find the address they're attempting to talk to. I can see broadcast and discovery packets on the outside interfaces too but just nothing is getting between them.
My rules are all on the bridge itself, not on member interfaces, and the relevant tunables are set accordingly.
Any thoughts on what I'm doing wrong? Would it be better to add a trunk interface and create a VLAN interface from that trunk inside OpnSense, and add that VLAN to the bridge, rather than add the VLAN to VMware?
If all else fails, I will move all of the VMs to the same host and lock them there but I'd really rather not do that if I can possibly help it!
Thanks in advance,
Background: I have a need for a transparent bridging firewall for a small number of low-traffic machines. Specifically a few elderly VMs running an EOL OS which, for various reasons, cannot be upgraded/reinstalled (server application will not work with newer OS/x64; application cannot be reinstalled due to vendor no longer existing, so cannot obtain either installation or license code). Trying to phase them out completely but in the meantime they need to be protected as they no longer receive security fixes. For further complications, they need to remain on their existing IP (or at least their existing DNS name, which cannot be made to resolve to an RFC1918 address), hence they need to be on the same network as they currently are. Therefore a bridging firewall between them and the rest of the network to very tightly constrain what on the outside is able to talk to the servers on the inside, limited to specific ports and IPs.
I have created an OpnSense VM on one of my VMware servers with 3 interfaces - one for management of OpnSense itself, and an 'inside' and 'outside' interface. The inside interface is connected to an internal VMWare vSwitch which test servers are attached to and this is working - the servers can make outbound connections, and approved external clients can talk to specific ports on specific servers, everything else is blocked - all on the same network subnet on both sides of the bridge.
To further, further complicate things, not all of the 'insecure' VMs are on the same physical host as the firewall VM, and with vMotion, could move between hosts. Therfore cannot be directly connected to that 'inside' interface. So, I created an 'insecure' port group on the hosts, carried on a VLAN. I added another interface to the OpnSense VM, in that portgroup (which should then tag/untag traffic leaving/entering at a VMware level). The test servers are then put in that port group which is common across all hosts.
As soon as I add the 3rd interface to the bridge, it all stops. The traffic that was already passing across the two interface bridge stops. I can see on the inside interface, the hosts are trying to send ARP requests to find the address they're attempting to talk to. I can see broadcast and discovery packets on the outside interfaces too but just nothing is getting between them.
My rules are all on the bridge itself, not on member interfaces, and the relevant tunables are set accordingly.
Any thoughts on what I'm doing wrong? Would it be better to add a trunk interface and create a VLAN interface from that trunk inside OpnSense, and add that VLAN to the bridge, rather than add the VLAN to VMware?
If all else fails, I will move all of the VMs to the same host and lock them there but I'd really rather not do that if I can possibly help it!
Thanks in advance,