Hello everyone,
I followed the instructions here the create a LAN bridge: https://docs.opnsense.org/manual/how-tos/lan_bridge.html (https://docs.opnsense.org/manual/how-tos/lan_bridge.html). I made sure to set the tunables as instructed and to be sure did the reboot. LAN is assigned the newly created bridge0 interface and bridge0 consists of OPT0,OPT1 and OPT2, while WAN is assigned to the fourths interface.
My computer is connected to OPT0, another computer connected to OPT1. I can send a ping from my computer on OPT0 to the computer on OPT1, but I can not use SSH or access a webinterface running on port 80 and port 443. I have checked that all my firewall rules refer to and are applied to the LAN interface, not and of the OPTx interfaces.
What am I doing wrong here, is there anything else I need to pay attention too? Thank you.
For the software stack:
OPNsense 23.1.5_4-amd64
FreeBSD 13.1-RELEASE-p7
OpenSSL 1.1.1t 7 Feb 2023
If that's the only problematic port(s), then maybe needs changing the OPN UI ports it runs on and tweaking interface, etc.
I recommend trying to see what the traffic is doing in a little more detail.
- Check arp -a on source and dest host immediately after trying to ping to see if ARP resolution is happening
- Check arp -a on the firewall to confirm it sees both hosts
- Run a packet dump on incoming and outgoing interface on the firewall and/or hosts during ping to see where the packets are stopping
I initiated a ping between both computers and run apr -a after that on both.
On computer 1:
_gateway (192.168.1.1) at xx:xx:xx:xx:xx:xx [ether] on enp37s0f0
? (192.168.1.160) at xx:xx:xx:xx:xx:xx [ether] on enp37s0f0
On computer 2:
_gateway (192.168.1.1) at xx:xx:xx:xx:xx:xx [ether] on enp0s13f0u4u2
? (192.168.1.220) at xx:xx:xx:xx:xx:xx [ether] on enp0s13f0u4u2
While computer 1, which is connected to OPT0, can go into the internet, computer 2 can not. Computer 2 can connect to the firewalls web UI tough.
Quote from: n-dolce on April 19, 2023, 05:55:24 PM
While computer 1, which is connected to OPT0, can go into the internet, computer 2 can not. Computer 2 can connect to the firewalls web UI tough.
This makes me think the outbound NAT configuration could be wrong. Can you attach a screen shot of the outbound NAT settings?
Quote from: clarknova on April 19, 2023, 06:10:25 PM
Quote from: n-dolce on April 19, 2023, 05:55:24 PM
While computer 1, which is connected to OPT0, can go into the internet, computer 2 can not. Computer 2 can connect to the firewalls web UI tough.
This makes me think the outbound NAT configuration could be wrong. Can you attach a screen shot of the outbound NAT settings?
It would explain why computer 2 is not able to access the internet, but it would not explain why I can't SSH from computer 1 to computer 2. Okay so running iperf3 between the 2 computers works, SSH is still a no-go.
I attached the NAT configuration to this post.
I just managed to get SSH to also work. So the problem seems to be that the connection is breaking every now and then. When I run iperf3 I sometimes get 200 to 500 mbps and sometimes 0 mbps for a couple seconds. Hardware offloading is already deactivated as far as I can see, and computer 2 still cannot access the internet.
This sounds like it could be an MTU mismatch. The MTU on both computers must match, and the MTU on the firewall interfaces (and any L2 interfaces in the path) must be larger by some value that I don't recall at the moment. I usually just set switch ports to the largest possible value. You can do the same on the OPTx interfaces, but the bridge interface MTU must match that of the attached hosts.
I set the MTU to 1500 on all OPTx interfaces, the bridge does not seem to have an option for it. Still the same, sometimes up to 1Gbps then again nothing for a couple seconds.
I am unsure, but it seems something is wrong with the routing. Computer 2 says that I am connected from a 10.* address, that my VPNs uses for example, it seems I am not coming from a LAN address.
Under System -> Routes -> Status, this is the entry my SSH server on computer 2 says I am coming from when connecting via SSH
10.1.114.242 link#11 UHS NaN 16384 lo0 Loopback
I was able to fix the instability with connection loss. I was forcing 2.5Gbits speed and it seems the cable is not up to it. It seems it wend under my radar since my connections are mostly TCP.
Interesting. Thanks for posting back with the solution.
It is the solution for the connection loss at least, routing is still somehow broken.
Computer 1 IP is 192.168.1.220, computer 2 IP is 192.168.1.160. When I SSH from computer 1 to computer 2, computer 2 still says I am connecting from 10.1.114.242. Computer 2 still has no internet connectivity.
You need to run Wireshark on both hosts and do packet captures on OPNsense. Something isn't working and we could keep guessing, but nothing cuts to the truth faster than actually watching where the packets are going.