[SOLVED] Help configuring additional NIC as LAN -> Just use a bridge

[nickbreen]

I've added an additional NIC to use as another LAN interface. So now I have this structure:

Code Select Expand


WAN --- OPNSense +--- LAN1 (192.168.1.1/24) --- switch --- switch --- host1 (192.168.1.33/24)
                 +--- LAN2 (192.168.2.1/24) --- switch --- host2 (192.168.2.34/24)


Other than the default automatic FW rules I have configured an interface group called "LAN" that contains both LAN1 and LAN2. The only difference between the FW rules for LAN1 and LAN2 is the automatic anti-lockout rule on LAN1.

LAN has one in rule: IPv4+6 immediate allow any protocol from anywhere to anywhere.

Problem

I can IPv6 ping perfectly in all cases.
I can ping from all hosts to 8.8.8.8.
I can ping from all hosts to WAN IP.
I can ping between all hosts on 192.168.1.0/24.
I can ping between all hosts on 192.168.2.0/24.
I can ping from all hosts on 192.168.2.0/24 to all interfaces on OPNSense (i.e. WAN, LAN1, LAN2).
I can ping from all hosts on 192.168.1.0/24 to all hosts on 192.168.2.0/24.
I cannot ping from any hosts on 192.168.2.0/24 to any hosts on 192.168.1.0/24 (other than OPNSense).

I cannot see any ICMP packets blocked when enabling the default deny rule' logging.
I also cannot see any ICMP packets passed when enabling logging on my LAN group rule mentioned above.

TCP and UDP traffic also appears to be affected in the same way. I.e. I can SSH from 192.168.1.0/24 to 192.168.2.0/24 but not the other way.

Oddly, traceroutes to the WAN/LAN1/LAN2 IPs fail completely from any host on either .

Questions

Where should I be looking to diagnose this further?
What should I be looking for?

Hunches

Is there some default configuration somewhere that has been applied to LAN1 (as the original LAN interface when initially setting up the system, renamed to LAN1) that is not also applied to the (new) LAN2?

Is there some more complex NAT configuration needed given that IPv6 works perfectly?

[Patrick M. Hausen]

You possibly want to iplement this?

https://docs.opnsense.org/manual/how-tos/lan_bridge.html

[meyergru]

Hunches

Is there some default configuration somewhere that has been applied to LAN1 (as the original LAN interface when initially setting up the system, renamed to LAN1) that is not also applied to the (new) LAN2?

Is there some more complex NAT configuration needed given that IPv6 works perfectly?

There is a default rule on the first LAN that allows "LAN to any". You should replicate that for LAN2.

[nickbreen]

You possibly want to iplement this?

https://docs.opnsense.org/manual/how-tos/lan_bridge.html

Yeah, that was something I was considering.

All the docs scream about CPU usage and "it's probably not what you need" without really explaining why. So I thought I'd try for two networks.

[nickbreen]

Hunches

Is there some default configuration somewhere that has been applied to LAN1 (as the original LAN interface when initially setting up the system, renamed to LAN1) that is not also applied to the (new) LAN2?

Is there some more complex NAT configuration needed given that IPv6 works perfectly?

There is a default rule on the first LAN that allows "LAN to any". You should replicate that for LAN2.

The only NAT rule I can see is the automatic anti-lockout rule, specifically for TCP ports 22,80,443 so wouldn't apply to ICMP.

I'm sure that's already in place: as LAN2 can reach the public internet and also any interface on OPNSense. There's also an XBOX and TV on LAN2 that are operating happily (but just cannot see anything on LAN1).

Specifically, there is a rule for IPv4+6 any protocol any source any destination that applies to both LAN1 and LAN2. I replaced the wizard-generated rule with this group rule to be sure that the same rule is applied to both LAN1 and LAN2.

[Patrick M. Hausen]

There is no problem with the FreeBSD bridge for speeds up to 1 Gbit/s and a decent CPU. People keep repeating "facts" that were true five years ago but were fixed in 2020:

https://freebsdfoundation.org/blog/500-if_bridge-performance-improvement/

I run 100 servers with 1000 customer jails all bridged because that is the best option for jails. No negative performance impact in any way.

Kind regards,
Patrick

[nickbreen]

There is no problem with the FreeBSD bridge for speeds up to 1 Gbit/s and a decent CPU. People keep repeating "facts" that were true five years ago but were fixed in 2020:

That's what I was thinking was the case.

I am going to configure a bridge!

I have more PCIe slots available for a third card so I can always fight this fight again when I stick a WiFi card in there later.

[nickbreen]

I am going to configure a bridge!

Well that was ridiculously easy.

[nickbreen]

For future reference, bridges have IPv6 Link-Local Addresses disabled by default.

I needed to enable the bridge's Link-local address so that WAN's IPV6 prefix would propagate to all hosts downstream of the bridge.