I have 2 OPNsense that I'm trying to get talking to each other and failing.
FW1 has a lan of 172.17.0.1 (production)
FW2 has a lan of 172.30.0.1 (lab)
FW2 has a lan of 172.17.0.4 connected to the production network
I want to be able to move data between production and lab and I'm failing to understand how it wants me to do this.
Do I need to set up additional gateways on each end, and if I do, what happens to the main traffic on the production network?
Do I need to set up virtual peer networks on each side and make route or gateway across those peers?
I did some reading last night, but now I'm failing to get things working.
There are two aspects to this: routing and filtering.
For the former, "System: Routes: Status" will give you a view of your routing table. Your FW2 should have routes for both 172/8 LAN networks (as both are assigned to interfaces); FW1 likely does not have a route for 172.30.0.0/?, so you'll need to add one under "System: Routes: Configuration", pointing to 172.17.0.4. If you would like Internet access from/through FW2, you'll need a default route pointing to FW1 - you can configure that under "System: Gateways: Configuration".
For the latter, you can test with permissive filtering on FW2, if necessary. You can generally set up a reasonable set of rules and simply watch the live log ("Firewall: Log Files: Live View") to see what's happening and adjust your rules accordingly.
That should do it. Of course there are many options - you can get as wacky as you like. This is simply the minimum.
That's what I was thinking, just add a route on FW1 to be able to see FW2.
Adding a route to FW1 pointing at 172.17.0.4 broke all traffic through the 172.17.0.0 network. Had to run over to the other building with my laptop and connect to the management port to delete the route and reboot. I'll have to do more reading this weekend as it's almost time to go home.
I know this is possible, I just have to figure it out.
What are the current default gateways of each of the OPNsense firewalls and of the clients in the two different networks?
Quote from: Greg_E on August 15, 2025, 10:54:08 PM[...]
Adding a route to FW1 pointing at 172.17.0.4 [...]
Specifics? A route with destination 172.30.0.0/? gateway 172.17.0.4 should not affect traffic other than that destined for the destination subnet. Also, what mask are you using on your 172/8 LAN networks? I always recommend including the mask when referring to a network (as opposed to an individual address).
You could also post your routing tables.
So, you have hosts on prod 172.17.0/24 (assumed /24), they need to reach lab 172.30.0/24 (assumed /24), and I assume the hosts point to FW1 for DFG. So you somehow need to get traffic to FW2 prod lan iface, a route on FW1 will suffice, however, you just created the asymmetric routing pitfall that FW state tables don't like.
Add route on FW2, and use a hide-NAT on FW2 so the traffic flow is symmetric.
OR, place a one-leg router on the LAN network, then in your DHCP configuration make the DFG to be that router. Route the lab net to FW2, everything else to FW1.
OR, create a lab iface on FW1 and assign it ip in lab net (lab net local), but you'll need to use NAT so that return traffic comes back via FW1.
Lots of ways, what can you stomach? ;)
Quote from: BrandyWine on August 16, 2025, 12:46:53 AM[...]
however, you just created the asymmetric routing pitfall [...]
Ha! Good point, there's a potential (as in certain) state mismatch. I whiffed that completely. "Firewall: Rules: [interface]" -> "Advanced features" -> "State Type" = "none" on FW1 would fix it, but that's probably moving outside Greg_E's desired solution set.
It might be easier to use a dedicated link between the firewalls, so FW2 would no longer exist on the 172.17.0.0 subnet, eliminating the asymmetry.