Issues with allowing traffic between interfaces "let out anything from firewall"

[tumma72]

Hi everyone,
I am relatively new to OPNSense and just started few weeks ago to implement different network scenarios for testing, first in my closed home environment, and then on more complicated setups. I am facing some issues in understanding how to properly allow or not allow specific traffic between interfaces. In particular, in one of the setups I am working on at the moment, I have a Protectli Vault FW4c with 4x 2.5Gbit/s NICs, and I want to use 2 of those ports to attach Wifi6 EAPs and maximize the traffic towards the internet. As most of the traffic goes out I have managed to create VLANs on OPT1, OPT2 and LAN and bridge them so that a device connecting to either of the EAPs or to specific ports on the network LAN will be tagged to follow specific rules. Now as each VLAN runs its own subnetwork, and the physical devices are connected to different ones, I would like to allow the management of all network related infrastructure devices, only from IPs (a list of) coming from the LAN interface.

This includes the following:
1. LAN: 192.168.142.0/24 with OPNSense on .1 and switch on .2 (VLAN switch)
2. OPT1: 192.168.143.0/24 with OPNSense on .1 and EAP on .2 (an SSID for each of the VLANs)
3. OPT3: 192.168.144.0/24 with OPNSense on .1 and EAP on .2 (same as above)

While ICMP are enabled between all interfaces, I would like to allow TCP traffic towards the network_infrastructure (a Firewall Alias with all of the 6 above IPs) only from the LAN net addresses. I have setup a simple rule to do so, and to my surprise the log shows that the traffic is allowed out of LAN towards OPT1 (this is an IN rule) but then when it reaches OPT1 the default OUT rule "let out anything from firewall" kicks in, and instead of allowing to connect to the local address (i.e.: 192.168.143.2 on port 443) is actually forwarding the request to the WAN port...

The rule I setup on LAN is the following:
IPv4 TCP   LAN net   *   network_infrastructure    22 (SSH)   *   *      Allow to connect to infrastructure devices via SSH

And network_infrastructure contains the following IPs: 192.168.142.1, 192.168.142.2, 192.168.143.1, 192.168.143.2, 192.168.144.1, 192.168.144.2

I have been trying to understand why OPT1 would think that a request incoming from LAN net towards an OPT1 net address would be considered an OUT traffic, but I have found no valid answer so far.

Any help would be greatly appreciated, thanks in advance

[chemlud]

only use OUT rules if you know what you are doing. normally IN rules work well.

[tumma72]

only use OUT rules if you know what you are doing. normally IN rules work well.

Thanks for the quick reply @chemlud I guess my message wasn't clear 😉 I am not trying to use OUT rules, it is the default created rule "let out anything from firewall itself" which kicks in, even if in my opinion shouldn't because the traffic is incoming into OPT1 from LAN and as the connection is initiated in the LAN interface, it should continue with the TCP handshake and provide a connection. Instead what I can see in the live view log is the following:

Interface      Time   Source   Destination   Proto   Label   
OPT1      2023-07-25T15:48:26   192.168.142.100:52745   192.168.143.2:443   tcp   let out anything from firewall host itself   
LAN      2023-07-25T15:48:26   192.168.142.100:52745   192.168.143.2:443   tcp   Allow to connect to infrastructure devices via HTTPS   

My understanding is that the rule on LAN is kicking in as expected, but then OPT1 instead of replying as expected seems to consider the traffic OUTgoing... which doesn't make sense as the address is on the same subnetwork of OPT1. If I check in the diagnostic this is what I get:

Proto
Source
Gateway
Destination
State
Age (sec)
Expires (sec)
Pkts
Bytes
Rule
tcp   192.168.142.100:52769       192.168.143.2:443   SYN_SENT:CLOSED   9   28   7.00   448.00   let out anything from firewall host itself
tcp   192.168.142.100:52769       192.168.143.2:443   CLOSED:SYN_SENT   9   28   7.00   448.00   Allow to connect to infrastructure devices via HTTPS
udp   192.168.143.2:42462       255.255.255.255:29810   NO_TRAFFIC:SINGLE   194   29   20.00   16.66 K   Allow all traffic towar

Which is strange as I wouldn't expect the SYN_SENT:CLOSED to happen...

Does this make sense to you?

[Patrick M. Hausen]

But traffic towards hosts on the OPT1 network is outgoing through the OPT1 interface. In and out are always in relation to the interface in question.

[tumma72]

But traffic towards hosts on the OPT1 network is outgoing through the OPT1 interface. In and out are always in relation to the interface in question.

Hey Patrick, thanks for your reply... ok I thought naively that OUT means out of the Firewall and IN means into the firewall, while it actually means IN and OUT of each interface. Then if that is the case the problem remains as the "let out anything from the firewall itself" is actually a very unrestrictive rule and it is higher in priority than the one I am going to write. Unless the "last match" vs "fist match" makes any difference, which it doesn't seem to at the moment.

If I want to restrict access to OPT1 only from LAN, but I need to allow an OUT rule from OPT1 towards LAN to allow the TCP connection to be established, wouldn't that expose the LAN (administrative network) to more attacks from OPT1?

I think I am more confused that I already was, thanks for opening more doors :-)

[Patrick M. Hausen]

OPNsense is stateful. If you have a rule on LAN:

Source: LAN net
Source port: *
Destination: OPT1 net
Destination port: HTTPS, SSH, whatever, or *
Action: allow

That is all that is necessary to establish a TCP connection. The first packet from LAN to OPT1 will setup a state that will permit bi-directional flow of packets belonging to the same TCP connection but nothing else.

I'd try a packet trace on the target system before blaming the firewall in this particular case.

[chemlud]

these "discussions" start here every other day or so. can we please have a topic pinned to the top "what is IN and OUT in firewall rules? and why does it matter"... just asking...

[Patrick M. Hausen]

@chemlud What puzzles me most is why people are so confused about this - this is how firewalls have worked since day 1. Livingston, Cisco IOS, Cisco PIX (before ASA), Checkpoint, Sidewinder, FreeBSD, Linux, ... all of them.

Are there consumer devices that treat this different? Where do people get these ideas?

[tumma72]

OPNsense is stateful. If you have a rule on LAN:

Source: LAN net
Source port: *
Destination: OPT1 net
Destination port: HTTPS, SSH, whatever, or *
Action: allow

That is all that is necessary to establish a TCP connection. The first packet from LAN to OPT1 will setup a state that will permit bi-directional flow of packets belonging to the same TCP connection but nothing else.

That is exactly what is in place, and the result that I am seeing is that the connection is initiated from the LAN address, but there is no return packet sent back from the OPT1 address. As shown in the log that I have posted above.

I didn't blame the firewall, I genuinely am trying to learn how to use it. I still don't understand why if the traffic is sent towards the address correctly (as I can ping the address, it connects to HTTPS if I plug in the network cable directly, and it sends a SYN request) and as you say that is all I need to do, then why there isn't a SYN_ACK sent back, what could the problem be?

Thanks

[tumma72]

I have tried also to define 2 different floating rules, with direction: any for both HTTPS and SSH ports, on the specific network_infrastructure IPs, with the same results. I have tried to connect from the LAN address to the OPNSense instance on the OPT1 network, and that works, but it triggers the no-lockout-rule after the one that I have created to allow traffic with source LAN net towards one of the defined IPs.

So it would appear that there is something else missing, or the second rules shouldn't be triggered, right?

[Patrick M. Hausen]

tcpdump on LAN, tcpdump on OPT1, tcpdump on both end devices ...

BTW what do you mean by "OPT1 address"? You are supposed to initiate the connection with the IP address of the device connected to OPT1, not the address of the OPT1 interface of your OPNsense ...

EDIT: just a shot in the dark - the "anti-lockout" rule does weird things with NAT for ports 443 and 22. I always disable it and simply make sure UI and SSH access is allowed where I want it. Try to disable that.

Firewall > Settings > Advanced >  Disable anti-lockout

[tumma72]

tcpdump on LAN, tcpdump on OPT1, tcpdump on both end devices ...

BTW what do you mean by "OPT1 address"? You are supposed to initiate the connection with the IP address of the device connected to OPT1, not the address of the OPT1 interface of your OPNsense ...

Yes, thanks for asking... I initiate the connection from 192.168.142.100 → 192.168.143.2:443 and it doesn't connect, it goes in timeout. I can't run tcpdump on the second one because it is a tp-link EAP and the address I am trying to reach is the admin interface of the device.

Even if I try using SSH, with a similar floating rule:

tcp   192.168.142.100:53274       192.168.143.2:22   SYN_SENT:CLOSED   7   28   6.00   384.00   let out anything from firewall host itself
tcp   192.168.142.100:53274       192.168.143.2:22   CLOSED:SYN_SENT   7   28   6.00   384.00   Allow SSH traffic from LAN to infrastructure devices

You can see I am connecting from an address in the LAN (192.168.142.0/24) to an address in the OPT1 network (192.168.143.0/24) and I get a SYN_SENT:CLOSED back...

Any other idea to try troubleshooting this?

[Patrick M. Hausen]

Disable anti-lockout for a test ...

[tumma72]

Disable anti-lockout for a test ...

Ok, I didn't lock myself out :-) And the connection to the IP address of the OPNsense on the network of the OPT1 interface works as expected. Therefore I guess it isn't a problem with the rule configuration but something else specific to the TP-Link management interface. To check that everything works there, I have connected directly to the AP using an ethernet cable, and I was able to login and verify that there isn't any management restriction related to the access via SSH or HTTPS. And still won't connect, any other idea?

[Patrick M. Hausen]

Does the TP-Link have a proper default gateway? I.e. the address of your OPNsense in the OPT1 network?

You could try to configure outbound NAT on OPT1 with the source limited to LAN net - that will make the connection appear local to the TP-Link device.