Issues with allowing traffic between interfaces "let out anything from firewall"

[tumma72]

Does the TP-Link have a proper default gateway? I.e. the address of your OPNsense in the OPT1 network?

Yes it has that configured manually as well as the IP address. I have set it manually to be sure it was correct.

You could try to configure outbound NAT on OPT1 with the source limited to LAN net - that will make the connection appear local to the TP-Link device.

I will try this, but so far it appears there is no restriction on the access via network, even though they suggest to use the Omada Controller interface, I have decided not to, as it will add more unnecessary complexity to the network management.

Thanks for your help :-)

[tumma72]

I am back to my original point now, after having experimented with multiple different setups. I will try to summarize again what is my intention, and what I am trying to achieve. I will also share what I have already tried, and I am looking forward to any suggestion that would come from the group:

Intent
I want to setup a network with different VLANs to isolate traffic, and I want to allow the management of the network infrastructure only to admins (only to a specific set of IPs → Firewall Alias: admin_ips).

Current Situation
I have setup the following VLANs which are successfully connecting to the internet and do not have access to each other:

• UNET: user network, going from access points to port OPT1 and OPT2 directly on the firewall/router, as well as specifically tagged ports on the main switch connected to the firewall/router LAN port. Traffic from the UNET to the internet works, and I have also enabled traffic within the UNET so that devices can connect to each other and use different services. I will restrict this further in the future, once the rest works.
• GUEST: guests network, only Wifi and only towards the internet.
• SECD: secure devices such as cameras, motion detectors, doors and windows sensors, isolated only towards the internet.
• MEDIA: all smart TV and players, which are accessible from the UNET only.
• HAUTO: home automation and IoT all small quirky and dangerous things, which are also accessible only from UNET and can only go to the internet.

I didn't want to setup a dedicated management VLAN because it is quite overwhelming for only a couple of IPs to manage, and because I have the access points attached to the firewall directly I had to setup VLAN bridges between OPT1, OPT2 and LAN specific VLANs, and assign different subnetworks with DHCPs (both 4 and 6 in some networks).

Open Problem
I am still not able to access from a specific IPs set (admin_ips alias) network infrastructure devices, from different subnetworks. If the devices are in the same subnetwork of the admin_ip(s) then it is reachable, otherwise it isn't. The behavior is pretty much what I have captured below which shows the "live view" first and then the "diagnostic view" after:

LIVE VIEW:
Interface      Time   Source   Destination   Proto   Label   
OPT1   2023-07-25T15:48:26   192.168.142.100:52745   192.168.143.2:443   tcp   let out anything from firewall host itself   
LAN      2023-07-25T15:48:26   192.168.142.100:52745   192.168.143.2:443   tcp   Allow to connect to infrastructure devices via HTTPS   

DIAGNOSTIC VIEW:
tcp   192.168.142.100:52769       192.168.143.2:443   SYN_SENT:CLOSED   9   28   7.00   448.00   let out anything from firewall host itself
tcp   192.168.142.100:52769       192.168.143.2:443   CLOSED:SYN_SENT   9   28   7.00   448.00   Allow to connect to infrastructure devices via HTTPS

Which is strange as I wouldn't expect the SYN_SENT:CLOSED to happen... what I have tried so far:

• Created floating rules for HTTPS and SSH with source admin_ips to allow incoming on both protocols (which is the "allow to connect from infrastructure devices via HTTPS" above.
• Created specific interface/IP rules to allow IN connection from the admin_ips with the same result as above.

At this point I am a bit puzzled, as the log shows that the rule is triggered and the traffic reaches the second subnetwork (VLAN) but then doesn't come back. I have checked that all network have configured - both via DHCP and on the devices themselves - the right gateway address as their own network IP address of the OPNsense firewall. There aren't any competing rules in place, I have actually removed everything that was there to test this specific behavior, and I am still not getting anywhere.

This is what doesn't work:
a. I am connected with an admin_ip in the 192.168.148.0/24 network, and I am trying to connect to a switch which is on 192.168.148.0/24 network both on port HTTPS and SSH, but the traffic doesn't go through. I am able to ping the device though, which means that ICMP traffic is going through, and I have a rule for that which states that all ICMP requests from admin_ips are allowed in, and then the allow everything out of the firewall allows for the response to come back.
b. I am not sure how to configure the FailoverWAN properly, meaning that I have followed the guide, and I have a working FailoverWAN with a fiber connection and a backup LTE connection, that works, but I had to add a rule to all subnetworks to use the FailoverWAN (which is a gateway group) to access the internet. Without that rule the connection towards the internet won't work.

Any help would be greatly appreciated :-) Thanks in advance!

[tumma72]

I suppose I am writing to myself for the next one having similar issues...

But as I make progress I am adding information in the hope that will be helpful.

a. I am connected with an admin_ip in the 192.168.148.0/24 network, and I am trying to connect to a switch which is on 192.168.148.0/24 network both on port HTTPS and SSH, but the traffic doesn't go through. I am able to ping the device though, which means that ICMP traffic is going through, and I have a rule for that which states that all ICMP requests from admin_ips are allowed in, and then the allow everything out of the firewall allows for the response to come back.

Solution
All TP-Link smart devices have an explicit setting for the route (other than the gateway) and have an explicit option for L3 connection which basically means they are routing to another subnetwork and allow devices coming from different subnetworks to connect. I have enabled the L3 options on both Access Points (EAP670) and the switch (TL-SG2008P) and I am now able to access from other networks. The reason why I didn't see this before is because it isn't in the Management or Security part of the interface but rather hidden on 3rd level menus...

Anyhow I had to add a VLAN ID: 1 to be able to manage all devices, because they are configured in such a way that you can't remove it, but I found a rather secure way to enter the management VLAN using a tagged port on the switch, that by default assigns the user network VLAN tag, while at the same time allows the already tagged VLAN 1 traffic to pass through. To create the VLAN for management spanning on all networks I had to create one VLAN connection with the tag 1 for each of the physical interfaces to which there are infrastructure devices connected and then I had to create a VLAN Bridge interface to include them all, and finally enable DHCP on that bridge. It seems to be a bit cumbersome but it works so far. I prefer to have DHCP running and then assign static leases rather than having static IPs, it makes the management easier and I do not have to access all of the devices individually to change IPs in case it is necessary.

Now the next challenge... making Apple devices and HomeKit work while being on different subnetworks... I will check around if there is already documentation about this, or I will start another post.

Thanks to everyone who contributed to help out...