can not reach web UI from different subnet than LAN

[wodec]

I have been banging my head for hours over the following, but can't figure out where my problem could be.

I am running an opnsense (latest release) in a KVM hypervisor on linux.
I have it connected to a trunk port on a managed switch.

I've decided to configure the VLAN interfaces on the level of the hypervisor as follows:
- for every VLAN on the trunk a VLAN interface on the hypervisor
- for each of those VLAN interfaces a bridge has been created
- these bridges are then attached to my opnsense VM, who manages the traffic in between (basically opnsense isn't aware that underneath VLAN's are in use, it's handled on the hypervisor level).

Now I want to make the opnsense web ui accessible on one of those VLAN's.
I have the web ui listening on all interfaces.
I also have a firewall rule which allows traffic to port 443 on the firewall interface in that specific network segment.

I am now connected from my laptop to that same managed switch via a port which has that same vlan configured as an untagged port.

My hypervisor also has an IP on the bridge in that same VLAN.
I can successfully ssh from my laptop, so in that VLAN, to the hypervisor and login.
I can however NOT connect to the webui of opnsense in that same VLAN.

When I look in the firewall logs of opnsense, I see that the rule I configured for access to the firewall interface on port 443 from that specific VLAN/network segment is hit, it goes green and is a rule of type "pass".
So from the firewall rules, it seems as if I'm hitting the correct rule with a pass.

However, the Web ui is not loading and I can not access it.
I have no idea where the root cause of this problem could be, anybody here perhaps an idea?

[CJ]

Post a screenshot of your rules.

[wodec]

I think I'm already past that stage.

What I've tried until now:


One of the first things that I tested today: created a floating rule on top for all interfaces to allow access to port 443 on "this firewall". Same behaviour: I see that this rule is then hit before my network-specific rule for access to port 443 on the firewall, again in the green, rule hit with pass. However, same behaviour, no web interface loading.

The web ui only loads over the LAN-bridge, which is the only bridge not linked to a vlan interface. All other bridges are linked to a VLAN interface. The Lan-bridge was created initially during install and has the web ui access, without VLAN.

Maybe also interesting to mention: even already used the firewall-settings to disable packet filtering, basically disabling whole firewall: still no access to web ui from that specific network segment.

Also tried completely disabling firewalld on the hypervisor, in the assumption that this was blocking something, also no luck.

My instinct is telling me it's almost as if the vlan tagging is disappearing when the response packets are coming from the opnsense vm to my laptop, but I don't see how that's possible.

For the hypervisor itself it's working, I can SSH to it over that same bridge in that same VLAN.

It's just your normal eth0 -> eth0.100 -> bridge100 setup, so as I understand it, it's not necessary to do any further config on the hypervisor side to keep those VLAN tags.

The packets coming from the opnsense VM over that bridge will automatically be tagged with the correct bridge right?

What I am wondering at the moment; I just read this article:

https://computingpost.medium.com/create-linux-bridge-on-vlan-interface-in-debian-11-10-e5679e3894bd
This is basically what I did, times 7.

Especially this part I found interesting:

echo "net.ipv4.ip_forward=1" | sudo tee -a /etc/sysctl.conf
echo "net.ipv4.conf.all.arp_filter=0" | sudo tee -a /etc/sysctl.conf
echo "net.ipv4.conf.all.rp_filter=2" | sudo tee -a /etc/sysctl.conf

=> I didn't make these changes on my hypervisor, but I'm wondering if these could explain the behaviour I'm seeing.

Especially since these settings appear to be relevant for multihomed machines but also to determine on which interface to send replies...

I'm still wondering if the ip_forward is even necessary. But those other 2 settings I think I would need to check.

[CJ]

Please stop doing random things in order to make things work correctly.  It's very hard to help someone if they keep changing the state from a known config.

By adding a floating rule allowing access on 443 to all interfaces you've now opened up your UI to the internet.  At this point you need to reinstall OPNsense.

Once that's done, we can start from a known fresh install state, verify that your WAN and LAN are working correctly, and then add in the additional VLAN setups that you want.