Trouble migrating to segmented network

[glubarnt]

Hi,

I am in the process of migrating from a flat network to a segmented network and have some problems with opnsense (or the network) which I do not understand.

The current state of the my network is:
one subnet (192.168.11.0/24) in one vlan (the default vlan on my switches).
This works.

The plan is to have multiple different vlans, e.g. for servers, clients, iot-devices, the usual.

I use OPNsense as a my router fand it runs as a VM on KVM (CentOS .

So, I tagged the necessary vlans on the needed ports, while the default vlan is still passed untagged, so that users do not notice anything while the network gets migrated.
I added bridges on top of vlan-interfaces on the KVM-host which are used by OPNsense. OPNsense should be the gateway in all the new networks.

Lets take two networks as an example:

vlan12: 192.168.12.0/24
vlan13: 192.168.13.0/24

OPNsense has an interface in each. vlan12 uses static-ip's for clients, vlan13 uses dhcp for clients. OPNsense always has the first ip in the network.
Just to make it clear: OPNsense does not really know about the vlans, they are handled by the KVM-host. OPNsense has just normal interfaces.

Now here comes my problem:
Clients on the same network can reach eachother, to different networks, I get nothing.
I initially thought I messed up the vlan tagging. But to test that, I tagged vlan12 to my workstation and gave myself an ip in that network and I can reach clients who have a ip in that network just fine. Note: my workstation of course still has an IP in the default network.

I am really scratching my head here, but I think I forgot something really obvious.
After I assigned the IP in OPNsense, I created basic firewallrules on the interfaces.

The rules are basically any->any->any rules.
Those are not for security. I just wanted to make sure the connection even works first, before implemeting actual rules.

Anyways, from the fw livelog, I can see that OPNsense does not block my packets. It seems like OPNsense does not route between the networks. But the routes seem to be right.

I cannot even ping between the networks. Clients inside the same vlan can ping each other just fine. Other services like ssh work fine too inside the same vlan.

Any help, idea or bump in the right direction is greatly appreciated.

[lfirewall1243]

But you see packets in live log?

Which virtualization are you using? Proxmox?

And make a network plan please

[glubarnt]

Yes, I see packets.

I use plain kvm for virtualization, running on CentOS 8.
Just libvirt, virsh and virtmanager.

I attached a small network plan.

So, if the interface for vlan12 on the workstation is active, I can ping the hypervisor and the dns server on that network. This is not visible in the fw log, as it does not pass the fw afaik.
If I take that interface down, I cannot ping them anymore. The icmp packets for this ping are visible in fw log. They are allowed, but they do not seem to reach the target.  I can still ping opnsense on the ip 192.168.12.1, but nothing else in vlan 12.

The test machine is just there to have a machine in vlan 13 to play around with. I can ping the gateway from test01, but no hosts in other networks.

[glubarnt]

I cannot even do a traceroute from one vlan to another. Tried that in the webui of OPNsense.
Just getting timeouts there.

To be sure: do I have to create any routes or gateways by hand for this to work?

I assigned the interfaces, gave them IPs and left the rest on default.
OPNsense created some routes automatically.