Okay I ended up removing 1 VLAN interface of server, change Firewall rules on OPNSense. Fiddling with various bridge_modes of docker vlan networks didn't fix lack of network isolation neither did adding IPtables rules. So simplified stuff on hindsight having server on LAN segment did little sense for only convenience of SMB. Case closed.