Firewall seems to flip a coin and sometime pass or block

[myradon]

Hi,

I'm facing a weird issue with Firewall rules. It concerns traffic from LAN segment to host in IoT segment. I setup an allow rule. But the Firewall log I see traffic gets both blocked and passed. The pass entry shows the Description of Pass Rule but deny entry shows generic "Default deny / state violation rule".

Pass rule: Pass on LAN Interface a IPv4/TCP any from LAN Net to a host (in IOT segment) at port 9000. Firewall rules are like at blocks or it passes right? I've got a case of mwehhh let's flip a coin. I reckon that's why docker container running Portainer on host(-net) feels sluggish.

Here a snippet of my firewall plain log;

Code Select Expand

2025-10-13T17:13:43    Informational    filterlog     10,,,02f4bab031b57d1e30553ce08e0ec131,igc1_vlan130,match,block,in,4,0x0,,64,0,0,DF,6,tcp,1045,192.168.130.135,192.168.132.2,52266,9000,993,PA,3670106294:3670107287,2916009586,2048,,nop;nop;TS
2025-10-13T17:13:40    Informational    filterlog     10,,,02f4bab031b57d1e30553ce08e0ec131,igc1_vlan130,match,block,in,4,0x0,,64,0,0,DF,6,tcp,1045,192.168.130.135,192.168.132.2,52266,9000,993,PA,3670106294:3670107287,2916009586,2048,,nop;nop;TS
2025-10-13T17:13:37    Informational    filterlog     10,,,02f4bab031b57d1e30553ce08e0ec131,igc1_vlan130,match,block,in,4,0x0,,64,0,0,DF,6,tcp,1045,192.168.130.135,192.168.132.2,52266,9000,993,PA,3670106294:3670107287,2916009586,2048,,nop;nop;TS
2025-10-13T17:13:36    Informational    filterlog     115,,,7c0eac9da28d053d5496835ca6b1a5bc,igc1_vlan130,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,192.168.130.135,192.168.132.2,52281,9000,0,S,4177888861,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol
2025-10-13T17:13:36    Informational    filterlog     115,,,7c0eac9da28d053d5496835ca6b1a5bc,igc1_vlan130,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,192.168.130.135,192.168.132.2,52280,9000,0,S,3532974004,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol
2025-10-13T17:13:36    Informational    filterlog     115,,,7c0eac9da28d053d5496835ca6b1a5bc,igc1_vlan130,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,192.168.130.135,192.168.132.2,52279,9000,0,S,1485386341,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol
2025-10-13T17:13:35    Informational    filterlog     115,,,7c0eac9da28d053d5496835ca6b1a5bc,igc1_vlan130,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,192.168.130.135,192.168.132.2,52278,9000,0,S,1097104973,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol
2025-10-13T17:13:35    Informational    filterlog     115,,,7c0eac9da28d053d5496835ca6b1a5bc,igc1_vlan130,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,192.168.130.135,192.168.132.2,52277,9000,0,S,4212048164,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol
2025-10-13T17:13:33    Informational    filterlog     10,,,02f4bab031b57d1e30553ce08e0ec131,igc1_vlan130,match,block,in,4,0x0,,64,0,0,DF,6,tcp,52,192.168.130.135,192.168.132.2,52264,9000,0,A,,3169677142,2591,,nop;nop;TS
2025-10-13T17:13:33    Informational    filterlog     10,,,02f4bab031b57d1e30553ce08e0ec131,igc1_vlan130,match,block,in,4,0x0,,64,0,0,DF,6,tcp,52,192.168.130.135,192.168.132.2,52261,9000,0,A,,513305460,2048,,nop;nop;TS
2025-10-13T17:13:33    Informational    filterlog     10,,,02f4bab031b57d1e30553ce08e0ec131,igc1_vlan130,match,block,in,4,0x0,,64,0,0,DF,6,tcp,52,192.168.130.135,192.168.132.2,52263,9000,0,A,,2990894775,2071,,nop;nop;TS
2025-10-13T17:13:33    Informational    filterlog     10,,,02f4bab031b57d1e30553ce08e0ec131,igc1_vlan130,match,block,in,4,0x0,,64,0,0,DF,6,tcp,52,192.168.130.135,192.168.132.2,52265,9000,0,A,,808996891,2048,,nop;nop;TS
2025-10-13T17:13:33    Informational    filterlog     10,,,02f4bab031b57d1e30553ce08e0ec131,igc1_vlan130,match,block,in,4,0x0,,64,0,0,DF,6,tcp,52,192.168.130.135,192.168.132.2,52262,9000,0,A,,3823069798,2048,,nop;nop;TS
2025-10-13T17:13:33    Informational    filterlog     10,,,02f4bab031b57d1e30553ce08e0ec131,igc1_vlan130,match,block,in,4,0x0,,64,0,0,DF,6,tcp,52,192.168.130.135,192.168.132.2,52266,9000,0,A,,2916009586,2048,,nop;nop;TS

edit:
With some googling I found more info on block rule at "Firewall/Diagnostics/Statistics/rules"

Code Select Expand

@10 block drop in log inet all label "02f4bab031b57d1e30553ce08e0ec131"
17:
34:58 2025
evaluations: 5445
packets: 1207
bytes: 300083
states:0
inserted: uid 0 pid 76129
state_creations: 0
time: mon oct 13


I don't know by the way what rule_id 10 is reffering to. I can't open it (from Live View). Who can make sense of this weird behaviour?

[cookiemonster]

have you verified that you are not mixing tagged and untagged traffic? First thing I expect you to be asked here due to known strange behaviours

[myradon]

Untagged? OPNSense [igc1] interface is not assigned. OPNSense internal segments only have VLANs.

Both pass and block entries have same VLAN (igc1_vlan130)  in above logging snippet. So I don't understand untagged traffic.

[Patrick M. Hausen]

Does that host in the IoT segment have a second interface?

[myradon]

Does that host in the IoT segment have a second interface?

Host machine has 3 VLANs defined. Docker container in question runs on one of the 2 defined macvlans. One host VLAN is purely for LAN segment. No docker containers running on these. So I had to do some Unbound custom zones for proper name resolving. But I believe it's out of scope for this issue.

[Patrick M. Hausen]

I suspect somehow the reply packets get routed directly via the LAN interface. Docker does not do proper network isolation. Use a packet trace to diagnose.

[myradon]

I did a Packet Capture both LAN and IOT vlans and opened dumps in Wireshark. It goes beyond my knowledge how to interpret this huge branches of data in packets. At some point Wireshark packets colored in red. I'm not comfortable with interpreting these pcap-files. I've put them on links down below;

Pack Capture Segment IOT with host running Docker Container

Pack Capture Segment LAN Network

EDIT:

I've also checked and tweaked routing setting for vlan isolation on Linux host machine running docker macvlans. It doesn't make any difference.

[cookiemonster]

Docker containers = I'm out. Sorry, can't help.