OPNsense Forum

English Forums => 25.7 Series => Topic started by: myradon on October 13, 2025, 05:30:27 PM

Title: Firewall seems to flip a coin and sometime pass or block
Post by: myradon on October 13, 2025, 05:30:27 PM
Hi,

I'm facing a weird issue with Firewall rules. It concerns traffic from LAN segment to host in IoT segment. I setup an allow rule. But the Firewall log I see traffic gets both blocked and passed. The pass entry shows the Description of Pass Rule but deny entry shows generic "Default deny / state violation rule".

Pass rule: Pass on LAN Interface a IPv4/TCP any from LAN Net to a host (in IOT segment) at port 9000. Firewall rules are like at blocks or it passes right? I've got a case of mwehhh let's flip a coin. I reckon that's why docker container running Portainer on host(-net) feels sluggish.

Here a snippet of my firewall plain log;
2025-10-13T17:13:43    Informational    filterlog     10,,,02f4bab031b57d1e30553ce08e0ec131,igc1_vlan130,match,block,in,4,0x0,,64,0,0,DF,6,tcp,1045,192.168.130.135,192.168.132.2,52266,9000,993,PA,3670106294:3670107287,2916009586,2048,,nop;nop;TS
2025-10-13T17:13:40    Informational    filterlog     10,,,02f4bab031b57d1e30553ce08e0ec131,igc1_vlan130,match,block,in,4,0x0,,64,0,0,DF,6,tcp,1045,192.168.130.135,192.168.132.2,52266,9000,993,PA,3670106294:3670107287,2916009586,2048,,nop;nop;TS
2025-10-13T17:13:37    Informational    filterlog     10,,,02f4bab031b57d1e30553ce08e0ec131,igc1_vlan130,match,block,in,4,0x0,,64,0,0,DF,6,tcp,1045,192.168.130.135,192.168.132.2,52266,9000,993,PA,3670106294:3670107287,2916009586,2048,,nop;nop;TS
2025-10-13T17:13:36    Informational    filterlog     115,,,7c0eac9da28d053d5496835ca6b1a5bc,igc1_vlan130,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,192.168.130.135,192.168.132.2,52281,9000,0,S,4177888861,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol
2025-10-13T17:13:36    Informational    filterlog     115,,,7c0eac9da28d053d5496835ca6b1a5bc,igc1_vlan130,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,192.168.130.135,192.168.132.2,52280,9000,0,S,3532974004,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol
2025-10-13T17:13:36    Informational    filterlog     115,,,7c0eac9da28d053d5496835ca6b1a5bc,igc1_vlan130,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,192.168.130.135,192.168.132.2,52279,9000,0,S,1485386341,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol
2025-10-13T17:13:35    Informational    filterlog     115,,,7c0eac9da28d053d5496835ca6b1a5bc,igc1_vlan130,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,192.168.130.135,192.168.132.2,52278,9000,0,S,1097104973,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol
2025-10-13T17:13:35    Informational    filterlog     115,,,7c0eac9da28d053d5496835ca6b1a5bc,igc1_vlan130,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,192.168.130.135,192.168.132.2,52277,9000,0,S,4212048164,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol
2025-10-13T17:13:33    Informational    filterlog     10,,,02f4bab031b57d1e30553ce08e0ec131,igc1_vlan130,match,block,in,4,0x0,,64,0,0,DF,6,tcp,52,192.168.130.135,192.168.132.2,52264,9000,0,A,,3169677142,2591,,nop;nop;TS
2025-10-13T17:13:33    Informational    filterlog     10,,,02f4bab031b57d1e30553ce08e0ec131,igc1_vlan130,match,block,in,4,0x0,,64,0,0,DF,6,tcp,52,192.168.130.135,192.168.132.2,52261,9000,0,A,,513305460,2048,,nop;nop;TS
2025-10-13T17:13:33    Informational    filterlog     10,,,02f4bab031b57d1e30553ce08e0ec131,igc1_vlan130,match,block,in,4,0x0,,64,0,0,DF,6,tcp,52,192.168.130.135,192.168.132.2,52263,9000,0,A,,2990894775,2071,,nop;nop;TS
2025-10-13T17:13:33    Informational    filterlog     10,,,02f4bab031b57d1e30553ce08e0ec131,igc1_vlan130,match,block,in,4,0x0,,64,0,0,DF,6,tcp,52,192.168.130.135,192.168.132.2,52265,9000,0,A,,808996891,2048,,nop;nop;TS
2025-10-13T17:13:33    Informational    filterlog     10,,,02f4bab031b57d1e30553ce08e0ec131,igc1_vlan130,match,block,in,4,0x0,,64,0,0,DF,6,tcp,52,192.168.130.135,192.168.132.2,52262,9000,0,A,,3823069798,2048,,nop;nop;TS
2025-10-13T17:13:33    Informational    filterlog     10,,,02f4bab031b57d1e30553ce08e0ec131,igc1_vlan130,match,block,in,4,0x0,,64,0,0,DF,6,tcp,52,192.168.130.135,192.168.132.2,52266,9000,0,A,,2916009586,2048,,nop;nop;TS

edit:
With some googling I found more info on block rule at "Firewall/Diagnostics/Statistics/rules"

@10 block drop in log inet all label "02f4bab031b57d1e30553ce08e0ec131"
17:
34:58 2025
evaluations: 5445
packets: 1207
bytes: 300083
states:0
inserted: uid 0 pid 76129
state_creations: 0
time: mon oct 13


I don't know by the way what rule_id 10 is reffering to. I can't open it (from Live View). Who can make sense of this weird behaviour?
Title: Re: Firewall seems to flip a coin and sometime pass or block
Post by: cookiemonster on October 13, 2025, 05:59:17 PM
have you verified that you are not mixing tagged and untagged traffic? First thing I expect you to be asked here due to known strange behaviours
Title: Re: Firewall seems to flip a coin and sometime pass or block
Post by: myradon on October 13, 2025, 06:23:46 PM
Untagged? OPNSense [igc1] interface is not assigned. OPNSense internal segments only have VLANs.

Both pass and block entries have same VLAN (igc1_vlan130)  in above logging snippet. So I don't understand untagged traffic.
Title: Re: Firewall seems to flip a coin and sometime pass or block
Post by: Patrick M. Hausen on October 13, 2025, 06:54:19 PM
Does that host in the IoT segment have a second interface?
Title: Re: Firewall seems to flip a coin and sometime pass or block
Post by: myradon on October 13, 2025, 07:15:48 PM
Quote from: Patrick M. Hausen on October 13, 2025, 06:54:19 PMDoes that host in the IoT segment have a second interface?

Host machine has 3 VLANs defined. Docker container in question runs on one of the 2 defined macvlans. One host VLAN is purely for LAN segment. No docker containers running on these. So I had to do some Unbound custom zones for proper name resolving. But I believe it's out of scope for this issue.
Title: Re: Firewall seems to flip a coin and sometime pass or block
Post by: Patrick M. Hausen on October 13, 2025, 07:18:27 PM
I suspect somehow the reply packets get routed directly via the LAN interface. Docker does not do proper network isolation. Use a packet trace to diagnose.
Title: Re: Firewall seems to flip a coin and sometime pass or block
Post by: myradon on October 13, 2025, 09:01:18 PM
I did a Packet Capture both LAN and IOT vlans and opened dumps in Wireshark. It goes beyond my knowledge how to interpret this huge branches of data in packets. At some point Wireshark packets colored in red. I'm not comfortable with interpreting these pcap-files. I've put them on links down below;

Pack Capture Segment IOT with host running Docker Container (http://www.myradon.nl/misc/igc1_vlan132.pcap)

Pack Capture Segment LAN Network (http://www.myradon.nl/misc/igc1_vlan130.pcap)

EDIT:

I've also checked and tweaked routing setting for vlan isolation on Linux host machine running docker macvlans. It doesn't make any difference.
Title: Re: Firewall seems to flip a coin and sometime pass or block
Post by: cookiemonster on October 14, 2025, 11:44:59 AM
Docker containers = I'm out. Sorry, can't help.