How reliable is Firewall:Diagnostics:States for troubleshooting

Started by DaElephant, July 11, 2026, 03:22:51 PM

Previous topic - Next topic
so what are the next steps @franco ?
Prod: OPNsense 26.1.11_10-amd64 running on ESXi 6.7 U2 VM, 4Gbytes RAM, 2 x vCPU
frr OSPF + eBGP, IDS, AdGuard Home, mDNS proxy, OpenVPN, Kea DHCP

Pre-PROD LAB: OPNsense 26.7.2-amd64 running on ESXi 6.7 U2 VM, 4Gbytes RAM, 2 x vCPU
frr OSPF + eBGP, IDS, AdGuard Home, mDNS proxy, OpenVPN, Kea DHCP



Any testers?

# opnsense-update -zbr 26.7.2-label
# opnsense-patch https://github.com/opnsense/core/commit/a698c1d94a
(does not need a reboot)

The base should have been fixed in 26.7.2 already but there was a little mistake with the final cherry-pick if anyone is wondering: https://github.com/opnsense/src/commit/49721cdf35a


Cheers,
Franco

i tried and still replicated the issue on 26.7.2_2-amd64 yesterday, but i can apply the patch later today and do some testing...

as my OPNsense is already running 26.7.2_2-amd64, i only need to apply the patch, yes ?

# opnsense-patch https://github.com/opnsense/core/commit/a698c1d94a
Prod: OPNsense 26.1.11_10-amd64 running on ESXi 6.7 U2 VM, 4Gbytes RAM, 2 x vCPU
frr OSPF + eBGP, IDS, AdGuard Home, mDNS proxy, OpenVPN, Kea DHCP

Pre-PROD LAB: OPNsense 26.7.2-amd64 running on ESXi 6.7 U2 VM, 4Gbytes RAM, 2 x vCPU
frr OSPF + eBGP, IDS, AdGuard Home, mDNS proxy, OpenVPN, Kea DHCP

i successfully applied the patch as below, then rebooted, however the issue still persists, some sample screenshots as below. None of the below sessions match the anti-lockout rule, yet that's how they appear in the UI...and none of the sessions match the 'Block IPC8 rule' too...., yet that's how they appear in the UI..there are many more such samples that could be provided....

root@OPNsense_LAB:~ # opnsense-patch https://github.com/opnsense/core/commit/a698c1d94a
Fetched a698c1d94a via https://github.com/opnsense/core
Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|From a698c1d94a75d0f3389257bcd21278801bf4d0e7 Mon Sep 17 00:00:00 2001
|From: Ad Schellevis <ad@opnsense.org>
|Date: Thu, 13 Aug 2026 16:10:17 +0200
|Subject: [PATCH] Firewall: Diagnostics - use new rlabel from pfctl, closes
| https://github.com/opnsense/core/issues/10552
|
|since pftop uses numbers as well, we need to map these to labels too, which means mapping labels to numbers with `pfctl -vvPsr` inside the query_top() function.
|---
| src/opnsense/scripts/filter/lib/states.py    | 78 ++++++++------------
| src/opnsense/scripts/filter/list_rule_ids.py | 16 ++--
| src/opnsense/scripts/filter/list_states.py   |  2 +-
| 3 files changed, 39 insertions(+), 57 deletions(-)
|
|diff --git a/src/opnsense/scripts/filter/lib/states.py b/src/opnsense/scripts/filter/lib/states.py
|index 55f71249ec1..198e1174c12 100755
|--- a/src/opnsense/scripts/filter/lib/states.py
|+++ b/src/opnsense/scripts/filter/lib/states.py
--------------------------
Patching file opnsense/scripts/filter/lib/states.py using Plan A...
Hunk #1 succeeded at 1.
Hunk #2 succeeded at 24.
Hunk #3 succeeded at 67.
Hunk #4 succeeded at 122.
Hunk #5 succeeded at 227.
Hmm...  The next patch looks like a unified diff to me...
The text leading up to this was:
--------------------------
|diff --git a/src/opnsense/scripts/filter/list_rule_ids.py b/src/opnsense/scripts/filter/list_rule_ids.py
|index dc353952f1b..2840e3b4ecb 100755
|--- a/src/opnsense/scripts/filter/list_rule_ids.py
|+++ b/src/opnsense/scripts/filter/list_rule_ids.py
--------------------------
Patching file opnsense/scripts/filter/list_rule_ids.py using Plan A...
Hunk #1 succeeded at 1.
Hunk #2 succeeded at 31.
Hmm...  The next patch looks like a unified diff to me...
The text leading up to this was:
--------------------------
|diff --git a/src/opnsense/scripts/filter/list_states.py b/src/opnsense/scripts/filter/list_states.py
|index bf74722c22f..dc4a6c5e42c 100755
|--- a/src/opnsense/scripts/filter/list_states.py
|+++ b/src/opnsense/scripts/filter/list_states.py
--------------------------
Patching file opnsense/scripts/filter/list_states.py using Plan A...
Hunk #1 succeeded at 1.
done
All patches have been applied successfully.  Have a nice day.
root@OPNsense_LAB:~ #

Prod: OPNsense 26.1.11_10-amd64 running on ESXi 6.7 U2 VM, 4Gbytes RAM, 2 x vCPU
frr OSPF + eBGP, IDS, AdGuard Home, mDNS proxy, OpenVPN, Kea DHCP

Pre-PROD LAB: OPNsense 26.7.2-amd64 running on ESXi 6.7 U2 VM, 4Gbytes RAM, 2 x vCPU
frr OSPF + eBGP, IDS, AdGuard Home, mDNS proxy, OpenVPN, Kea DHCP

here is a 'sample' additional debug info ( when running with patch applied, and after reboot ) for the 'Block IPC8' rule session/state

This rule isn't even applied to the vmx3 interface, it is applied to the vmx2 interface, yet the session appears under this rule, as originating from vmx3 interface on rule 'Block IPC8', which is not applied to vmx3 interface, really messed up!

vmx2 and vmx3 are separate interfaces, not connected to each other, they are separate and completely mutually exclusive L2 domains entirely...handled by ESXi vswitch vlans.

mappings as below, from OPNsense point of view

vmx2 <> untagged native VLAN 1 <> 10.0.0.0/24
vmx3 <> untagged VLAN 4 <> 10.0.1.0/24

In addition, the the state/session does not match the 'Block IPC8' match criteria either.





root@OPNsense_LAB:~ # pfctl -vvPsr | grep -A4 @41
No ALTQ support in kernel
ALTQ related functions disabled
@41 block drop in quick on vmx2 inet from <IPC8:1> to any label "dadac13d-fc61-4e50-9ff3-4769740d87b1"
  [ Evaluations: 0         Packets: 0         Bytes: 0           States: 0     ]
  [ Source Nodes: 0      Limit: 0      NAT/RDR: 0      Route: 0      ]
  [ Inserted: uid 0 pid 0 State Creations: 0     ]
  [ Last Active Time: N/A ]
root@OPNsense_LAB:~ # pfctl -vvs states | grep -C2 rule.41
No ALTQ support in kernel
ALTQ related functions disabled
all udp 255.255.255.255:6667 <- 10.0.1.2:59239       NO_TRAFFIC:SINGLE
   age 00:21:12, expires in 00:00:25, 254:0 pkts, 58928:0 bytes, rule 41, rlabel c0b243c6-f1d4-471d-8dcf-62046c6215b0
   id: a8547e6a00000000 creatorid: 30335776
   origif: vmx3
root@OPNsense_LAB:~ #

A tcpdump shows the traffic is, as expected arriving on vmx3 interface only


root@OPNsense_LAB:~ # tcpdump -nevpi vmx2 host 10.0.1.2
tcpdump: listening on vmx2, link-type EN10MB (Ethernet), snapshot length 262144 bytes
^[[A^C
0 packets captured
827 packets received by filter
0 packets dropped by kernel
root@OPNsense_LAB:~ # tcpdump -nevpi vmx3 src host 10.0.1.2 and dst host 255.255.255.255
tcpdump: listening on vmx3, link-type EN10MB (Ethernet), snapshot length 262144 bytes
12:34:18.270377 3c:0b:59:df:a5:cc > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 246: (tos 0x0, ttl 255, id 54285, offset 0, flags [none], proto UDP (17), length 232)
    10.0.1.2.59239 > 255.255.255.255.6667: UDP, length 204
12:34:23.273712 3c:0b:59:df:a5:cc > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 246: (tos 0x0, ttl 255, id 54286, offset 0, flags [none], proto UDP (17), length 232)
    10.0.1.2.59239 > 255.255.255.255.6667: UDP, length 204
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel
root@OPNsense_LAB:~ # tcpdump -nevpi vmx2 src host 10.0.1.2 and dst host 255.255.255.255
tcpdump: listening on vmx2, link-type EN10MB (Ethernet), snapshot length 262144 bytes





Prod: OPNsense 26.1.11_10-amd64 running on ESXi 6.7 U2 VM, 4Gbytes RAM, 2 x vCPU
frr OSPF + eBGP, IDS, AdGuard Home, mDNS proxy, OpenVPN, Kea DHCP

Pre-PROD LAB: OPNsense 26.7.2-amd64 running on ESXi 6.7 U2 VM, 4Gbytes RAM, 2 x vCPU
frr OSPF + eBGP, IDS, AdGuard Home, mDNS proxy, OpenVPN, Kea DHCP


Quote from: franco on Today at 08:20:23 AM
Quote from: franco on August 13, 2026, 04:21:54 PM# opnsense-update -zbr 26.7.2-label
# opnsense-patch https://github.com/opnsense/core/commit/a698c1d94a
(does not need a reboot)

Please follow these instructions.

Already did, and the issues noted above all still persist.
Prod: OPNsense 26.1.11_10-amd64 running on ESXi 6.7 U2 VM, 4Gbytes RAM, 2 x vCPU
frr OSPF + eBGP, IDS, AdGuard Home, mDNS proxy, OpenVPN, Kea DHCP

Pre-PROD LAB: OPNsense 26.7.2-amd64 running on ESXi 6.7 U2 VM, 4Gbytes RAM, 2 x vCPU
frr OSPF + eBGP, IDS, AdGuard Home, mDNS proxy, OpenVPN, Kea DHCP

> as my OPNsense is already running 26.7.2_2-amd64, i only need to apply the patch, yes ?

vs.

> # opnsense-update -zbr 26.7.2-label

The answer is no.