Packet Loss over all VLAN's (22.7.2 and 22.1.6)

[c-mu]

I have a strange problem since Sunday.
Two hardware identical firewalls are working in HA mode with CARP. OPNsense 22.1.6. It works for years.

Since Sunday I have packet loss on all VLAN's. Both firewalls have problems to define the CARP master. As a first step I shut down the slave. The log then throws the same message across all CARP interfaces, the order is random (master firewall):

example:
carp: 60@ixl1_vlan141: BACKUP-> MASTER (master timed out)
carp: 60@ixl1_vlan141: MASTER -> INIT (hardware interface up)
carp: 60@ixl1_vlan141: INIT -> BACKUP (initialization complete)
carp: 60@ixi1_vlan141: BACKUP -> MASTER (master timed out)

and again: it is configured as master itself and its slave is offline. Base 3 and skew 0 is configured.

Next I updated the master to 22.7.2, because of a possible software bug. No success.
Then I replaced the Intel card with Mellanox. No success.
With the current version 22.7.2 the DHCP service also fails at some point and does not assign any more leases until the reboot.

Then I changed a VLAN CARP IP to the IP alias to exclude CARP - drops still follow.

Packet losses occur exclusively to the firewall, or across the VLANs. VLAN internally from host to host there are no losses.
It does'nt matter if the Master is the only one online, or the Slave, or both.
Does anyone have an idea what else I can do?

[c-mu]

I have found a defective switch component in my network. It was not easy, because 2 ports of an 8 port fiber optic module were defective and this was not visible in any log files. Here I had to use the exclusion method to approach the defect port by port.