LAGGs and Load

[CleBoatGuy]

I'm hoping someone can help me understand what's going on.  When there is more than 1 interface member of the LAG, and start to put even a little bit of traffic through the firewall (around 40Mb/s), ping times skyrocket, and then eventually all pings to the firewall timeout (around 60Mb/s) and the web interface is unreachable.  As soon as the traffic stops (<1Mb/s), ping times drop to a reasonable 1.5ms, and I can access the web interface again.

My LAN interface is set up as a LAG.  The above behavior only happens if is more than one member of the LAG.  I've tried both LACP and loadbalancing configs, and get the same behavior with both.
The switch is a Netgear GS324TP (I know, Netgear, no jokes, please), but it does support both LACP and static LAG configs.

I tested single interface LACP and Loadbalancing LAG settings for each of the physical interfaces individually, and can push +800Mb/s through without any issue.  It's only when I have multiple members of the LAG that issues arise with even the smallest amount of traffic.

Setup:

• OPNSesne: Qotom i5-7200U, 16GB RAM, 250GB SSD - all latest patches
• Switch: Netgear GS724TP with the latest firmware
• The LAG is a member of 2 VLANS (1, and 30).  All test traffic is using VLAN 1.

I've Fluke tested the CAT6 network cables, all good.
Any ideas?

[mimugmail]

Can you check on switch if both ports are full duplex and same speed?

[CleBoatGuy]

Yes, both switch ports are full duplex and 1000Mb/s.

I also verified LACP seems to be working properly setting up the LAGG. When I add a second physical interface to the LACP LAGG interface in OPNSense, it drops pings until I add the corresponding interface to the LACP LAG group on the switch. If I drop one of the interfaces on the switch, pings timeout, too. While this isn't proof that there isn't a broadcast storm, I think it it does prove that LACP needs to be happy on both sides.