NAT Reflection & LACP

[Dean E. Weimer]

I switch my server over from a single port to an LACP 3 port aggregation on the switch. I can talk to all the local IPs and ports just fine, but NAT Reflection isn't working at all. External IP to internal NAT appears to be hit or miss, however this could be a result of certain services talking to other services with NAT reflection.

The server is running FreeBSD the applications are all running from within Jails on the system. When I look at the firewalls ARP table through the interfaces diagnostic page it only shows the base systems ARP entry. However when I ssh to the internal IP of the firewall and list arp entries using arp -a it properly shows the individual jails ARP entries as well.

Looking at tcpdump on the server it appears to show the traffic coming from the firewall and being replied to but the reply never makes it to the client.

Has anyone seen this before?

I am running OPNsense 16.7.3-amd64

[Dean E. Weimer]

Nevermind, turns out it was jumbo frames not working on the LACP aggregate, Since the remote connections were coming in from INET without jumbo frames they talked just fine. Disabling Jumbo frames on the FreeBSD interfaces restored normal functionality. starting a new thread over on FreeBSD mailing list to determine proper LACP and Jumbo Frames configuration.