SSH and RDP connections dropping every 30s between VLANs

[albal]

So I was using pfSense but the upgrade to 2.5.1 broke (failed to detect partition) so I decided I would give OPNsense a try.

I have two VLANs for now 192.168.0.0/24 (VLAN1/LAN) and 10.0.10.0/10 (VLAN10/10G).  All my network devices are plugged into a Dell N1524 switch and the ips for the VLANs on this switch are set to 192.168.0.252 and 10.0.10.254 - gateway is set to 192.168.0.254

In OPNsense I have setup the interface for the LAN as 192.168.0.254 and for the 10G network as 10.0.10.252 - both have IPv4 Upstream Gateway set to Auto Detect. 

Traffic seems to pass freely from both networks to the internet but when I connect from my workstation on the LAN to the 10G network the connection drops after 30 to 32 seconds.  With SSH the connection drops and with RDP the connection freezes after around 30 seconds and then it eventually reconnects.

This setup was working with pfSense but I've heard good things about OPNsense so want to persevere.

I've attached the firewall rules I have for LAN and 10G.

[chemlud]

Hi!

Virtual or real install? Which interfaces on the sense? Anything in the logs? Hardware offloading disabled? What did you try to debug? ;-)

[albal]

I have a physical install on a Supermicro SYS-E300-9A with a Intel C3858.  I have 2 WANs connected with the LAN port connected to the Dell N1524.  I've not disabled anything from a default install.   I configured my VLANs and DHCP and setup some port forwards which all seem to be working fine.

I've had a look in the live firewall log rule and I can see the connection:
   10G      May 5 16:44:21   192.168.0.11:61988   10.0.10.136:22   tcp   let out anything from firewall host itself

But no deny after 30 seconds or further communication during that period.

[chemlud]

If it's a driver issue then filter reloads would be seen in the system log, not firewall logs.

Play around with hardware offloading, try the same settings you used in pfsense, but be aware they are not using the same BSD version. :-)

PS: Your mobo has Intel NICs

https://www.supermicro.com/en/products/system/Mini-ITX/SYS-E300-9A.cfm

[albal]

Oh how embarrassing, I had the gateway for my VLAN as the router and not the switch.  Changed the IP and all is fine.   What was happening was the initial connection was going through the firewall but the reply was coming back on the switch, so after 30 seconds the firewall was dropping the connection.

Note: drawing a diagram of what is going on helps.

Thanks for your time.