OPNsense Forum

Archive => 17.1 Legacy Series => Topic started by: kyferez on February 18, 2017, 07:24:07 pm

Title: [WORKAROUND FOUND] Trouble with Firewall Rules on return communication
Post by: kyferez on February 18, 2017, 07:24:07 pm
So I have OPNsense as a firewall between 2 VLANs. I have 2 Win 2012 R2 servers that since upgrading to 17.1 cannot communicate over Kereros, LDAP, etc. However SMB currently works fine, which is weird because they are all the same firewall rule (using aliases). I did packet traces on both servers and checked the logs on the firewall. Screenshot attached. You see the firewall logs passed traffic, which the receiving server gets, but the SYN ACK reply is lost.

In this test, I initiate port 88 traffic from server s1 to server lic1. lic1 receives the SYN, sends a SYN ACK. s1 does not receive the SYN ACK.

Any ideas? How do I verify if the firewall received the SYN ACK from lic1? Do you have a GUI tcpdump utility?


Here is the network details:

VLAN1: 192.168.1.0/24
OPNsense Interface: 192.168.1.1
Server s1: 192.168.1.220
Firewall disabled.
s1 default gateway: 192.168.1.254 but has a permanent route to 192.168.2.0/24 via 192.168.1.1, verified working with tracert.

VLAN2: 192.168.2.0/24
OPNSense Interface: 192.168.2.1
Server lic1: 192.168.2.230
Firewall disabled.
lic1 default route is 192.168.2.1

Thanks!

PS the image max attachment of 192kb is way too small; makes it hard to post multiple screenshots.
Title: Re: Trouble with Firewall Rules on return communication
Post by: bartjsmit on February 18, 2017, 07:34:58 pm
The dumps taken by Interfaces->Diagnostics->Packet capture can be downloaded for analysis by GUI tools, most commonly Wireshark.

Bart...
Title: Re: Trouble with Firewall Rules on return communication
Post by: djGrrr on February 18, 2017, 07:39:21 pm
Make sure you are on 17.1.1 first, then run this command in shell / console as root:
sysctl net.pf.share_forward=0
Title: Re: Trouble with Firewall Rules on return communication
Post by: kyferez on February 19, 2017, 03:37:32 am
djGrr, what does that command do? Nevermind, I found it. Waiting for the latest updates to apply and will report back.

Thanks!
Title: Re: Trouble with Firewall Rules on return communication
Post by: kyferez on February 19, 2017, 04:11:23 am
No change, setting it to 0 or to 1. Either way on 17.1.1 I get the same results.
Title: Re: Trouble with Firewall Rules on return communication
Post by: kyferez on February 19, 2017, 04:27:27 am
So  OPNsense is sending the SYN ACK packet back to VLAN1, but instead of directly to the s1 server, it sends it to the MAC address of the default gateway. See image. Is this expected behavior?

Franco, can you comment on this?
Title: Re: Trouble with Firewall Rules on return communication
Post by: JeGr on February 19, 2017, 06:31:24 pm
did you configure both default gateways on the corresponding interfaces of opnsense?
Title: Re: Trouble with Firewall Rules on return communication
Post by: djGrrr on February 19, 2017, 07:28:21 pm
You may need to use the "disable reply-to" advanced option on one of your rules, or add a higher priority firewall rule before any other rule with the gateway set for traffic between the 2 vlans; for example, if you have a "default route" firewall rule for traffic with source of vlan 2 net, destination any, add a rule above it: source vlan 2: destination vlan1 net (or an alias of destination networks), without a gateway set.

A screenshot of firewall rules would help to debug this.
Title: Re: Trouble with Firewall Rules on return communication
Post by: kyferez on February 20, 2017, 12:00:43 am
There is no default gateway for VLAN2. Only VLAN1.
None of the rules have a Default Gateway set.
I do have a Allow VLAN2 any to VLAN1 rule... see attachment for Gateway and rules.
Title: Re: Trouble with Firewall Rules on return communication
Post by: djGrrr on February 20, 2017, 04:11:23 am
Try editing the "Allow VLAN2 any to VLAN1" rule, under advanced options, check "disable reply-to", then save and apply and see if anything changes
Title: Re: Trouble with Firewall Rules on return communication
Post by: kyferez on February 20, 2017, 04:38:17 pm
djGrrr,

Done. No change. (yes I applied the change)
Title: Re: Trouble with Firewall Rules on return communication
Post by: kyferez on February 22, 2017, 08:38:17 pm
Any other ideas?
Title: Re: Trouble with Firewall Rules on return communication
Post by: djGrrr on February 22, 2017, 09:35:14 pm
try disabling your floating rules temporarily and see what happens.
Title: Re: Trouble with Firewall Rules on return communication
Post by: kyferez on February 22, 2017, 10:05:44 pm
So it appears we have found a bug...? I saw a number of posts reading the forums yesterday that looks like this may be related to issues others are having...

Anyway, thanks for the idea djGrrr, that lead me to a workaround.

I noticed that port 445 stopped working when I disabled my floating rules - I had that port in my Remote Management Alias on one of the Floating rules.

Sooooo.... I added a new floating rule with Destination AD_DNS_Servers and Destination Ports AD_Services and applied the changes. BOOM it now works! YAA servers can sync again!

Title: Re: [WORKAROUND FOUND] Trouble with Firewall Rules on return communication
Post by: djGrrr on February 23, 2017, 01:59:48 am
Keep In mind that floating rules are processed first, and if you have the quick option selected (the default), no further rules will be processed once it is matched. Also, if you do not select an interface, or perhaps a direction for the rule, it can override rules on interface tabs. Generally you should put all rules under specific interfaces, and only use floating rules when they are required (like for a direction other than "in")