IPsec traffic blocked by Firewall despite allow rules - works in 16.7.14

[amp]

Hi all,

i am experiencing issues with IPSec and the Firewall on the 17.1 release. I currently run 17.1.3 on an appliance by deciso. We have this appliance here for a couple of weeks now.

Basically the firewall is blocking traffic on IPSec interface for Roadwarriors even with a "IPv4 *    *    *    *    *    * " any to any rule. Basic ping tests show blocked in the fw log despite the allow rule. It would be better to even restrict the rule to only allow the remote roadwarriors subnet to the LAN net but for tests i set it to allow any to any.

I did a test install in a VM with 16.7.14 and the basic setup with IPSec and Firewall and it works: i can ping the IP of the firewall itself. Then i upgraded to 17.1 it did not work any more, the ping is blocked by the fw. I reverted to the former snapshot and it worked again.

For both version 16.7.14 and 17.1.3 i exported the firewall rules from /tmp/rules.debug and tried to compare them but as the writing is quite different (the v16 file has nice comments while the new one not) i can't find the difference between them to figure out the missing rule in 17.

Here are the two rules.debug files, hopfully you can compare them better and see what might be missing or wrong in 17.1.3.

First rules 16.7.14 (working config)

OPNsense 16.7.14_2-amd64 FreeBSD 10.3-RELEASE-p14 OpenSSL 1.0.2j 26 Sep 2016

Code: [Select]

set optimization normal
set timeout { adaptive.start 0, adaptive.end 0 }
set limit states 201000
set limit src-nodes 201000

# System aliases
loopback = "{ lo0 }"
IPsec = "{ enc0 }"
LAN = "{ vtnet1 }"
WAN = "{ vtnet0 }"

# SSH Lockout Table
table <sshlockout> persist
table <webConfiguratorlockout> persist
# Other tables
table <virusprot>
table <bogons> persist file "/usr/local/etc/bogons"

# User Aliases
table <ipsec_net> {   10.10.1.0/24 }
ipsec_net = "<ipsec_net>"
 
# Gateways
GWWAN_DHCP = " route-to ( vtnet0 192.168.0.1 ) "
GWNull4 = " route-to ( lo0 127.0.0.1 ) "
GWNull6 = " route-to ( lo0 ::1 ) "

 
set loginterface vtnet1

set skip on pfsync0

scrub on $LAN all   
scrub on $WAN all   

no nat proto carp
no rdr proto carp

# Outbound NAT rules (automatic)

# Subnets to NAT
tonatsubnets  = "{ 127.0.0.0/8 192.168.2.1/32 10.10.1.0/24 }"
nat  on $WAN from $tonatsubnets to any port 500 -> 192.168.0.49/32  static-port
nat  on $WAN from $tonatsubnets to any -> 192.168.0.49/32 port 1024:65535 

# Anti lockout, prevent redirects for protected ports to this interface ip
no rdr on vtnet1 proto tcp from any to ( vtnet1 ) port { 443 80 }


#---------------------------------------------------------------------------
# default deny rules
#---------------------------------------------------------------------------
block in log inet all label "Default deny rule IPv4"
block out log inet all label "Default deny rule IPv4"
block in log inet6 all  label "Default deny rule IPv6"
block out log inet6 all  label "Default deny rule IPv6"

# IPv6 ICMP is not auxilary, it is required for operation
# See man icmp6(4)
# 1    unreach         Destination unreachable
# 2    toobig          Packet too big
# 128  echoreq         Echo service request
# 129  echorep         Echo service reply
# 133  routersol       Router solicitation
# 134  routeradv       Router advertisement
# 135  neighbrsol      Neighbor solicitation
# 136  neighbradv      Neighbor advertisement
pass log quick inet6 proto ipv6-icmp from any to any icmp6-type {1,2,135,136}  keep state

# Allow only bare essential icmpv6 packets (NS, NA, and RA, echoreq, echorep)
pass out log quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type {129,133,134,135,136}  keep state
pass out log quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type {129,133,134,135,136}  keep state
pass in log quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type {128,133,134,135,136}  keep state
pass in log quick inet6 proto ipv6-icmp from ff02::/16 to fe80::/10 icmp6-type {128,133,134,135,136}  keep state
pass in log quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type {128,133,134,135,136}  keep state

# We use the mighty pf, we cannot be fooled.
block log quick inet proto { tcp, udp } from any port = 0 to any
block log quick inet proto { tcp, udp } from any to any port = 0
block log quick inet6 proto { tcp, udp } from any port = 0 to any
block log quick inet6 proto { tcp, udp } from any to any port = 0

# SSH lockout
block in log quick proto tcp from <sshlockout> to (self) port 22 label "sshlockout"

# webConfigurator lockout
block in log quick proto tcp from <webConfiguratorlockout> to (self) port 443  label "webConfiguratorlockout"
block in log quick from <virusprot> to any label "virusprot overload table"
antispoof log for $LAN
antispoof log for $WAN
# allow our DHCP client out to the WAN
pass in log on $WAN proto udp from any port = 67 to any port = 68 label "allow dhcp client out WAN"
pass out log on $WAN proto udp from any port = 68 to any port = 67 label "allow dhcp client out WAN"
# Not installing DHCP server firewall rules for WAN which is configured for DHCP.

# loopback
pass in log on $loopback inet all label "pass IPv4 loopback"
pass out log on $loopback inet all label "pass IPv4 loopback"
pass in log on $loopback inet6 all label "pass IPv6 loopback"
pass out log on $loopback inet6 all label "pass IPv6 loopback"

# let out anything from the firewall host itself and decrypted IPsec traffic
pass out log inet all keep state allow-opts label "let out anything IPv4 from firewall host itself"
pass out log inet6 all keep state allow-opts label "let out anything IPv6 from firewall host itself"
pass out log route-to ( vtnet0 192.168.0.1 ) from 192.168.0.49 to !192.168.0.0/24 keep state allow-opts label "let out anything from firewall host itself"
pass out log on $IPsec all keep state label "IPsec internal host to host"

# make sure the user cannot lock himself out of the webConfigurator or SSH
pass in log quick on vtnet1 proto tcp from any to (vtnet1) port { 443 80 } keep state label "anti-lockout rule"

# User-defined rules follow
pass  in log  quick  on $IPsec inet from $ipsec_net to 192.168.2.1/32 keep state  label "USER_RULE: Allow IPsec traffic to LAN net"
pass  in  quick  on $WAN inet proto tcp  from any to 192.168.0.49 port 443  allow-opts flags S/SA keep state  label "USER_RULE"
pass  in  quick  on $WAN reply-to ( vtnet0 192.168.0.1 ) inet proto esp  from any to 192.168.0.49 keep state  label "USER_RULE: IPSec ESP"
pass  in  quick  on $WAN reply-to ( vtnet0 192.168.0.1 ) inet proto { tcp udp }  from any to 192.168.0.49 port 500 keep state  label "USER_RULE: IPSec ISAKMP"
pass  in  quick  on $WAN reply-to ( vtnet0 192.168.0.1 ) inet proto { tcp udp }  from any to 192.168.0.49 port 4500 keep state  label "USER_RULE: IPSec NAT-T"

# VPN Rules
pass out log on $WAN   proto udp from any to  any  port = 500 keep state label "IPsec: MobileIPsec - outbound isakmp"
pass in log on $WAN   proto udp from  any  to any port = 500 keep state label "IPsec: MobileIPsec - inbound isakmp"
pass out log on $WAN   proto udp from any to  any  port = 4500 keep state label "IPsec: MobileIPsec - outbound nat-t"
pass in log on $WAN   proto udp from  any  to any port = 4500 keep state label "IPsec: MobileIPsec - inbound nat-t"
pass out log on $WAN   proto esp from any to  any  keep state label "IPsec: MobileIPsec - outbound esp proto"
pass in log on $WAN   proto esp from  any  to any keep state label "IPsec: MobileIPsec - inbound esp proto"
then 17.1.3 (not working)

OPNsense 17.1.3-amd64 FreeBSD 11.0-RELEASE-p8 OpenSSL 1.0.2k 26 Jan 2017

Code: [Select]

set ruleset-optimization basic
set optimization normal
set timeout { adaptive.start 0, adaptive.end 0 }
set limit states 201000
set limit src-nodes 201000

# System aliases
loopback = "{ lo0 }"
IPsec = "{ enc0 }"
LAN = "{ vtnet1 }"
WAN = "{ vtnet0 }"

# SSH Lockout Table
table <sshlockout> persist
table <webConfiguratorlockout> persist
# Other tables
table <virusprot>
table <bogons> persist file "/usr/local/etc/bogons"

# User Aliases
table <ipsec_net> {   10.10.1.0/24 }
ipsec_net = "<ipsec_net>"
### define internal aliases
table <internal-enc0> {  }
table <internal-enc0ip> {  }
table <internal-lan> { 192.168.2.1 }
table <internal-lanip> { 192.168.2.1 }
table <internal-wan> { 192.168.0.0 }
table <internal-wanip> { 192.168.0.49 }
 
# Gateways
GWWAN_DHCP = " route-to ( vtnet0 192.168.0.1 ) "
GWNull4 = " route-to ( lo0 127.0.0.1 ) "
GWNull6 = " route-to ( lo0 ::1 ) "

 
set loginterface vtnet1

set skip on pfsync0

scrub on $LAN all   
scrub on $WAN all   

no nat proto carp
no rdr proto carp

# Outbound NAT rules (automatic)

# Subnets to NAT
tonatsubnets  = "{ 127.0.0.0/8 192.168.2.1/32 10.10.1.0/24 }"
nat  on $WAN from $tonatsubnets to any port 500 -> 192.168.0.49/32  static-port
nat  on $WAN from $tonatsubnets to any -> 192.168.0.49/32 port 1024:65535 

# Anti lockout, prevent redirects for protected ports to this interface ip
no rdr on vtnet1 proto tcp from any to ( vtnet1 ) port { 443 80 }

antispoof log for vtnet1
antispoof log for vtnet0
#pass in  log quick on lo0 inet6 from {any} to {any} label "Pass all loopback IPv6"
#block in  log quick inet6 from {any} to {any} label "Block all IPv6"
block in  log inet from {any} to {any} label "Default deny rule"
block in  log inet6 from {any} to {any} label "Default deny rule"
pass in  log quick inet6 proto ipv6-icmp from {any} to {any} icmp6-type {1,2,135,136} keep state label "IPv6 requirements (ICMP)"
pass out log quick inet6 proto ipv6-icmp from {fe80::/10} to {fe80::/10,ff02::/16} icmp6-type {129,133,134,135,136} keep state label "IPv6 requirements (ICMP)"
pass in log quick inet6 proto ipv6-icmp from {fe80::/10} to {fe80::/10,ff02::/16} icmp6-type {128,133,134,135,136} keep state label "IPv6 requirements (ICMP)"
pass in log quick inet6 proto ipv6-icmp from {ff02::/16} to {fe80::/10} icmp6-type {128,133,134,135,136} keep state label "IPv6 requirements (ICMP)"
block in  log quick inet proto {tcp udp}  from {any}  port {0} to {any}
block in  log quick inet6 proto {tcp udp}  from {any}  port {0} to {any}
block in  log quick inet proto {tcp udp}  from {any} to {any}  port {0}
block in  log quick inet6 proto {tcp udp}  from {any} to {any}  port {0}
#block in log quick proto carp from {(self)} to {any}
pass in  log quick proto carp from {any} to {any}
block in log quick proto tcp from {<sshlockout>} to {(self)}  port {22} label "sshlockout"
block in log quick proto tcp from {<webConfiguratorlockout>} to {(self)}  port {443} label "webConfiguratorlockout"
block in  log quick from {<virusprot>} to {any} label "virusprot overload table"
#block in log quick on lo0 from {<bogons>} to {any} label "block bogon IPv4 networks from loopback"
#block in log quick on lo0 from {<bogonsv6>} to {any} label "block bogon IPv6 networks from loopback"
#block in log quick on lo0 from {10.0.0.0/8,127.0.0.0/8,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16,fc00::/7} to {any} label "Block private networks from loopback"
#block in log quick on enc0 from {<bogons>} to {any} label "block bogon IPv4 networks from IPsec"
#block in log quick on enc0 from {<bogonsv6>} to {any} label "block bogon IPv6 networks from IPsec"
#block in log quick on enc0 from {10.0.0.0/8,127.0.0.0/8,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16,fc00::/7} to {any} label "Block private networks from IPsec"
#block in log quick on vtnet1 from {<bogons>} to {any} label "block bogon IPv4 networks from LAN"
#block in log quick on vtnet1 from {<bogonsv6>} to {any} label "block bogon IPv6 networks from LAN"
#block in log quick on vtnet1 from {10.0.0.0/8,127.0.0.0/8,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16,fc00::/7} to {any} label "Block private networks from LAN"
#block in log quick on vtnet0 from {<bogons>} to {any} label "block bogon IPv4 networks from WAN"
#block in log quick on vtnet0 from {<bogonsv6>} to {any} label "block bogon IPv6 networks from WAN"
#block in log quick on vtnet0 from {10.0.0.0/8,127.0.0.0/8,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16,fc00::/7} to {any} label "Block private networks from WAN"
pass in log on vtnet0 proto udp from {any}  port {67} to {any}  port {68} label "allow DHCP client on WAN"
pass out log on vtnet0 proto udp from {any}  port {68} to {any}  port {67} label "allow DHCP client on WAN"
pass in  log quick on lo0 from {any} to {any} label "pass loopback"
pass out log from {any} to {any} keep state allow-opts label "let out anything from firewall host itself"
pass out log on enc0 from {any} to {any} keep state label "IPsec internal host to host"
pass in log quick on vtnet1 proto tcp from {any} to {(self)}  port {443 80} keep state label "anti-lockout rule"
pass out log route-to ( vtnet0 192.168.0.1 ) from 192.168.0.49 to !192.168.0.0/24 keep state allow-opts label "let out anything from firewall host itself"

# User-defined rules follow
pass  in log  quick  on $IPsec inet from $ipsec_net to 192.168.2.1/32 keep state  label "USER_RULE: Allow IPsec traffic to LAN net"
pass  in  quick  on $WAN inet proto tcp  from any to 192.168.0.49 port 443  allow-opts flags S/SA keep state  label "USER_RULE"
pass  in  quick  on $WAN reply-to ( vtnet0 192.168.0.1 ) inet proto esp  from any to 192.168.0.49 keep state  label "USER_RULE: IPSec ESP"
pass  in  quick  on $WAN reply-to ( vtnet0 192.168.0.1 ) inet proto { tcp udp }  from any to 192.168.0.49 port 500 keep state  label "USER_RULE: IPSec ISAKMP"
pass  in  quick  on $WAN reply-to ( vtnet0 192.168.0.1 ) inet proto { tcp udp }  from any to 192.168.0.49 port 4500 keep state  label "USER_RULE: IPSec NAT-T"

# VPN Rules
pass out log on $WAN   proto udp from any to  any  port = 500 keep state label "IPsec: MobileIPsec - outbound isakmp"
pass in log on $WAN   proto udp from  any  to any port = 500 keep state label "IPsec: MobileIPsec - inbound isakmp"
pass out log on $WAN   proto udp from any to  any  port = 4500 keep state label "IPsec: MobileIPsec - outbound nat-t"
pass in log on $WAN   proto udp from  any  to any port = 4500 keep state label "IPsec: MobileIPsec - inbound nat-t"
pass out log on $WAN   proto esp from any to  any  keep state label "IPsec: MobileIPsec - outbound esp proto"
pass in log on $WAN   proto esp from  any  to any keep state label "IPsec: MobileIPsec - inbound esp proto"

I left all IPs in the file as it is a test VM and WAN is connected to my main LAN.

I hope i can help debugging this. If you need anything else tell me.

Best, Alex

[djGrrr]

Please try the kernel here: https://forum.opnsense.org/index.php?topic=4804.0

[franco]

The post dj refers to is scheduled for inclusion in 17.1.4 next week.

You are affected by this problem when you can see from the firewall logs that the traffic from the IPsec tunnel is being dropped by the default rule...


Cheers,
Franco

[abel408]

I'm wondering if this is related to my issue: https://forum.opnsense.org/index.php?topic=4848.0

I hope so!

[amp]

Hi, thanks for your replies.

I installed the new kernel
# opnsense-update -bkr 17.1.3-next and rebooted
but it did not change the behaviour of blocking the traffic on IPSec Interface.

If i have a rule
IPv4 *    ipsec_net     *    LAN net    *    *       Allow IPsec traffic to LAN net
on the IPSec interface (while ipsec_net is an alias for my 10.10.1.0/24 roadwarriors subnet)
then traffic is blocked in the log on the IPSec interface:
X   Mar 24 16:04:16    IPsec    10.10.1.1    192.168.2.1    ICMP

If i change the rule on IPSec to
IPv4 *    *    *    *    *    *       Allow IPsec traffic any to any
then traffic is passed on IPSec but blocked on WAN:
X     Mar 24 16:07:01    WAN    10.10.1.1    192.168.2.1    ICMP
>     Mar 24 16:07:01    IPsec    10.10.1.1    192.168.2.1    ICMP

So i am missing a rule on WAN again to allow IPSec traffic? In my understanding, also relating to the DOCs https://docs.opnsense.org/manual/how-tos/sslvpn_client.html this would not be necessary.

Looking forward to your replies.

Best, Alex

[franco]

Yup, that's it, one more step to apply this commit: https://github.com/opnsense/core/commit/8ec53e30

# opnsense-patch 8ec53e30


Cheers,
Franco

[franco]

(and reboot)

[amp]

Hi franco,

thanks for the patch. Ping to the firewall and back to the mobile client is working now!

Rule on the firewall is set to
"IPv4 *    *    *    *    *    *       Allow IPsec traffic any to any".

It seems not possible to restrict access further to
"IPv4 *    ipsec_net     *    LAN net    *    *       Allow IPsec traffic to LAN net"
then traffic will be blocked (ipsec_net is an alias to the Virtual Subnet of IPSec roadwarriors).
Any possibility to restrict access to subnets or is it necessary to keep to the any any rule?

This was tested now on a VM test install. I will report after the update of the production system when 17.1.4 is out.

Alex

[franco]

Hi Alex,

Glad to hear that's better. There's still some weirdness in filter matching, we have had discussed this previously with another user:

https://github.com/opnsense/core/issues/1441

If one restricts filtering on IPsec to ICMP pings don't work, but a blank rule will make it work (like every other traffic with it). I think your problem is the same bug?

There will be more testing / fixing for sure, but we should start to open new threads when we can verify this on 17.1.4. Chasing multiple IPsec regressions in the kernel through one thread is difficult.


Cheers,
Franco