Strange behavior - facebook.com TCP paquets droped by default deny rule

[burno]

Hello all,

I noticed a very strange behavior since last couple of days.

Code: [Select]

OPNsense 22.1.6-amd64
FreeBSD 13.0-STABLE
OpenSSL 1.1.1n 15 Mar 2022
No IDS/IPS , no sensei, direct DNS transfert to 1.1.1.1 with dnsmasq.


Facebook.com on TCP 443 match the default deny rule.
It's happening from any device in my network.

Code: [Select]

V10_DATA 2022-05-06T14:22:37 MBP13-*********.lan:58718 edge-star-shv-01-cdt1.facebook.com:443 tcp Default deny / state violation rule
V10_DATA 2022-05-06T14:22:37 MBP13-*********.lan:58719 edge-star-shv-01-cdt1.facebook.com:443 tcp Default deny / state violation rule
V10_DATA 2022-05-06T14:22:32 MBP13-*********.lan:58718 edge-star-shv-01-cdt1.facebook.com:443 tcp Default deny / state violation rule
V10_DATA 2022-05-06T14:22:32 MBP13-*********.lan:58719 edge-star-shv-01-cdt1.facebook.com:443 tcp Default deny / state violation rule
V10_DATA 2022-05-06T14:22:31 MBP13-*********.lan:58747 edge-star-shv-01-cdt1.facebook.com:443 tcp Facebook out
V10_DATA 2022-05-06T14:22:31 MBP13-*********.lan:58746 edge-star-shv-01-cdt1.facebook.com:443 tcp Facebook out
V10_DATA 2022-05-06T14:22:27 MBP13-*********.lan:58718 edge-star-shv-01-cdt1.facebook.com:443 tcp Default deny / state violation rule
V10_DATA 2022-05-06T14:22:21 MBP13-*********.lan:58718 edge-star-shv-01-cdt1.facebook.com:443 tcp Default deny / state violation rule
V10_DATA 2022-05-06T14:22:06 MBP13-*********.lan:58731 edge-star-mini-shv-01-mrs2.facebook.com:443 tcp Facebook out
V10_DATA 2022-05-06T14:22:06 MBP13-*********.lan:58730 edge-star-mini-shv-01-mrs2.facebook.com:443 tcp Facebook out
V10_DATA 2022-05-06T14:21:51 MBP13-*********.lan:58719 edge-star-shv-01-cdt1.facebook.com:443 tcp Facebook out
V10_DATA 2022-05-06T14:21:51 MBP13-*********.lan:58718 edge-star-shv-01-cdt1.facebook.com:443 tcp Facebook out

for this example I have created an Alias with certain FQDN of facebook, and a FW rule to explicit allow traffic to it.
The rule "facebook out" match few seconds but after that, it's the "Default deny / state violation rule".

Rules are pretty simple and permissives

Code: [Select]

Protocole Source Port Destination Port Passerelle Planifier Description    
  IPv4 TCP/UDP V10_DATA net * Ce Pare-feu 53 (DNS) * *    
  IPv4 TCP/UDP    V10_DATA net * Ce Pare-feu 123 (NTP) * *    
  IPv4+6 ICMP     V10_DATA net * * * * *    
  IPv4+6 *    V10_DATA net * host_facebook_com * * * Facebook out    
  IPv4+6 *    V10_DATA net * ! net_local_home * * *


Code: [Select]


Alias : host_facebook_com
 
157.240.195.17
157.240.21.10
157.240.21.11
157.240.21.16
157.240.21.35
179.60.192.2
179.60.192.3
2a03:2880:f030:f:face:b00c:0:2
2a03:2880:f042:12:face:b00c:0:2
2a03:2880:f130:83:face:b00c:0:25de


Code: [Select]

traceroute to facebook.com (157.240.21.35), 64 hops max, 52 byte packets
 1  fw01 (10.1.10.254)  3.496 ms  2.479 ms  4.009 ms
 2  1.217.144.77.rev.sfr.net (77.144.217.1)  5.204 ms  6.602 ms  5.929 ms
 3  2.17.197.77.rev.sfr.net (77.197.17.2)  8.256 ms  7.849 ms  6.866 ms
 4  233.10.136.77.rev.sfr.net (77.136.10.233)  16.514 ms  16.973 ms  15.648 ms
 5  233.10.136.77.rev.sfr.net (77.136.10.233)  15.389 ms  14.476 ms  16.330 ms
 6  ae20.pr06.cdg1.tfbnw.net (157.240.65.214)  15.247 ms  62.459 ms  61.611 ms
 7  po161.asw03.cdg1.tfbnw.net (31.13.24.192)  14.967 ms
    po161.asw02.cdg1.tfbnw.net (31.13.24.190)  15.745 ms
    po161.asw01.cdg1.tfbnw.net (31.13.24.188)  15.161 ms
 8  po232.psw01.cdg1.tfbnw.net (157.240.50.51)  15.083 ms
    po244.psw02.cdg1.tfbnw.net (157.240.50.5)  15.907 ms
    po233.psw04.cdg1.tfbnw.net (157.240.50.121)  15.975 ms
 9  173.252.67.9 (173.252.67.9)  15.326 ms
    157.240.38.93 (157.240.38.93)  14.915 ms
    173.252.67.47 (173.252.67.47)  14.701 ms
10  edge-star-mini-shv-01-cdt1.facebook.com (157.240.21.35)  15.574 ms  15.951 ms  14.948 ms


I found a similar topic on Reddit :
https://www.reddit.com/r/opnsense/comments/ubo9ve/packets_from_lan_facebook_whatsapp_are_getting/

I wonder if anyone here as the same problem/solution.

thx

[meyergru]

I sometimes see packets originating from port 443 getting rejected in my firewall, e.g. from some Microsoft servers. Those are most often packets that are rejected for the invalid tcp state (e.g. "PA").
I just tried facebook and see the same kind of log entries, while the connection itself does work.

I suspect that effect being attributable to some kind of late TCP FIN packets (https://docs.netgate.com/pfsense/en/latest/troubleshooting/log-filter-blocked.html) for web servers trying to reuse connections with HTTP, or maybe even artifacts of QUIC (https://engineering.fb.com/2020/10/21/networking-traffic/how-facebook-is-bringing-quic-to-billions/).

[burno]

Thank you for your help.

I was able to resolve this issue by disabling the DDoS parameter :

Firewall / settings / advanced

Code: [Select]

Anti DDOS
 Enable syncookies
never (default)
When syncookies are active, pf will answer each incoming TCP SYN with a syncookie SYNACK, without allocating any resources.