different routing for TCP/UDP and ICMP

[ronald]

Hi All,

I'm encountering the following behaviour which I fail to understand:

Code: [Select]

root@FW2:~ # traceroute -I fw3
traceroute to fw3.independit.de (192.168.36.60), 64 hops max, 48 byte packets
 1  dns2 (192.168.16.3)  0.362 ms  0.268 ms  0.243 ms
 2  fw3 (192.168.36.60)  2.574 ms  2.522 ms  2.278 ms
root@FW2:~ # traceroute fw3
traceroute to fw3.independit.de (192.168.36.60), 64 hops max, 40 byte packets
 1  fw1 (192.168.16.60)  0.384 ms  0.284 ms  0.250 ms
 2  zyxel (192.168.2.1)  0.895 ms  0.728 ms  0.744 ms
 3  62.156.244.22 (62.156.244.22)  12.491 ms  12.249 ms  12.061 ms
^C
root@FW2:~ # traceroute -P TCP fw3
traceroute to fw3.independit.de (192.168.36.60), 64 hops max, 40 byte packets
 1  fw1 (192.168.16.60)  0.393 ms  0.262 ms  0.261 ms
 2  zyxel (192.168.2.1)  0.826 ms  0.815 ms  0.789 ms
 3  62.156.244.22 (62.156.244.22)  12.015 ms  12.158 ms  11.605 ms
^C

IOW, if I traceroute from FW2 to FW3 using the standard UDP or TCP, the packages are sent to the default GW, if I use ICMP the packages are (correctly) sent to the staitcally configured GW.

The routing table of FW2 looks like:

Code: [Select]

Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.16.60      UGS        igb0
127.0.0.1          link#6             UH          lo0
192.168.2.0/24     192.168.16.60      UGS        igb0
192.168.9.0/24     192.168.16.60      UGS        igb0
192.168.16.0/24    link#1             U          igb0
192.168.16.38      link#1             UHS         lo0
192.168.25.0/24    link#2             U          igb1
192.168.25.60      link#2             UHS         lo0
192.168.36.0/24    192.168.16.3       UGS        igb0
192.168.49.0/24    192.168.16.3       UGS        igb0

If I try to reach FW3 from behind FW2 (i.e. from the 192.168.25.0/24 network), this works without issues.

I used to have the problem that I couldn't even ping from FW2 to FW3, but was able to "repair" this by adding a firewall rule.
Adding the same firewall rule for UDP instead of ICMP didn't change the behaviour shown above though.

The mistake is probably sitting in front of the computer, but still, I don't understand a bit of this.
I also don't understand the concept that firewall rules influence routing.

In my naive understanding, a firewall is a kind of custom office that checks the documents of the packet that wants to pass the border (and maybe also checks the luggage space). The routing table is like a bunch of road signs telling the packet what path to follow in order to reach its destination.
But obviously this is an incorrect understanding.

I already had defined very loose rules that permit all traffic from the 192.168.25.0/24 network to the outside.
But first after adding the redundant more specific rule for ICMP, the "ping" and "traceroute -I" selected the correct route.

Oh, and for the record, FW2 has installed:
OPNsense 20.7.7_1-amd64
FreeBSD 12.1-RELEASE-p11-HBSD
OpenSSL 1.1.1i 8 Dec 2020

If more information is required, please tell me.

Could anyone please try to explain me why my understanding of this is so horribly wrong?
And why UDP or TCP packets are forced to take a different route than ICMP packets (if sent from FW2) ?

Thanks in advance!

Ronald