Messages

Hi all,

I have already searched the forum, but unfortunately I haven't found anything that could solve my problem.

My setup:

Code Select Expand

fritzbox: internal 192.168.9.254, exposed host to 192.168.9.1
opnsense CARP WAN IP 192.168.9.1
opnsense CARP DMZ IP 10.66.6.1
ftp server 10.66.6.3

Code Select Expand

port forward rule:
interface: WAN
destination: 192.168.9.1 (CARP WAN IP)
destination port range: FTP
redirected target: 10.66.6.3
redirected target port: FTP
Unfortunately, no package is arriving at the FTP server.

When I connect the FTP server to the network between Fritz and OPNsense (192.168.9.3) for testing purposes and set the exposed host on Fritz to 192.168.9.3, everything works, meaning that forwarding on Fritz is functioning.

I also unchecked the two boxes for 'Block private networks' and 'Block bogon networks' on the WAN interface, as we have RFC1918 addresses on both sides of the opnsense cluster.

the NAT rule created a rule for the WAN interface::

Code Select Expand

source: *
port: 21 (FTP)
destination: 10.66.6.3
gateway: *

Version: OPNsense 25.7.6-amd64

Ping between OPNsense and Fritzbox:

Code Select Expand

root@OPNsense1:~ # ping 192.168.99.254
PING 192.168.99.254 (192.168.99.254): 56 data bytes
64 bytes from 192.168.99.254: icmp_seq=0 ttl=64 time=0.069 ms
64 bytes from 192.168.99.254: icmp_seq=1 ttl=64 time=0.070 ms
64 bytes from 192.168.99.254: icmp_seq=2 ttl=64 time=0.068 ms
^C
--- 192.168.99.254 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.068/0.069/0.070/0.001 ms

Ping between OPNsense and FTP-Server:

Code Select Expand

root@OPNsense1:~ # ping 10.66.6.3
PING 10.66.6.3 (10.66.6.3): 56 data bytes
64 bytes from 10.66.6.3: icmp_seq=0 ttl=64 time=0.175 ms
64 bytes from 10.66.6.3: icmp_seq=1 ttl=64 time=0.137 ms
64 bytes from 10.66.6.3: icmp_seq=2 ttl=64 time=0.145 ms
64 bytes from 10.66.6.3: icmp_seq=3 ttl=64 time=0.202 ms
64 bytes from 10.66.6.3: icmp_seq=4 ttl=64 time=0.150 ms
64 bytes from 10.66.6.3: icmp_seq=5 ttl=64 time=0.138 ms
64 bytes from 10.66.6.3: icmp_seq=6 ttl=64 time=0.134 ms
^C
--- 10.66.6.3 ping statistics ---
7 packets transmitted, 7 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.134/0.154/0.202/0.023 ms

tcpdump on WAN if, from internet server telnet to port 21 of Fritzbox externally

Code Select Expand

root@OPNsense1:~ # tcpdump -i igb0 -n src host 138.201.203.161 and port 21
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on igb0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
13:25:43.352331 IP 138.201.203.161.38174 > 192.168.9.1.21: Flags [S], seq 1231877551, win 64240, options [mss 1452,sackOK,TS val 1634746021 ecr 0,nop,wscale 7], length 0
13:25:44.406669 IP 138.201.203.161.38174 > 192.168.9.1.21: Flags [S], seq 1231877551, win 64240, options [mss 1452,sackOK,TS val 1634747076 ecr 0,nop,wscale 7], length 0
13:25:45.431783 IP 138.201.203.161.38174 > 192.168.9.1.21: Flags [S], seq 1231877551, win 64240, options [mss 1452,sackOK,TS val 1634748101 ecr 0,nop,wscale 7], length 0
13:25:46.454730 IP 138.201.203.161.38174 > 192.168.9.1.21: Flags [S], seq 1231877551, win 64240, options [mss 1452,sackOK,TS val 1634749124 ecr 0,nop,wscale 7], length 0
13:25:47.479088 IP 138.201.203.161.38174 > 192.168.9.1.21: Flags [S], seq 1231877551, win 64240, options [mss 1452,sackOK,TS val 1634750148 ecr 0,nop,wscale 7], length 0
13:25:48.502637 IP 138.201.203.161.38174 > 192.168.9.1.21: Flags [S], seq 1231877551, win 64240, options [mss 1452,sackOK,TS val 1634751172 ecr 0,nop,wscale 7], length 0
13:25:50.550939 IP 138.201.203.161.38174 > 192.168.9.1.21: Flags [S], seq 1231877551, win 64240, options [mss 1452,sackOK,TS val 1634753220 ecr 0,nop,wscale 7], length 0
13:25:54.582609 IP 138.201.203.161.38174 > 192.168.9.1.21: Flags [S], seq 1231877551, win 64240, options [mss 1452,sackOK,TS val 1634757252 ecr 0,nop,wscale 7], length 0
13:25:54.617809 IP 138.201.203.161.21 > 192.168.9.1.15052: Flags [S.], seq 2802514873, ack 3951982751, win 65160, options [mss 1460,sackOK,TS val 1759046142 ecr 8452315,nop,wscale 7], length 0
13:25:54.700266 IP 138.201.203.161.21 > 192.168.9.1.15052: Flags [P.], seq 1:49, ack 1, win 510, options [nop,nop,TS val 1759046226 ecr 8452343], length 48: FTP: 220 ProFTPD Server (ProFTPD) [138.201.203.161]
13:25:54.728277 IP 138.201.203.161.21 > 192.168.9.1.15052: Flags [.], ack 7, win 510, options [nop,nop,TS val 1759046254 ecr 8452426], length 0
13:25:54.728296 IP 138.201.203.161.21 > 192.168.9.1.15052: Flags [P.], seq 49:63, ack 8, win 510, options [nop,nop,TS val 1759046254 ecr 8452426], length 14: FTP: 221 Goodbye.
13:25:54.731597 IP 138.201.203.161.21 > 192.168.9.1.15052: Flags [F.], seq 63, ack 8, win 510, options [nop,nop,TS val 1759046255 ecr 8452426], length 0
13:26:02.838641 IP 138.201.203.161.38174 > 192.168.9.1.21: Flags [S], seq 1231877551, win 64240, options [mss 1452,sackOK,TS val 1634765508 ecr 0,nop,wscale 7], length 0
^C
14 packets captured
20050 packets received by filter
0 packets dropped by kernel

tcpdump on DMZ if, identical test

Code Select Expand

root@OPNsense1:~ # tcpdump -i igb1 -n src host 138.201.203.161 and port 21
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on igb1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
^C
0 packets captured
197877 packets received by filter
0 packets dropped by kernel

Code Select Expand

root@OPNsense1:~ # tcpdump -i vlan0.666 -n src host 138.201.203.161 and port 21
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on vlan0.666, link-type EN10MB (Ethernet), snapshot length 262144 bytes
^C
0 packets captured
104706 packets received by filter
0 packets dropped by kernel
OPNsense ifconfig

Code Select Expand

root@OPNsense1:~ # ifconfig
igb0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: WAN (wan)
        options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,HWSTATS,MEXTPG>
        ether 20:7c:14:a1:68:2c
        inet 192.168.9.11 netmask 0xffffff00 broadcast 192.168.9.255
        inet 192.168.9.1 netmask 0xffffff00 broadcast 192.168.9.255 vhid 9
        inet6 fe80::227c:14ff:fea1:682c%igb0 prefixlen 64 scopeid 0x1
        carp: MASTER vhid 9 advbase 1 advskew 0
              peer 224.0.0.18 peer6 ff02::12
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
igb1: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
        description: LAN (lan)
        options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,HWSTATS,MEXTPG>
        ether 20:7c:14:a1:68:2d
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
igb2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: OPT1 (opt1)
        options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,HWSTATS,MEXTPG>
        ether 20:7c:14:a1:68:2e
        inet6 fe80::227c:14ff:fea1:682e%igb2 prefixlen 64 scopeid 0x3
        media: Ethernet autoselect
        status: no carrier
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
igb3: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
        description: HALink (opt2)
        options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,HWSTATS,MEXTPG>
        ether 20:7c:14:a1:68:2f
        inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
enc0: flags=0 metric 0 mtu 1536
        options=0
        groups: enc
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
pfsync0: flags=1000041<UP,RUNNING,LOWER_UP> metric 0 mtu 9000
        options=0
        syncdev: igb3 syncpeer: 192.168.0.2 maxupd: 128 defer: off version: 1400
        syncok: 1
        groups: pfsync
pflog0: flags=20100<PROMISC,PPROMISC> metric 0 mtu 33152
        options=0
        groups: pflog
vlan0.21: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
        description: LAN_VLAN21 (opt6)
        options=4000000<MEXTPG>
        ether 20:7c:14:a1:68:2d
        inet 192.168.21.251 netmask 0xffffff00 broadcast 192.168.21.255
        inet 192.168.21.254 netmask 0xffffff00 broadcast 192.168.21.255 vhid 21
        groups: vlan
        carp: MASTER vhid 21 advbase 1 advskew 0
              peer 224.0.0.18 peer6 ff02::12
        vlan: 21 vlanproto: 802.1q vlanpcp: 0 parent interface: igb1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vlan0.51: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
        description: LAN_VLAN51 (opt3)
        options=4000000<MEXTPG>
        ether 20:7c:14:a1:68:2d
        inet 192.168.51.251 netmask 0xffffff00 broadcast 192.168.51.255
        inet 192.168.51.254 netmask 0xffffff00 broadcast 192.168.51.255 vhid 51
        groups: vlan
        carp: MASTER vhid 51 advbase 1 advskew 0
              peer 224.0.0.18 peer6 ff02::12
        vlan: 51 vlanproto: 802.1q vlanpcp: 0 parent interface: igb1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vlan0.52: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
        description: LAN_VLAN52 (opt4)
        options=4000000<MEXTPG>
        ether 20:7c:14:a1:68:2d
        inet 192.168.52.251 netmask 0xffffff00 broadcast 192.168.52.255
        inet 192.168.52.254 netmask 0xffffff00 broadcast 192.168.52.255 vhid 52
        groups: vlan
        carp: MASTER vhid 52 advbase 1 advskew 0
              peer 224.0.0.18 peer6 ff02::12
        vlan: 52 vlanproto: 802.1q vlanpcp: 0 parent interface: igb1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vlan0.53: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
        description: LAN_VLAN53 (opt5)
        options=4000000<MEXTPG>
        ether 20:7c:14:a1:68:2d
        inet 192.168.53.251 netmask 0xffffff00 broadcast 192.168.53.255
        inet 192.168.53.254 netmask 0xffffff00 broadcast 192.168.53.255 vhid 53
        groups: vlan
        carp: MASTER vhid 53 advbase 1 advskew 0
              peer 224.0.0.18 peer6 ff02::12
        vlan: 53 vlanproto: 802.1q vlanpcp: 0 parent interface: igb1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vlan0.666: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
        description: LAN_VLAN666 (opt7)
        options=4000000<MEXTPG>
        ether 20:7c:14:a1:68:2d
        inet 10.66.6.251 netmask 0xffffff00 broadcast 10.66.6.255
        inet 10.66.6.1 netmask 0xffffff00 broadcast 10.66.6.255 vhid 66
        groups: vlan
        carp: MASTER vhid 66 advbase 1 advskew 0
              peer 224.0.0.18 peer6 ff02::12
        vlan: 666 vlanproto: 802.1q vlanpcp: 0 parent interface: igb1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
wg0: flags=10080c1<UP,RUNNING,NOARP,MULTICAST,LOWER_UP> metric 0 mtu 1420
        options=80000<LINKSTATE>
        inet 192.168.101.0 netmask 0xffffff00
        groups: wg wireguard
        nd6 options=109<PERFORMNUD,IFDISABLED,NO_DAD>
vlan0.99: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
        description: LAN_VLAN99 (opt8)
        options=4000000<MEXTPG>
        ether 20:7c:14:a1:68:2d
        inet 192.168.99.251 netmask 0xffffff00 broadcast 192.168.99.255
        inet 192.168.99.254 netmask 0xffffff00 broadcast 192.168.99.255 vhid 99
        groups: vlan
        carp: MASTER vhid 99 advbase 1 advskew 0
              peer 224.0.0.18 peer6 ff02::12
        vlan: 99 vlanproto: 802.1q vlanpcp: 0 parent interface: igb1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
OPNsense netstat -rn4

Code Select Expand

root@OPNsense1:~ # netstat -rn4
Routing tables

Internet:
Destination        Gateway            Flags         Netif Expire
default            192.168.9.253      UGS            igb0
1.1.1.1            192.168.9.253      UGHS           igb0
9.9.9.9            192.168.9.254      UGHS           igb0
10.66.6.0/24       link#13            U         vlan0.666
10.66.6.1          link#5             UHS             lo0
10.66.6.251        link#5             UHS             lo0
127.0.0.1          link#5             UH              lo0
192.168.0.0/24     link#4             U              igb3
192.168.0.1        link#5             UHS             lo0
192.168.9.0/24     link#1             U              igb0
192.168.9.1        link#5             UHS             lo0
192.168.9.11       link#5             UHS             lo0
192.168.21.0/24    link#9             U          vlan0.21
192.168.21.251     link#5             UHS             lo0
192.168.21.254     link#5             UHS             lo0
192.168.51.0/24    link#10            U          vlan0.51
192.168.51.251     link#5             UHS             lo0
192.168.51.254     link#5             UHS             lo0
192.168.52.0/24    link#11            U          vlan0.52
192.168.52.251     link#5             UHS             lo0
192.168.52.254     link#5             UHS             lo0
192.168.53.0/24    link#12            U          vlan0.53
192.168.53.251     link#5             UHS             lo0
192.168.53.254     link#5             UHS             lo0
192.168.99.0/24    link#15            U          vlan0.99
192.168.99.251     link#5             UHS             lo0
192.168.99.254     link#5             UHS             lo0
192.168.101.0      link#5             UHS             lo0
192.168.101.0/24   link#14            U               wg0

Any suggestion where I made a mistake?

Slainte
Harald

Ich werde den thread nochmal im englischen Bereich posten, dort lesen wesentlich mehr mit.

Wenn ich herausgefunden habe, welchen Fehler der typ vor meiner Tastatur gemacht hat, werde ich es hier posten.

so, todos erledigt.
FTP-Server hat nur noch 1 if
auf der LAN Seite der Sense ist nur noch tagged

aber Fehler ist immer noch da, tcpdump sieht Pakete auf dem WAN if, aber nicht auf dem DMZ if.

bei nem linux server mit iptables würde ich sagen ip forwarding ist nicht gesetzt, aber hier funktioniert es abgehend.

Noch irgend einen Tipp?

Die Erklärung für die 99.30 hab ich gefunden. Das ist mein icinga, der bei dem Host prüft, ob ftp funktioniert. Kann also ignoriert werden.

Ergo geht am DMZ if VLAN 666 kein einziges Paket von port forwarding raus.

Also liegt es doch irgendwo an einer Regel.
Kann ich in irgendeinem log sehen, warum das ankommende Paket nicht durch geht?
Bei der palo war das Schön im log mit Angabe der Regel, warum so entschieden wurde.

Die Sense hat:
Physikalisch 4, angeschlossen 3
WAN untagged VLAN 9
OPT2 ha link
LAN untagged VLAN 99, tagged 21,51,52,53,666

Der FTP hat folgende Interfaces:
4x physikalisch, 1 angeschlossen
VLAN666 untagged, 10.66.6.3, hier ist auch das default Gateway drauf, 10.66.6.1
VLAN9 tagged, 192.168.9.3, testweise
VLAN99 tagged, 192.168.99.3
Die beiden getaggten könnte ich auch rausnehmen, werde ich mal testen.

Mich wundert es, dass das Paket, welches auf WAN ankommt und auf die 10.66.6.3 in VLAN 666 gehen soll, auf die 192.168.99.30 im VLAN 99 (untagged) geht. Das einzige was an den beiden targets gleich ist, ist der Port 21


Meine nächsten todos:
FTP auf 1 if runterkonfigurieren
Auf dem LAN if der Sense, reiner Trunk ohne native VLAN

Du solltest auf dem igb1 kein untagged Interface haben sondern nur VLANs.

kann ich umstellen, kann das problem damit zu tun haben?

die 192.168.99.30 ist im vlan99 (untagged).

Code Select Expand

root@OPNsense1:~ # netstat -rn4
Routing tables

Internet:
Destination        Gateway            Flags         Netif Expire
default            192.168.9.253      UGS            igb0
1.1.1.1            192.168.9.253      UGHS           igb0
9.9.9.9            192.168.9.254      UGHS           igb0
10.66.6.0/24       link#13            U         vlan0.666
10.66.6.1          link#5             UHS             lo0
10.66.6.251        link#5             UHS             lo0
127.0.0.1          link#5             UH              lo0
192.168.0.0/24     link#4             U              igb3
192.168.0.1        link#5             UHS             lo0
192.168.9.0/24     link#1             U              igb0
192.168.9.1        link#5             UHS             lo0
192.168.9.11       link#5             UHS             lo0
192.168.21.0/24    link#9             U          vlan0.21
192.168.21.251     link#5             UHS             lo0
192.168.21.254     link#5             UHS             lo0
192.168.51.0/24    link#10            U          vlan0.51
192.168.51.251     link#5             UHS             lo0
192.168.51.254     link#5             UHS             lo0
192.168.52.0/24    link#11            U          vlan0.52
192.168.52.251     link#5             UHS             lo0
192.168.52.254     link#5             UHS             lo0
192.168.53.0/24    link#12            U          vlan0.53
192.168.53.251     link#5             UHS             lo0
192.168.53.254     link#5             UHS             lo0
192.168.99.0/24    link#2             U              igb1
192.168.99.251     link#5             UHS             lo0
192.168.99.254     link#5             UHS             lo0
192.168.101.0      link#5             UHS             lo0
192.168.101.0/24   link#14            U               wg0

was könnte bewirken, dass ein anderer host aus einer anderen zone die Pakete bekommt?

Und in die pcaps darfst du auch selbst reingucken :-)

hab ich schon, dabei ist mir das ja aufgefallen.

Code Select Expand

root@OPNsense1:~ # ifconfig
igb0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: WAN (wan)
        options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,HWSTATS,MEXTPG>
        ether 20:7c:14:a1:68:2c
        inet 192.168.9.11 netmask 0xffffff00 broadcast 192.168.9.255
        inet 192.168.9.1 netmask 0xffffff00 broadcast 192.168.9.255 vhid 9
        inet6 fe80::227c:14ff:fea1:682c%igb0 prefixlen 64 scopeid 0x1
        carp: MASTER vhid 9 advbase 1 advskew 0
              peer 224.0.0.18 peer6 ff02::12
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
igb1: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
        description: LAN (lan)
        options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,HWSTATS,MEXTPG>
        ether 20:7c:14:a1:68:2d
        inet 192.168.99.251 netmask 0xffffff00 broadcast 192.168.99.255
        inet 192.168.99.254 netmask 0xffffff00 broadcast 192.168.99.255 vhid 99
        carp: MASTER vhid 99 advbase 1 advskew 0
              peer 224.0.0.18 peer6 ff02::12
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
igb2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: OPT1 (opt1)
        options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,HWSTATS,MEXTPG>
        ether 20:7c:14:a1:68:2e
        inet6 fe80::227c:14ff:fea1:682e%igb2 prefixlen 64 scopeid 0x3
        media: Ethernet autoselect
        status: no carrier
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
igb3: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
        description: HALink (opt2)
        options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,HWSTATS,MEXTPG>
        ether 20:7c:14:a1:68:2f
        inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
enc0: flags=0 metric 0 mtu 1536
        options=0
        groups: enc
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
pfsync0: flags=1000041<UP,RUNNING,LOWER_UP> metric 0 mtu 9000
        options=0
        syncdev: igb3 syncpeer: 192.168.0.2 maxupd: 128 defer: off version: 1400
        syncok: 1
        groups: pfsync
pflog0: flags=20100<PROMISC,PPROMISC> metric 0 mtu 33152
        options=0
        groups: pflog
vlan0.21: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
        description: LAN_VLAN21 (opt6)
        options=4000000<MEXTPG>
        ether 20:7c:14:a1:68:2d
        inet 192.168.21.251 netmask 0xffffff00 broadcast 192.168.21.255
        inet 192.168.21.254 netmask 0xffffff00 broadcast 192.168.21.255 vhid 21
        groups: vlan
        carp: MASTER vhid 21 advbase 1 advskew 0
              peer 224.0.0.18 peer6 ff02::12
        vlan: 21 vlanproto: 802.1q vlanpcp: 0 parent interface: igb1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vlan0.51: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
        description: LAN_VLAN51 (opt3)
        options=4000000<MEXTPG>
        ether 20:7c:14:a1:68:2d
        inet 192.168.51.251 netmask 0xffffff00 broadcast 192.168.51.255
        inet 192.168.51.254 netmask 0xffffff00 broadcast 192.168.51.255 vhid 51
        groups: vlan
        carp: MASTER vhid 51 advbase 1 advskew 0
              peer 224.0.0.18 peer6 ff02::12
        vlan: 51 vlanproto: 802.1q vlanpcp: 0 parent interface: igb1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vlan0.52: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
        description: LAN_VLAN52 (opt4)
        options=4000000<MEXTPG>
        ether 20:7c:14:a1:68:2d
        inet 192.168.52.251 netmask 0xffffff00 broadcast 192.168.52.255
        inet 192.168.52.254 netmask 0xffffff00 broadcast 192.168.52.255 vhid 52
        groups: vlan
        carp: MASTER vhid 52 advbase 1 advskew 0
              peer 224.0.0.18 peer6 ff02::12
        vlan: 52 vlanproto: 802.1q vlanpcp: 0 parent interface: igb1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vlan0.53: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
        description: LAN_VLAN53 (opt5)
        options=4000000<MEXTPG>
        ether 20:7c:14:a1:68:2d
        inet 192.168.53.251 netmask 0xffffff00 broadcast 192.168.53.255
        inet 192.168.53.254 netmask 0xffffff00 broadcast 192.168.53.255 vhid 53
        groups: vlan
        carp: MASTER vhid 53 advbase 1 advskew 0
              peer 224.0.0.18 peer6 ff02::12
        vlan: 53 vlanproto: 802.1q vlanpcp: 0 parent interface: igb1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vlan0.666: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
        description: LAN_VLAN666 (opt7)
        options=4000000<MEXTPG>
        ether 20:7c:14:a1:68:2d
        inet 10.66.6.251 netmask 0xffffff00 broadcast 10.66.6.255
        inet 10.66.6.1 netmask 0xffffff00 broadcast 10.66.6.255 vhid 66
        groups: vlan
        carp: MASTER vhid 66 advbase 1 advskew 0
              peer 224.0.0.18 peer6 ff02::12
        vlan: 666 vlanproto: 802.1q vlanpcp: 0 parent interface: igb1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
wg0: flags=10080c1<UP,RUNNING,NOARP,MULTICAST,LOWER_UP> metric 0 mtu 1420
        options=80000<LINKSTATE>
        inet 192.168.101.0 netmask 0xffffff00
        groups: wg wireguard
        nd6 options=109<PERFORMNUD,IFDISABLED,NO_DAD>

Ich sehe gerade, dass ein anderer Server, welcher in einer anderen Zone ist antwortet.
Hierfür gibt es aber keine Regel.

Code Select Expand

> Ping zwischen OPNsense und Fritzbox tut?
root@OPNsense1:~ # ping 192.168.9.254
PING 192.168.9.254 (192.168.9.254): 56 data bytes
64 bytes from 192.168.9.254: icmp_seq=0 ttl=64 time=0.835 ms
64 bytes from 192.168.9.254: icmp_seq=1 ttl=64 time=0.454 ms
64 bytes from 192.168.9.254: icmp_seq=2 ttl=64 time=0.496 ms
^C
--- 192.168.9.254 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.454/0.595/0.835/0.170 ms

> Ping zwischen OPNsense und FTP-Server tut?
root@OPNsense1:~ # ping 10.66.6.3
PING 10.66.6.3 (10.66.6.3): 56 data bytes
64 bytes from 10.66.6.3: icmp_seq=0 ttl=64 time=0.212 ms
64 bytes from 10.66.6.3: icmp_seq=1 ttl=64 time=0.168 ms
64 bytes from 10.66.6.3: icmp_seq=2 ttl=64 time=0.131 ms
^C
--- 10.66.6.3 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss

> tcpdump auf WAN, von außen (anderer Internet-Uplink) telnet auf port 21 der Fritzbox extern - was passiert da?
root@OPNsense1:~ # tcpdump -i igb0 -n src host 138.201.203.161 and port 21
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on igb0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
15:18:25.956683 IP 138.201.203.161.57130 > 192.168.9.1.21: Flags [S], seq 244047560, win 64240, options [mss 1452,sackOK,TS val 1555108628 ecr 0,nop,wscale 7], length 0
15:18:26.964121 IP 138.201.203.161.57130 > 192.168.9.1.21: Flags [S], seq 244047560, win 64240, options [mss 1452,sackOK,TS val 1555109636 ecr 0,nop,wscale 7], length 0
15:18:27.989240 IP 138.201.203.161.57130 > 192.168.9.1.21: Flags [S], seq 244047560, win 64240, options [mss 1452,sackOK,TS val 1555110661 ecr 0,nop,wscale 7], length 0
15:18:29.012121 IP 138.201.203.161.57130 > 192.168.9.1.21: Flags [S], seq 244047560, win 64240, options [mss 1452,sackOK,TS val 1555111684 ecr 0,nop,wscale 7], length 0
15:18:30.036261 IP 138.201.203.161.57130 > 192.168.9.1.21: Flags [S], seq 244047560, win 64240, options [mss 1452,sackOK,TS val 1555112708 ecr 0,nop,wscale 7], length 0
15:18:31.059908 IP 138.201.203.161.57130 > 192.168.9.1.21: Flags [S], seq 244047560, win 64240, options [mss 1452,sackOK,TS val 1555113732 ecr 0,nop,wscale 7], length 0
15:18:33.108272 IP 138.201.203.161.57130 > 192.168.9.1.21: Flags [S], seq 244047560, win 64240, options [mss 1452,sackOK,TS val 1555115780 ecr 0,nop,wscale 7], length 0
15:18:33.853538 IP 138.201.203.161.21 > 192.168.9.1.48521: Flags [S.], seq 1226605640, ack 1865754026, win 65160, options [mss 1460,sackOK,TS val 1679405380 ecr 4223778784,nop,wscale 7], length 0
15:18:33.956501 IP 138.201.203.161.21 > 192.168.9.1.48521: Flags [P.], seq 1:49, ack 1, win 510, options [nop,nop,TS val 1679405483 ecr 4223778810], length 48: FTP: 220 ProFTPD Server (ProFTPD) [138.201.203.161]
15:18:34.013135 IP 138.201.203.161.21 > 192.168.9.1.48521: Flags [.], ack 7, win 510, options [nop,nop,TS val 1679405540 ecr 4223778913], length 0
15:18:34.013152 IP 138.201.203.161.21 > 192.168.9.1.48521: Flags [P.], seq 49:63, ack 7, win 510, options [nop,nop,TS val 1679405540 ecr 4223778913], length 14: FTP: 221 Goodbye.
15:18:34.014686 IP 138.201.203.161.21 > 192.168.9.1.48521: Flags [F.], seq 63, ack 7, win 510, options [nop,nop,TS val 1679405541 ecr 4223778913], length 0
15:18:34.019385 IP 138.201.203.161.21 > 192.168.9.1.48521: Flags [.], ack 8, win 510, options [nop,nop,TS val 1679405547 ecr 4223778913], length 0
15:18:34.036363 IP 138.201.203.161.21 > 192.168.9.1.48521: Flags [.], ack 8, win 510, options [nop,nop,TS val 1679405564 ecr 4223778968,nop,nop,sack 1 {7:8}], length 0
15:18:37.140069 IP 138.201.203.161.57130 > 192.168.9.1.21: Flags [S], seq 244047560, win 64240, options [mss 1452,sackOK,TS val 1555119812 ecr 0,nop,wscale 7], length 0
15:18:45.332602 IP 138.201.203.161.57130 > 192.168.9.1.21: Flags [S], seq 244047560, win 64240, options [mss 1452,sackOK,TS val 1555128004 ecr 0,nop,wscale 7], length 0
15:18:54.678355 IP 138.201.203.161.21 > 192.168.9.1.7172: Flags [S.], seq 3076054274, ack 801963521, win 65160, options [mss 1460,sackOK,TS val 1679426205 ecr 4223799610,nop,wscale 7], length 0
15:18:54.755219 IP 138.201.203.161.21 > 192.168.9.1.7172: Flags [P.], seq 1:49, ack 1, win 510, options [nop,nop,TS val 1679426282 ecr 4223799634], length 48: FTP: 220 ProFTPD Server (ProFTPD) [138.201.203.161]
15:18:54.798347 IP 138.201.203.161.21 > 192.168.9.1.7172: Flags [.], ack 7, win 510, options [nop,nop,TS val 1679426325 ecr 4223799711], length 0
15:18:54.798448 IP 138.201.203.161.21 > 192.168.9.1.7172: Flags [P.], seq 49:63, ack 7, win 510, options [nop,nop,TS val 1679426326 ecr 4223799711], length 14: FTP: 221 Goodbye.
15:18:54.800097 IP 138.201.203.161.21 > 192.168.9.1.7172: Flags [F.], seq 63, ack 8, win 510, options [nop,nop,TS val 1679426327 ecr 4223799711], length 0
^C
21 packets captured
14093 packets received by filter
0 packets dropped by kernel

> tcpdump auf DMZ, identischer Test
root@OPNsense1:~ # tcpdump -i igb1 -n src host 138.201.203.161 and port 21
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on igb1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
15:21:54.680954 IP 138.201.203.161.21 > 192.168.99.30.51190: Flags [S.], seq 1678331187, ack 2477296451, win 65160, options [mss 1460,sackOK,TS val 1679606207 ecr 4223979612,nop,wscale 7], length 0
15:21:54.794893 IP 138.201.203.161.21 > 192.168.99.30.51190: Flags [P.], seq 1:49, ack 1, win 510, options [nop,nop,TS val 1679606322 ecr 4223979637], length 48: FTP: 220 ProFTPD Server (ProFTPD) [138.201.203.161]
15:21:54.827699 IP 138.201.203.161.21 > 192.168.99.30.51190: Flags [.], ack 7, win 510, options [nop,nop,TS val 1679606353 ecr 4223979751], length 0
15:21:54.827718 IP 138.201.203.161.21 > 192.168.99.30.51190: Flags [P.], seq 49:63, ack 8, win 510, options [nop,nop,TS val 1679606354 ecr 4223979751], length 14: FTP: 221 Goodbye.
15:21:54.829276 IP 138.201.203.161.21 > 192.168.99.30.51190: Flags [F.], seq 63, ack 8, win 510, options [nop,nop,TS val 1679606355 ecr 4223979751], length 0
^[[A^C
5 packets captured
79910 packets received by filter
0 packets dropped by kernel
ich hab die pcaps in Dateien geschrieben, aber keine Möglichkeit gefunden, die hier anzuhängen.---- gefunden.

Steht auf Rule, die erzeugte Regel sieht so aus:
source: *
port: 21 (FTP)
destination: 10.66.6.3
gateway: *

ich hatte vorher ein palo cluster, wenn das das versuchsweise wieder anschließe, geht alles. daher gehe ich davon aus, dass ich irgendwelche regeln falsch habe.

Hallo zusammen,

ich habe bereits im Forum gesucht, aber leider nichts gefunden, das mein Problem lösen könnte.

Mein setup ist wie folgt:

fritzbox: intern 192.168.9.254, exposed host auf 192.168.9.1
opnsense CARP WAN IP 192.168.9.1
opnsense CARP DMZ IP 10.66.6.1
ftp server 10.66.6.3

meine port forward regel auf der opnsense sieht wie folgt aus:
interface: WAN
destination: 192.168.9.1 (CARP WAN IP)
destination port range: FTP
redirected target: 10.66.6.3
redirected target port: FTP

leider kommt am FTP server kein Paket an.

wenn ich den ftp server testweise ins netz zwischen fritz und opnsense hänge (192.168.9.3) und auf der fritz den exposed host auf 192.168.9.3 stelle, funktioniert alles, also funktioniert das forwarding auf der fritz.

ich habe auch auf dem wan interface die beiden Haken für "Block private networks" und "Block bogon networks" rausgenommen, da wir ja auf beiden Seiten des opnsense clusters RFC1918 Adressen haben.

bis jetzt leider erfolglos.

Hat jemand einen Tipp?

salü
.h