Hello all,
I've done as much debugging on this one as I can work out, so, would appreciate suggestions.
I have a LAN network defined as 192.168.50.0/23 (with the LAN IP being 192.168.50.1).
I've set up the DHCPv4 server so that 51.10 through 51.200 are for DHCP leases.
I have a static IP on a local Raspberry Pi on 50.2, and the https port on the Pi is reachable from anything in the 50.0/24 part of the /23, but, nothing in the DHCP lease pool can reach the Pi.
The Pi's eth0 is correct:
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.50.2 netmask 255.255.254.0 broadcast 192.168.51.255
Most of the devices in the 51.0/24 part of the network are WiFi, and, I've made sure the issue is not the WiFi access point; I've also set a WiFI device to have a static IP of 192.168.50.12 and using WiFi that can reach the 50.2 host no problem. It's only when the DHCPv4 lease IP of 51.x is used that the Default Deny rule of the firewall in OPNsense kicks in.
I can see the packets logged as such:
QuoteLAN 2022-09-13T09:08:35 192.168.50.2:443 192.168.51.10:53027 tcp Default deny / state violation rule
LAN 2022-09-13T09:08:34 192.168.50.2:443 192.168.51.10:53027 tcp Default deny / state violation rule
LAN 2022-09-13T09:08:33 192.168.50.2:443 192.168.51.10:53027 tcp Default deny / state violation rule
LAN 2022-09-13T09:08:33 192.168.50.2:443 192.168.51.10:53027 tcp Default deny / state violation rule
LAN 2022-09-13T09:08:32 192.168.50.2:443 192.168.51.10:53027 tcp Default deny / state violation rule
LAN 2022-09-13T09:08:31 192.168.50.2:443 192.168.51.10:53027 tcp Default deny / state violation rule
And if you look at the details of one of the rules, it says:
Quote
__timestamp__ 2022-09-13T09:08:35
ack 2551861249
action [block]
anchorname
datalen 0
dir [in]
dst 192.168.51.10
dstport 53027
ecn
id 0
interface igb0
interface_name LAN
ipflags DF
ipversion 4
label Default deny / state violation rule
length 60
offset 0
protoname tcp
protonum 6
reason match
rid 02f4bab031b57d1e30553ce08e0ec131
rulenr 14
seq 3121781003
src 192.168.50.2
srcport 443
subrulenr
tcpflags SAE
tcpopts
tos 0x0
ttl 64
urp 65160
And the default rules which are in Firewall -> Rules -> LAN include:
IPv4 * LAN net * * * * * Default allow LAN to any rule
IPv6 * LAN net * * * * * Default allow LAN IPv6 to any rule
Those rules are Pass, and In.
I have also tried adding rules which were IPv4+IPv6 in pass for any/any source destination, not specifically the LAN net, and, yes, I've also tried rules for OUT which were any/any.
The SYN packets are making it to the Pi, and the Pi responds:
09:14:50.815803 IP 192.168.51.10.53036 > 192.168.50.2.443: Flags [SEW], seq 2284235656, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 2341413872 ecr 0,sackOK,eol], length 0
09:14:50.815934 IP 192.168.50.2.443 > 192.168.51.10.53036: Flags [S.E], seq 2610416432, ack 2284235657, win 65160, options [mss 1460,sackOK,TS val 1298482186 ecr 2341413872,nop,wscale 7], length 0
09:14:51.818903 IP 192.168.50.2.443 > 192.168.51.10.53036: Flags [S.E], seq 2610416432, ack 2284235657, win 65160, options [mss 1460,sackOK,TS val 1298483189 ecr 2341413872,nop,wscale 7], length 0But the firewall is blocking the returning packets from the Pi to the 51.10 IP (in this case, an iPhone).
Curiously, I CAN reach other 50.0/24 hosts from the iPhone, which is puzzling.
And anything at all with 50.0/24 can reach 50.2 - it's purely running into that default deny rule for packets returning to 51.0/24 within the overall /23.
And the LAN network IP is correct as well:
igb0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: LAN
options=4800028<VLAN_MTU,JUMBO_MTU,NOMAP>
inet 192.168.50.1 netmask 0xfffffe00 broadcast 192.168.51.255
Any obvious thoughts as to what I'm overlooking for this incredibly simple and stupid question?
There's nothing like staring at a problem and changing something that you think "this could NOT possibly have ANY impact to the problem" - but it does.
Fixed.
WHY it broke is still unclear to me, but the fix was for me to move the 50.2 server to the same switch that is on my LAN port of my OPNsense device.
It was attached to a different switch, which is actually my WiFi AP device.
So the traffic WAS:
iPhone -> AP -> Server (physically attached to AP)
And is now:
iPhone -> AP -> Switch -> Server
I'm guessing that the Asus WiFi device in AP mode is doing something funky that is confusing the OPNsense firewall as to where the packet originated from.