Can't reach host from one interface to another

[ddutch206]

I am admit-tingly a very novice user of Opnsense but after struggling for two days, its time to ask for help.

Current setup.

Running latest version of OPNsense on a mini PC with 6x i226 ethernet ports (https://cwwk.net/products/j6412-j6413six-network-port-i226-nic-2-5g-soft-routing-mini-host-industrial-automation-retail-smart-city).

eth0 is WAN
eth1 is LAN, 10.0.0.1/24, unplugged
eth2 is Wifi, 10.0.2.1/23 - connected to a ethernet over power device that is powering two ASUS APs. Sixty devices sit on this. Its /23 on purpose because I've (not joking) run out of IPs before on /24.
eth3 is Media, 10.0.4.1/29 - directly connected to a NAS/Server (running on 10.0.4.2) serving media in the house
eth4 is VoIP, 10.0.5.1/29 - Gigaset VoIP (running on 10.0.5.2)

Most of the traffic sits on eth2, including the laptop I'm writing this on.

Reaching 10.0.0.1 (OPNsense) and 10.0.4.2 (various docker containers) HTTPS ports work w/o issue from 10.0.2.1/23. However I can't reach 10.0.5.2's HTTP port from my laptop. I can ping the IP just fine, but port 80 is non-responsive.

From the host running OPNsense, I can reach port 80

My laptop:

bash-5.2$ telnet 10.0.5.2 80                                                                                             
Trying 10.0.5.2...
telnet: connect to address 10.0.5.2: Operation timed out
telnet: Unable to connect to remote host


OPNsense:

XXXXXX@corerouter:~ $telnet 10.0.5.2 80
Trying 10.0.5.2...
Connected to SL450A-GO.XXXX
Escape character is '^]'.


I have four floating rules setup to allow access into the LAN, Wifi, Media and VOIP interfaces. This works fine for Wifi -> Media and I have an identical one for the VOIP network


https://imgur.com/a/IPHQDxY

I'm at a loss as to why I can't reach port 80 from any device on the 10.0.2.1/23 network to the device on the 10.0.5.1/29 network when pinging the device works w/o issue.

[bachmarc]

I am really not an expert for OPNS and have to admit, I never touched floating rules at all..
There maybe others that can point to the root cause but I would simple define rules per interface.

I created rules per interface and this does exactly the job: I can adjust who can reach which subnet from which side.

it may all be simpler, faster, cooler with floating rules BUT I never read the documentation behind and found it not to hard to start with by interface rules to get it run...
I thought floating is to logically group interfaces and apply similar filters across that groups. This sounds not like what you want and it is obviously not fast in your case

is there a special reason for that uncommon subnet masks? I like it to match with decimal IP numbers and the dots...
Marc 

[Demusman]

I agree with the above. Why are you using floating rules?? A good rule of thumb is to never use floating rules unless absolutely necessary. Too many chances for errors.
Put the rules where they belong, on the interface itself.

[ddutch206]

Ok two replies that say floating rules is a bad idea. So I will spend tonight changing my floating rules to be on the interface.

Honestly the reason was laziness. I knew I didn't want to create a bridge network, but did want the internal interfaces to have the ability to access each other. Coming into the WAN I only have 4 ports defined, everything else is closed down.

[slackadelic]

The only time I've ever used floating rules is when I needed the same rule to apply across multiple interfaces and weren't interface agnostic, if that makes sense.

[bachmarc]

Honestly the reason was laziness. I knew I didn't want to create a bridge network, but did want the internal interfaces to have the ability to access each other. Coming into the WAN I only have 4 ports defined, everything else is closed down.

If you apply a default "allow every protocol from every source to every target" to each interface, then all can talk to each other with an ease... no bridge needed, this is handled inside.

The only reason against it: if you anyway allow any subnet to reach everybody else, I see no point in subnets at all. Technically a bridge and one subnet would provide the same with a few lines in systemd.networkd or bridge command.
Just would cost you the fun in OPNS

Marc

[Demusman]

I knew I didn't want to create a bridge network, but did want the internal interfaces to have the ability to access each other. Coming into the WAN I only have 4 ports defined, everything else is closed down.

Why would you need a bridge?
This will work with rules alone.

[EdwinKM]

First, it is bad to test from the opnsense itself. Use a system *ON* the network.

Second, if you are running out of ip's you can just use another network with a normal mask (8, 16 or 32)

[WN1X]

Or shorten the DHCP lease times so as to recover inactive IPs.