Messages

Hello, it's been very long, but I wanted to at least provide some feedback on this. I've had other issues I had to solve in the meantime.

The solution was, indeed, using caddy as a reverse proxy. After evaluating a few possibilities, I just manually migrated things, and once I was able to solve an issue with name resolution, I was finally able to re-add state tracking on my LAN connection, and with fiber optics connecting things, it's blazing fast.

Thanks for the information that everyone provided. If anyone in the future finds itself in a similar situation, now you know. Either:

• VLANs to force your FW to see everything
• Reverse proxy on your FW (my solution, works wonders)
• Make sure your LAN only needs the FW for external connections, and route everything else before

A third option might be to use "sloppy" state tracking, but that's a bit of a hack, IMO.

Not bigger than the no state tracking I'm using right now, for the LAN interface ;)

In any case, thanks for all the suggestions. I'm going to dig down on them and see which makes more sense in my case.

So if I understand correctly, some unexpected routing is happening in this process. I will double check all netmasks just to be sure. Another thing is that I use an L2 switch after OPN, I do wonder if an L3 wouldn't do the trick too. Another point is that a lot of these services are running also inside the docker network, so I suppose some routing decisions might happen in the process that could cause this.

If you want traffic between the clients and the servers to pass through the firewall, you should use separate VLANs/subnets.

In essence, since OPN can't see the whole exchange, it's refusing the packets because it can't determine the state of the communication. Solutions are to either separate things physically or logically to force all communication to happen through the FW, or figure what part is being routed when it shouldn't, right?

[that's what I'm trying to solve]

Hello everyone, I'm very new to OPNsense (and proper firewalling), although I have passable knowledge in networking. I've hit a snag where I'm stuck in how to debug or solve this properly.

First things first, what I have and what I'm trying to achieve. My previous network was under a standard ISP router, so I was trying to first achieve some form of "parity" with what I had.

I have a PiHole for the network, which is configured as the default DNS, and it works quite well for what I want. I also have an Unraid server where I keep most of my network services. This includes an NPM (nginx) service, that works to provide network names for my services through reverse proxying. I don't do it directly on the Pi because I want to not remember the ports for all of them, so that means NPM does that job. I am using a custom domain name, out of the fun of it (.rabbit). What I always had was something like:

browser > plex.rabbit/ > get directed to plex > watch stuff
browser > nginx.rabbit/ > get directed to NPMs web page to configure my redirections
etc.

After I placed the OPNsense in between my ISP router and the rest of the network, part of those services stopped being properly redirected, or even allowing access through direct IP:Port to the webpages, and that's what I'm trying to solve.

The normal DNS resolution works perfectly (I know OPNs has something similar to PiHole, but I'm not going to use it for now), internet access seems to be working as expected, and a good portion of my services still work. The services that mostly failed critically for me are admin access to PiHole and nginx configuration pages.

After some sniffing around in the live view, I figured it was due to the firewall dropping TCP packets, which had the SA flags, which I'm assuming that the FW detected them as out of order, or the connection being incorrectly closed. Once I set a rule to disable state tracking inside my LAN, things started working again.

My network is as follows:

• My ISP router is now connected to my OPNsense box and works as the WAN, with network 10.0.0.0/24;
• My Lan is currently set up as 192.168.68.0/22, because I want a wider number of IPs, due to assigning a good portion of them to services (I know it can be done differently, but I have fun doing it this way);
• DHCPv4 is assigned from OPN, only in the .68 range. The LAN IP from OPN is 192.168.69.1;
• Unraid is on 69.254
• PiHole is on 69.250
• NPM is on 69.244

Name resolution for the LAN goes Device -> OPNs -> PiHole -> Nginx -> Service.

Now, I don't quite understand what's causing these packets to drop. I can see that the name is resolved as it should, but then it gets dropped once it is IP redirected. If I try to access the PiHole admin page "192.168.69.250/admin" from the browser, it still gets dropped, so it's not just from the reverse proxying acting up.

Can anyone give me some clues as to how to debug this? Or is this a known type of issue already that I completely missed when researching, because I don't know some important name of the issue I'm facing?