Nothing passing through, no matter what I do, for a simple configuration

[HH KD]

Hi all. I'm completely new to this so forgive me...and sorry it's a long one. I bought an OPNsense 'firewall' (* see below for version etc.) on eBay and so far, after days and days, I've not been able to get a single packet to go though it. Understandably it's driving me insane - perhaps naively I thought it would be easier to use than this.

My goal is really simple I believe, but perhaps the way I'm going about it might be non-standard. I just want the OPNsense to sit within my existing LAN and perform an additional layer of security and access control for certain machines that sit 'behind' it. I have little control over the existing LAN set up, so I'd rather not be directed to just do it differently.

So, to summarise what I need is:

Internet -> ISP provided modem -> ISP provided gateway (which is also the main WiFi) -> OPNsense firewall -> managed switch -> servers and work stations

Because of the WiFi for mobiles and laptops, and some lack of control over the gateway hardware, the LAN is effective predefined and I have to fit my box into it.

It seems so simple and obviously a common set up, yet it's a no go so far. Perhaps this is because of the way I am setting it up (which must also be pretty common) - that is: owing to the fact that I have connected servers and work stations I'd rather not disconnect them all during the OPNsense set up (and thank god I didn't try it that way as my devices would have been down for a week or more by now). So what I'm attempting is to set the OPNsense up elsewhere in the LAN, and then move it into position when ready (surely that's common?).

So my actual layout during config (which is the no-pass problem one) is as follows:

Internet -> ISP provided modem -> ISP provided gateway (which is also the main WiFi) -> switch -> OPNsense firewall -> non-critical client (laptop)

I am using a fully factory reset install and doing the basic level config to try to get any (and preferably all at first) packets to go in both directions across the firewall, but so far nothing at all has passed. I've allowed bogon and private addresses on both interfaces, I've set up a pass any to any rule in the firewall; and I've even tried completely disabling the firewall and still nothing gets through. I don't need NAT, and have tried disabling that separately, but the wording of the GUI on this is not as clear as it might be. I've tried lots of other things. The worst thing is that even with full logging on all rules no obvious 'block' messages appear in the firewall log - it's as if the packets don't exist (they're just pings and port 80's at first). So everything is blocked, but there are no blocks in the log! I suspect it might be either a routing thing, or something to do with IPv6 rules (which again I don't need).

Details of my IPv4 set up are as follows:

the gateway LAN address space is 192.168.1.0/24
gateway is on 192.168.1.1
I'm attempting to do a split scope DHCP thing - leave subnetting and VLANs for another day
gateway does DHCP for a split scope: 192.168.1.50 to 192.168.1.200 (basically for mobiles and laptops)
I have a MAC based reservation on the gateway for the OPNsense WAN interface of 192.168.1.2
I have a MAC based reservation on the gateway for the OPNsense LAN interface of 192.168.1.3
I set static IP on LAN and WAN on OPNsense accordingly (dot 3 on LAN and dot 2 on WAN)
I set DHCP server up on the OPNsense LAN interface serving split scope 192.168.1.10 to 192.168.1.48 (to serve up for any servers and workstations – no overlap)
The gateway device is a pretty standard Fritz!Box 7530
I have not configured anything for IPv6 (never do) but I have accepted default settings of OPNsense for IPv6 as I think some of my devices may be using it

The upshot is that anything on the upstream side of OPNsense can't see or even ping anything on the downstream side, and vice versa, no matter what I set on the rules or firewall or NAT; and I've tried a lot of allow rules, even as I say, turning packet filtering and NAT off completely. (I've checked / changed over the cabling too).

Naively (you'll probably find this funny), suspecting it might be a lack of route, I added a route for 192.168.1.0/24, pointing it to the gateway device 192.168.1.1 and hey presto it completely cut off all access to the web GUI from the client laptop (LAN-side) and I had to do another full factory reset.

Annoyingly OPNsense seems to need a lot reboots to actually fit in - half the time the web GUI isn't available on either side, but pops up randomly after reboots).

Despite my probably apparent disappointment with this experience so far, I have a lot of respect and gratitude for open source software creators, so thanks to you all and to the community in advance of any help...

* - I suppose technically I bought a mini-PC with OPNsense pre-installed on it. The idea was to avoid having to set up a firewall from scratch, so I did not install OPNsense and at the moment I have no install media and little knowledge or inclination to start completely from scratch. The version I have is 24.7.11 and it's on a Zoostorm mini-PC with just two interface cards "re0 and em0" - so the cards differ

[meyergru]

I have a MAC based reservation on the gateway for the OPNsense WAN interface of 192.168.1.2
I have a MAC based reservation on the gateway for the OPNsense LAN interface of 192.168.1.3

Look there for your problem.

A router (such as OpnSense) is a device that can forward packets between different networks. I suggest that you read this first. Your problem is literally the first point in there.

Oh, and BTW: your setup is not simple, but one of the more complex and unpreferable setups (see point #4 in the link above).

Also, you have one Realtek NIC, which is problematic, too (see point #6).

[HH KD]

Thanks for the really quick reply meyergru!

I'm so pleased the post you're sending me to is longer than mine - I just can't seem to write anything in brief.

I will read it, and I'm sorry that I didn't find it (I did a ton of research, honest).

Hopefully you won't mind if I follow up here if I'm still stuck?

[HH KD]

Hello again meyergru.

Thanks again for your input on this. I've read all the articles behind the links you suggested and used google translate for the german ones - they really are very informative and I'm surprised your main one isn't pinned in some way or set to appear higher in search results. I suppose I should have just searched for "read this first".

I'm left thinking that what I wanted to do is somewhere between difficult and impossible. I've actually been told before that it was not possible, but that was when I tried to do the same with a Mikrotik RouterOS unit. I was very surprised it could not do it, but I didn't realise it was a general networking thing, rather than a Mikrotik thing.

What would you suggest then, bearing in mind the issue I might have with the realtec NIC too? Specifically regarding your point #4, my Fritz!Box 7530 does allow me to set up static routes. It also has a guest network with different IP range which I could use (though I'm not sure about routing between the two) and has DMZ capability.

From what I can tell my options are:

• Configure a distinct 'inner LAN' for my servers and operate the double-NAT router-behind-router option
• Configure a distinct inner LAN for my servers and use Fritz!Box static route to handle it
• Sell the Zoostorm because of the realtec NIC and try to do either of the above with the Mikrotik
• Trust in (and be frustrated by) the capabilities (and lack of capabilities) of the Fritz!Box alone

My ultimate aims were more ambitious than basic packet filtering, but still not too complex. I wanted to:

• inspect/log inbound 'hack' attempts so that I could block specific traffic with aliases (such as geo referenced IPs, entire ranges, specific domains, specific ports) - not possible on Fritz!Box
• set up VLANs (my switch is managed) to ease local traffic congestion - also not possible on Fritz!Box (except maybe using the guest LAN or a DMZ maybe?)
• set up an outbound WireGuard tunnel (to another domain of mine which also uses a Fritz!Box) - Fritz!Box can do inbound WireGuard but not outbound
• block certain types of bandwidth hogs such as advertising
• set up more robust static connectivity for certain devices such as IP TV

Will the realtec cause problems for these? (I bought the Zoostom with NIC from an ebay user who had run OPNsense on it for years, apparently).

Any direction would be helpful - just advice. I'll read up on the details myself.

Thanks

[meyergru]

I cannot tell if the realtek NIC will in fact pose any problems. That depends on the connected hardware and on whether you use the os-realtek driver. Results have been mixed in the past.

Setting up a router-behind-router scenario is not impossible, but complicates things. I prefer to use a modem/ONT only and let OpnSense handle routing and all.

[EricPerl]

Internet ->
  - DMZ
  - LAN
  - GUEST
is pretty basic.

I've seen:
Internet ->
  - DMZ
  -> LAN
And GUEST at either level. Essentially, using double NAT for the internal network.

It's not clear what you want to put behind the inner router. The DMZ is typically for servers that are targets of port forwards.
In your case, you have laptops and phones on the outer network. If they needed to access the servers behind the inner router, it could get cumbersome fast.

You can persist with the Realtek NIC for now (if it worked well for the previous owner - assuming you trust him/her). The workaround is to virtualize OPNsense under Proxmox. It's not that hard. There are a few videos on YT...

[passeri]

HH KD, you say your LAN is "effectively predefined" when the key statement is "I'd rather not be directed to just do it differently." I will suggest you do it differently but for 'reasons', not arbitrarily.

Apart from the addressing problem already pointed out (the same would have killed your effort with Mikrotik) your proposed setup will not offer you any effective security, unless nothing whatsoever on your WiFi needs more than the FritzBox provides. If that is the case then your DMZ should be in the same place.

It is implied that you have enough control over the FritzBox to do NAT. Does your Mikrotik device have WiFi (or which model is it)? If so, move it below the Opnsense as a bridged WAP and pass everything from FritzBox to Opnsense where you can actually configure stuff with a single set of rules and effective level of NAT. One potential problem is that with only one LAN interface on the Zoostorm you will immediately need VLANs to isolate LAN and WiFi from DMZ.

Do you need access from WAN or WiFi to LAN, or only to DMZ (as it should be)?