Opnsense has internet access but clients do not?

[StarsAndBars]

Greetings All!

I have an issue where 24.1 itself has internet access and can download updates, new repositiories, etc. However, clients connected to this instance on the LAN interface do not have internet access. I have double and triple checked the settings on the clients, and neither DHCP nor compatible manual settings for the LAN will restore internet access for the clients. This has me thinking I need to check Firewall Rules and/or NAT settings, but I am relatively new to OPNSense and I am not quite sure where to start in diagnosing this. It was a fresh install of 24.1 and I just used the wizard to try and get up and running quicky. What and where should I be looking? I would have thought by default all LAN clients would have internet access, and I certainly have not tinkered with any settings in that regard. Any/all assistance here would really be appreciated! THANKS IN ADVANCE!

[cookiemonster]

Assuming "normal" OPN host with the minimum 2 interfaces, the wizard will set one for WAN and one for LAN. The LAN one will be by default allow clients connecting to it, normally via a switch, to get "out" to the internet.
The next step is to check you have both services DHCP enabled on it, and some way for clients to resolve names i.e. DNS. Some people confuse "no internet access" ie connectivity, with failure to resolve names, which successful navigation of the internet relies on. So, check DNS on OPN is set for your clients. https://docs.opnsense.org/manual/unbound.html

[pasha-19]

Knowing which interface is bound to WAN and LAN could also be an issue when I first installed OpnSense 23.x the LAN defaulted to igb0 that was labeled WAN on the unit hosting the router and WAN was assigned to igb1 that was labeled LAN on the unit hosting the router.  I reversed the defaulted WAN and LAN in the configuration so they matched what was printed on the unit.  igb2 and igb3 on my device where unused and labeled accordingly.  However their location on the device allowed me to extrapolate that the port labeled LAN was igb1 and the port labeled WAN was igb0 meaning matching up the ports to the console display, if attached, after boot to the actual ports became straight forward.   You based on your own report have the WAN's physical port properly identified already.

On a Dell N2024 switch one of three different switches I have been using cause me problems until I enabled routing on the switch.  Still have not fully figured out that issue because there is a second level of routing that can enabled on each interface too.  I believe I have the fully routed settings working.  I have an untested guess about the settings that would configure the Dell switch similar to a TP-Link level 2+ switch with what I believe is it's single level routing option turned off.  My other TP-Link Switch has no routing option it is an old Level 2 only switch.

[StarsAndBars]

I do only have two of the 6 available copper ports assigned. Bare metal install of 24.1 so there is no virtualization here. igb0 is assigned to WAN, and igb1 is assigned to LAN. That corresponds to the physical connections made. I am not even using a separate switch right now. igb0 is connected to the ISP device and is definitely the WAN connection as I have one of my available static IPs assigned to it. As I mentioned earlier, this installation of 24.1 has internet access as I am able to update the packages directly from that WAN connection. The only device connected to igb1 is a computer I am using a browser with to access the web interface of this 24.1 instance. This computer is properly assigned an IP address when using DHCP, and I tested it also using a manually assigned address. This instance of 24.1 is running unbound for DNS and I have even tried manually assigned public DNS servers to the client, such as 8.8.8.8 and 1.1.1.1 and still do not have internet access. I am really confused and stumped right now. As a sanity check, I wiped this hardware and installed PFSense, and it just works using the same two physical connections, so this is definitely some sort of configuration issue specific to OPNSense and I just don't understand what I am obviously missing here. Thanks for your responses and I look forward to further assistance on what to do next.

[cookiemonster]

I don't want to be the one saying "it should work" as it isn't very useful but, to cover the basic, you could perhaps explain the physical setup in case there are other elements involved like other routers, other dns servers (pi-hole for instance, and so on).
That is because after running the wizard or when setting from scratch, after assigning WAN and LAN, DHCP and DNS, clients "just work". No need to add firewall rules, the default ones block anyting coming into WAN unless it has an state from a LAN, and LAN defaults to allow all out. p.s. can you please define the "do not have internet access" problem? Is it DNS ok for the client (not static on it but via OPN) or no out even via direct IP bypassing DNS resolution?

[StarsAndBars]

Thanks for the reply. The physical layout is really simple, which is why I am so confused and frustrated. I agree, it "should just work" as other installations of OPNSense have worked just fine for me on other hardware. Just a few minutes ago, I wiped this box and re-installed PFSense and boom, it "just works" without further setting changes.

Therefore, it MUST be something that OPNSense is doing differently on this box. Nothing different but the change in OS installed results in a working internet connection for the client on igb1.

As far as the complete picture, here it is...

The hardware in question is a generic, "white box" intel i3 unit with a total of 6 copper ports. No SFP or other networking ports. There is nothing else involved in the equation here. No other DNS servers, nothing. The first copper port from left to right is igb0 and that is what the WAN address (statically assigned) is connected to (ISP modem). The second copper port from the left is igb1 and it is LAN. Again, the OPNSense instance itself has internet access, because I can go to 'firmware' and run an update and it will go out and grab available update packages without issue when I perform an update check.

When I say no internet access, what I mean is that any client connected to the LAN interface when the box is running OPNSense does not have an internet connection, despite getting a DHCP assignment correctly from the OPNSense instance. As I mentioned above, this only applies to the LAN connection. The WAN connection definitely has a working connection because OPNSense itself can go out and download updates without issue. I don't think it is a DNS issue on the LAN side either, because when I manually assign public DNS servers to the client connected to the LAN, it still cannot reach the internet.

Again, if I leave the identical physical connections and WAN IP address settings the same, and wipe OPNSense and install pfSense, not only does the pfSense OS have internet access, but the client connected to LAN (igb1) has internet access as well.

[cookiemonster]

Ok. Any clues in dmesg ? What about the firewall live view: do you see the client hitting it and the "default allow lan to any". You might want to enable temporarily for debugging in System > Settings > Logging to see more.

[CJ]

The hardware in question is a generic, "white box" intel i3 unit with a total of 6 copper ports. No SFP or other networking ports. There is nothing else involved in the equation here. No other DNS servers, nothing. The first copper port from left to right is igb0 and that is what the WAN address (statically assigned) is connected to (ISP modem). The second copper port from the left is igb1 and it is LAN. Again, the OPNSense instance itself has internet access, because I can go to 'firmware' and run an update and it will go out and grab available update packages without issue when I perform an update check.

How are you determining what is WAN and what is LAN?  Labels on the ports or something else?  IIRC, pfsense and OPNsense use the opposite defaults for port order.

[StarsAndBars]

Thanks to all who took the time to chime in on this. Here is an update...

On a hunch, I went and downloaded the 23.7 installer and put that on this box. Guess what? It worked instantly after running through the wizard. Both the firewall and LAN clients worked flawlessly. Therefore, I can say with confidence that this is definitely an issue with the 24.x installer currently available.

As an anecdote, I was able to then upgrade in place from a perfectly functional 23.7 environment to 24.x and that also worked.

So, for anyone else having this issue, revert back to a 23.7 installer and verify it works (it did for me) and then upgrade to 24.x after you are first up and running in 23.7

I really wish the OPNSense team would do more testing and validation.

[CJ]

I really wish the OPNSense team would do more testing and validation.

Does this mean you're going to test why 24.1 doesn't work on your system and submit a bug report regarding it?

There are a massive amount of combinations of hardware, environment, and configurations.  There's no way that all of them can be tested and validated.

[Greg_E]

I just scanned through the posts, but can you ping to an IP like 9.9.9.9 from both OPNsense and from one of your clients?

If so then DNS is probably the issue and you need to make sure that you are either using a DNS server separate from OPN and that forwarding is present, or need to check the box to use the OPN DNS as a server and check that forwarding is also present.