Good day!
I am an OPNSense (25.1.5_5-amd64) newbie after having used ClarkCOnnect to ClearOS for many years. After much study, and buying a book or two, I finally have one of my OPNSSense setups done and put into production in my home system/lab. I couldn't believe that I had my FreePBX SIP,PJSIP and AIX Trunks all working flawlessly and all my port forwards for my security cameras... IT WORKED! And all seemed much quicker with better throughput. However...
I have used Nebula VPN, (https://github.com/slackhq/nebula), for several years and it has worked perfectly as a LAN to LAN VPN under ClearOS; Setup port forward for the port you want to use, make sure you have a route set on your OPNSense (or ClearOS router), and make sure that firewalls are opened up the Nebula VPN device, (NOT installed on the OPNSense box), whether a Raspberry Pi or in my case Ubuntu, and things just worked until OPNSense. In my case, OPNSense has a Gateway of 1.30 and I have a route for X.X.0.1/24 to X.X.1.30.
I have not fully migrated both of my sites to OPNSense and one still has ClearOS running. On my one OPNSense site, I have port 80 redirection turned off, I am using https with TCP port 81 and doing everything I have searched and read about OPNSense not allowing port 80. From my windows laptop under ClearOS, (net 0.1/24) I can ping every device on the OPNSense LAN (1.1/24) without issue it seems. I can open up and mess with the OPNSense router from the other LAN (miles away BTW), and I can SSH into my linux devices and I can also RDP to those that allow it via the VLAN. Bot http to web config UI's is not working.
In essence, the VPN works but I cannot do anything like view my cameras which are all set to port 80, or log into my freepbx which is on port 80, or IP Phones etc... anything that uses port 80 and a webgui, I cannot get into via a web browser. Just can't be found or takes to long to load, etc.
Does anyone have any idea what I might be missing? How do I allow an http (port 80) to work. Do I need a firewall rule for port 80 or the VPN port 4242, (which is port forwarded to 1.30)? If so where would it go?
Thanks for any help or advice you might provide.
John
I need to remember to look at the logs... Here is what I have and I do not know hjow to fix it. I have tried multiple firewall rules, floating rules etc...
Interface Time Source Destination Proto Label
LAN 2025-05-03T13:00:40-07:00 192.168.1.165:80 192.168.0.221:62696 tcp Default deny / state violation rule
How do I fix this? Really would like to be able to HTTP to my other remote network. There has to be away to modify the default rules but there seems to be no way to do so.
Thanks
About to give up... It just seems to me as though OPNSense should allow a VPN appliance, which is what I have as I am not sure I made that clear, to act as a gateway which is on the LAN and route everything without issue.
I just can't get around the default deny rule. Ugh. Must be a way.
After HOURS of gogling and STFing these and other forums, I found the answer in the following forum post, the very last post!
https://forum.opnsense.org/index.php?topic=20425.0
Under my LAN rule to allow my remote 192.168.0.0/24 network to access my remote LAN of 192.168.1.0/24, I went to ADVANCED and set STATE TYPE to NONE.
I can now get to all of my remote web guis on the remote LAN!!
YEAHHHH!
John
FWIW, I had never heard of Nebula until today.
Are you using this: https://nebula.defined.net/docs/guides/unsafe_routes/ (https://nebula.defined.net/docs/guides/unsafe_routes/)?
Is that what you call your VPN device?
When I see a log entry with source port 80 (HTTP), it tells me it's a reply from the web server.
It somehow hits the LAN gateway when it should go back to the Nebula VPN device through which the request came (IP forwarding + NAT according to that page).
It looks to me like the reply packets are following a different path than the way they came in.
And because the FW never saw the TCP connection, you have to resort to "advanced" state management (or lack thereof).
It looks like a workaround to me. I suspect your VPN device is not setup properly (NAT missing?).
You might want to look into Wireguard when both sites are on OPN.
There's a guide to site-to-site VPN. No software required on the hosts...
@EricPerl
Thank-you so much for your reply! When moving from an 'easy' router firewall to one that is more complex, I have had to learn a lot! And still learning.
To answer your question, yes! That is the VPN I am using on a Ubuntu box running Nebula and other stuff hung on the LAN. Gateway and routes are all set and unsafe routes baked in the certificate. Just to note, I settled on this after seeking a VPN that get through CGNAT or other double or maybe even triple NATed technologies for my brother who has an off grid cabin in the woods in AZ. He has used several satellite ISPs and is now on StarLink. Anyway, Nebula seemed to work and work well in this scenario with his main home having a static ISP IP and being the 'Lighthouse". I also felt it easy enough to setup.. in a way. At least he could sign his certs and move them where they need to go.
ANYWAY, (I digressed), as far as I know it "should" be set up properly as it worked well without any issues under ClearOS. But in looking things over there is a misstep somewhere but the workaround works. In my firewall rules I tried to ensure that I specified the Gateway IP on the LAN instead of default thinking that would solve my problem.
Unless I can figure out what rule to add and where to get it to properly route back, (BTW, pings, SSH, RDP and other stuff always worked with my basic rule, but port 80 didn't), I am stuck with he work around. Since I have added two more LAN segments for HA and Video stuff, I will have to resign my cert and figure out how to get it to 'see' the other LANs on OPNSense.
If you like to ponder stuff, I'll try any rule that you think might fit the bill! I am out of ideas!
Thanks again for the reply!
John
PS. When all is setup at both places, I do intend to move over to one of the built in OPNSense VPNs... but for some reason there seems to be a lot more to them to get them setup.
I suspect 192.168.0.221:62696 maps to your Windows client (running Nebula) that's trying to access web_server at 192.168.1.165:80
You probably have a "Nebula VPN device" on 192.168.1.0/24 that's helping broker that access using these 'unsafe_routes'.
It should NAT what's received from the client so that traffic between the VPN device and web_server is symmetric (replies going back to the source).
As is, it appears the source of the requests is still 192.168.0.221, which your web_server doesn't know how to talk to, so it sends the frame to you LAN GW (192.168.1.1?) which would be why you are getting a FW log entry there.
FWIW, I'm not entirely sure how the FW rule solved your problem, but you may have added a route back to 192.168.0.0/24 via the VPN device per
QuoteIn my case, OPNSense has a Gateway of 1.30 and I have a route for X.X.0.1/24 to X.X.1.30.
That does not look by design to me... These replies from the web server follow a questionable route.
Thanks again for the thoughts!
The 221 IP is my laptop on the SL LAN trying to reach my freepbx on the Tuc LAN. My route on OPNSense on the Tuc LAN is to send everything bound for the 192.168.0.0/24 network (I added the dot 1 out of habit above), to the gateway (VPN) which is on the OPNSense LAN at 192.168.1.30. Standard route out to tell it where to go when an ip is bound for the SL LAN (0.0/24) from the Tuc LAN (1.0/24). The thought of adding a reverse route which leads back to the OPNSense on the VPN might be an idea.
Thanks!
OK, that confirms my suspicion on how the loop is closed.
IMO, the detour via OPN for reply traffic out of the web server is questionable.
And again, it's probably due to misconfiguration on your VPN device.
You might want to look at the traffic between the VPN device and the web server.
I believe the source should be NATed (source is VPN device based on decapsulated VPN traffic from client to VPN device).
Given traffic received on OPN, it appears the source is still the W11 laptop (no NAT/masquerade).
Thanks again Eric!
I have messed with it enough and I have started setting up Wireguard. Hard to be in two places at once but maybe I can keep the Nebula VPN tunnel going while setting up and starting up Wireguard.
John