Topics

I've noticed a degradation in quality with the user experience in OPNSense.  The poorly designed (new) GUI's around IPSec, NAT, and now the DHCP server are one thing.  However, now I've also noticed that in the last few versions that just going through the setup wizard no longer results in a functioning system.  At least when it comes to using a static WAN address.

Just try it, download an image, assign your interfaces using the CLI, then open up a web browser.  Go through the setup wizard and experience how you get no internet connectivity on your client machines.  The OPNsense device it self can connect, but nothing behind it can.

You have to 'know' to go into your WAN interface and select your gateway because it defaults to "disabled", despite you being asked to enter it in the setup wizard.

I've been able to reproduce this multiple times, so please don't come back and say to me you that it works for you. :)

I noticed this odd behavior with the traffic graphs on the Lobby:Dashboard page.  I was able to observe it in multiple OPNSense firewalls of differing devices.  The version I am running (everywhere) is 25.1.10-amd64.

When the tab/window isn't in view, the stats drop down to 0.  Then when you return to the window it recovers back to it's normal rate.

I recently got my hands on a Mudi V2 (GL-E750V2) Portable 4G LTE Router.  The device has Wiregaurd capabilities (as well as OpenVPN).  Use Guide: https://docs.gl-inet.com/router/en/4/user_guide/gl-e750/

I'm still new to WG, but I have a good grasp on how it gets configured.  I would like to setup a "site-site" connection between this mobile router and an OPNSense device.  I am having trouble 'translating' the configuration concepts to the mobile router.  The OPNsense device has a static public WAN address.  The mobile router's IP will always be dynamic and may not always be public.  It could be sitting behind a NAT, it could be directly on the public Internet.  It depends where I am.

Has anyone had any experience with this device?  If so, could you share an example configuration?

I've embraced Wireguard and yes, it's good.  Not perfect, but it works good enough for the needs.  It's much closer to the PPTP style VPN protocol that was flawless and is a "Just Works" type of setup.  Unlike OpenVPN (which by the way, current documentation appears to be inaccurate).

There is one thing missing that would turn this into a "great" option.  With PPTP was that you had access to IPSec Tunnels "for free", meaning you didn't need to do anything extra so long as your firewall rules were in place.  Based on forum searches, the solution appears to be that I need to add manual SPD entries to my IPsec configuration.  However I do not see any documentation that describes the process.  Could someone kindly provide a step by step guide?  I have multiple IPSec tunnels and I can't risk breaking something and getting locked out from remote access.

Assume the following:

- 4 OPNSense devices.  One per site.  Some have multiple subnets: 192.168.0/24, 10.8.8/24 (A), 192.168.1/24, 10.8.7/24 (B), 192.168.7/24, 10.8.9/24 (C), 192.168.8/24 (D).
- Each site has a static public IP on the WAN side, and all 4 sites are interconnected using (legacy config) IPSec tunnels.
- Firewall rules are in place that allow all sites to communicate will all hosts in the subnets
- Site A is running a Wireguard server configured using OPNsense documentation for a road warrior setup.
- The Wiregaurd tunnel network is 192.168.20/24
- Site D IPSec is configured slightly different.  It's using a VTI for route a based tunnel to just site B.  This was an experiment that didn't produce any useful results.  If needed I can switch this to the regular policy based type setup.

Other things to note:

Site A also has a mobile IPsec VPN configured as per official docs.  It to lacks the ability to access remote subnets.

As per the page at https://docs.opnsense.org/manual/vpnet.html#migrating-from-tunnels-to-connections

1. Does it work exactly the same or will I see any loss/difference in functionality?
2. Do I need to change anything on the remote end?
3. If the IPSec tunnel is between two OPNSense gateways, what must I do to ensure I do not lose connectivity?

After apply the upgrade to the 24.7 series from 24.1, the DHCP service is refusing to assign an IP address to some clients.  I'm not sure how to troubleshoot this.  There are no entries in the logs.

UPDATE:

Looks like it's more than one client.  Some get a response, other's do not.

[What I have and currently works]

I just realized that I have been posting my question in the wrong forum, wondering why I'm not getting an answer.  :P.

What I have and currently works

I have a functional route based IPSec VPN tunnel using the instructions at https://docs.opnsense.org/manual/how-tos/ipsec-s2s-route.html.

Site A: LAN=192.168.1.0/24 WAN=1.2.3.4
Site B: LAN=192.168.8.0/24 WAN=4.3.2.1

Phase 2 Local address at Site A: 10.192.168.1
Phase 2 Local address at Site B: 10.192.168.8
Phase 2 Remote address at Site A: 10.192.168.8
Phase 2 Remote address at Site B: 10.192.168.1

Gateways added on each site, and static routes exists for making sure the 192.168.1.0/24 can talk to 192.168.8.0/24 and vice versa.

Good so far.

Context

For the purpose of simplification, there is a website http://showthisip.com that displays my public IP address.  That host name resolves to 63.247.147.167.

At Site A, when I visit that website from a client on the 192.168.1.0/24 subnet, I am shown the IP address 1.2.3.4, which is expected.  That's the public IP of the WAN interface of OPNsense at Site A.

At Site B, when I visit that website from a client on the 192.168.8.0/24 subnet, I am shown the IP address 4.3.2.1, which is expected.  That's the public IP of the WAN interface of OPNsense at Site B

What I want to do

I want, when I visit that website using a client on the 192.168.1.0/24 subnet at Site A, to show the IP address: 4.3.2.1.  The public IP of the WAN interface at Site B.  In other words, I would like to route traffic leaving Site A  for 63.247.147.167 through the VPN tunnel I setup previously so that http://showthisip.com thinks I am at Site B. 

I add a firewall rule on the IPsec interfaces (both sites) to allow all traffic in.  Then I proceed to add a static route at Site A:

Network Address: 63.247.147.167
Gateway: 10.192.168.8

I open my browser at Site A, type in the website address... and it times out.

I trace route and see:

Code Select Expand

traceroute -n 63.247.147.167
traceroute to 63.247.147.167 (63.247.147.167), 64 hops max, 40 byte packets
1  192.168.1.1  0.347 ms  0.272 ms  0.158 ms
2  10.192.168.8  151.799 ms  151.946 ms  151.913 ms
3  * * *
4  * * *
5  * * *


Am I missing another firewall rule?

I have been noticing my Internet dropping a lot.  At first I thought this was a problem with my ISP.  Turns out it's not them, it's me.  Specifically, OPNSense:

Code Select Expand


2024-06-07T14:31:39 Notice kernel KDB: enter: panic
2024-06-07T14:31:39 Notice kernel panic() at panic+0x43/frame 0xfffffe008565cac0
2024-06-07T14:31:39 Notice kernel vpanic() at vpanic+0x151/frame 0xfffffe008565ca60
2024-06-07T14:31:39 Notice kernel panic: page fault
2024-06-04T08:11:46 Notice kernel KDB: enter: panic
2024-06-04T08:11:46 Notice kernel panic() at panic+0x43/frame 0xfffffe00107aea50
2024-06-04T08:11:46 Notice kernel vpanic() at vpanic+0x151/frame 0xfffffe00107ae9f0
2024-06-04T08:11:46 Notice kernel panic: page fault
2024-04-22T15:28:49 Notice kernel KDB: enter: panic
2024-04-22T15:28:49 Notice kernel panic() at panic+0x43/frame 0xfffffe00107aea60
2024-04-22T15:28:49 Notice kernel vpanic() at vpanic+0x151/frame 0xfffffe00107aea00
2024-04-22T15:28:49 Notice kernel panic: page fault
2024-04-22T03:53:41 Notice kernel KDB: enter: panic
2024-04-22T03:53:41 Notice kernel panic() at panic+0x43/frame 0xfffffe00107aea60
2024-04-22T03:53:41 Notice kernel vpanic() at vpanic+0x151/frame 0xfffffe00107aea00
2024-04-22T03:53:41 Notice kernel panic: page fault
2024-03-01T12:53:33 Notice kernel KDB: enter: panic
2024-03-01T12:53:33 Notice kernel panic() at panic+0x43/frame 0xfffffe00107aea60
2024-03-01T12:53:33 Notice kernel vpanic() at vpanic+0x151/frame 0xfffffe00107aea00
2024-03-01T12:53:33 Notice kernel panic: page fault
2024-02-18T20:55:36 Notice kernel KDB: enter: panic
2024-02-18T20:55:36 Notice kernel panic() at panic+0x43/frame 0xfffffe008565cac0
2024-02-18T20:55:36 Notice kernel vpanic() at vpanic+0x151/frame 0xfffffe008565ca60
2024-02-18T20:55:36 Notice kernel panic: page fault
2024-02-18T04:22:37 Notice kernel KDB: enter: panic
2024-02-18T04:22:37 Notice kernel panic() at panic+0x43/frame 0xfffffe00107aebd0
2024-02-18T04:22:37 Notice kernel vpanic() at vpanic+0x151/frame 0xfffffe00107aeb70
2024-02-18T04:22:37 Notice kernel panic: page fault
2024-01-16T12:38:37 Notice kernel KDB: enter: panic
2024-01-16T12:38:37 Notice kernel panic() at panic+0x43/frame 0xfffffe00107aebd0
2024-01-16T12:38:37 Notice kernel vpanic() at vpanic+0x151/frame 0xfffffe00107aeb70
2024-01-16T12:38:37 Notice kernel panic: page fault


That's apparently been going on since I first installed the device (Protectli Vault).  Here's the full log of most recent panic:

Code Select Expand

2024-06-07T14:31:39 Notice kernel FreeBSD clang version 14.0.5 (https://github.com/llvm/llvm-project.git llvmorg-14.0.5-0-gc12386ae247c)
2024-06-07T14:31:39 Notice kernel FreeBSD 13.2-RELEASE-p7 stable/23.7-n254871-d5ec322cffc SMP amd64
2024-06-07T14:31:39 Notice kernel FreeBSD is a registered trademark of The FreeBSD Foundation.
2024-06-07T14:31:39 Notice kernel The Regents of the University of California. All rights reserved.
2024-06-07T14:31:39 Notice kernel Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
2024-06-07T14:31:39 Notice kernel Copyright (c) 1992-2021 The FreeBSD Project.
2024-06-07T14:31:39 Notice kernel ---<<BOOT>>---
2024-06-07T14:31:39 Notice kernel KDB: enter: panic
2024-06-07T14:31:39 Notice kernel --- trap 0x5424a868, rip = 0x89133a23244e489d, rsp = 0xd149a69345269a6c, rbp = 0xbb3576f2ede5dbc ---
2024-06-07T14:31:39 Notice kernel fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe008565cf30
2024-06-07T14:31:39 Notice kernel fork_exit() at fork_exit+0x7e/frame 0xfffffe008565cf30
2024-06-07T14:31:39 Notice kernel ithread_loop() at ithread_loop+0x25a/frame 0xfffffe008565cef0
2024-06-07T14:31:39 Notice kernel softclock() at softclock+0x79/frame 0xfffffe008565ce60
2024-06-07T14:31:39 Notice kernel softclock_call_cc() at softclock_call_cc+0x133/frame 0xfffffe008565ce40
2024-06-07T14:31:39 Notice kernel nd6_llinfo_timer() at nd6_llinfo_timer+0x478/frame 0xfffffe008565cd90
2024-06-07T14:31:39 Notice kernel __rw_wlock_hard() at __rw_wlock_hard+0x158/frame 0xfffffe008565ccf0
2024-06-07T14:31:39 Notice kernel --- trap 0xc, rip = 0xffffffff80c84288, rsp = 0xfffffe008565cc50, rbp = 0xfffffe008565ccf0 ---
2024-06-07T14:31:39 Notice kernel calltrap() at calltrap+0x8/frame 0xfffffe008565cb80
2024-06-07T14:31:39 Notice kernel trap_pfault() at trap_pfault+0x4f/frame 0xfffffe008565cb80
2024-06-07T14:31:39 Notice kernel trap_fatal() at trap_fatal+0x387/frame 0xfffffe008565cb20
2024-06-07T14:31:39 Notice kernel panic() at panic+0x43/frame 0xfffffe008565cac0
2024-06-07T14:31:39 Notice kernel vpanic() at vpanic+0x151/frame 0xfffffe008565ca60
2024-06-07T14:31:39 Notice kernel db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe008565ca10
2024-06-07T14:31:39 Notice kernel KDB: stack backtrace:
2024-06-07T14:31:39 Notice kernel time = 1717770612
2024-06-07T14:31:39 Notice kernel cpuid = 2
2024-06-07T14:31:39 Notice kernel panic: page fault
2024-06-07T14:31:39 Notice kernel trap number = 12
2024-06-07T14:31:39 Notice kernel current process = 12 (swi4: clock (0))
2024-06-07T14:31:39 Notice kernel processor eflags = interrupt enabled, resume, IOPL = 0
2024-06-07T14:31:39 Notice kernel = DPL 0, pres 1, long 1, def32 0, gran 1
2024-06-07T14:31:39 Notice kernel code segment = base 0x0, limit 0xfffff, type 0x1b
2024-06-07T14:31:39 Notice kernel frame pointer         = 0x28:0xfffffe008565ccf0
2024-06-07T14:31:39 Notice kernel stack pointer         = 0x28:0xfffffe008565cc50
2024-06-07T14:31:39 Notice kernel instruction pointer = 0x20:0xffffffff80c84288
2024-06-07T14:31:39 Notice kernel fault code = supervisor read data, page not present
2024-06-07T14:31:39 Notice kernel fault virtual address = 0x440
2024-06-07T14:31:39 Notice kernel cpuid = 2; apic id = 04
2024-06-07T14:31:39 Notice kernel Fatal trap 12: page fault while in kernel mode
2024-06-07T14:31:39 Notice syslog-ng syslog-ng starting up; version='4.4.0'


I'm not sure what to make of this, but it brings back bad memories from a few years ago when I was having the same exact problem with a different device.

I can try upgrading to 'the latest version', but will that actually help? (it didn't a few years ago).

I know this is probably a question that has been asked an answered multiple times.  Problem is that my searches on this forum are not producing any results.  I think it's because I am probably not working it correctly.

I have 2 OPNSense boxes:

- Site A: IP of WAN: 1.2.3.4
- Site B: IP of WAN: 9.8.7.6

There is a site-to-site tunnel created between the two.  LAN clients can communicate with each other.  Good so far.

I would like to have all LAN clients in Site A that connect to a specific public IP address (or host name if possible), appear as if they are in Site B.

In other words, a LAN client in Site A opens the page http://showthisip.com and see's 9.8.7.6, but only for that page.  If I go to http://seethisip.com (assume for the moment for the purpose of this example that the two sites resolve to different IP's), I will see 1.2.3.4

Am I correct that if I follow the instructions on one of these paged, I will achieve the above?
https://docs.opnsense.org/manual/how-tos/ipsec-s2s-route.html
https://docs.opnsense.org/manual/how-tos/ipsec-s2s-conn-route.html

Which one should I follow?

[Setup Overview]

Setup Overview

As seen in the diagram below I have two offices.  Office 1 has a single LAN (192.168.0.0/24) and Office 2 has a single LAN (192.168.8.0/24).  The two offices are connected to each other using a site-site IPSEC VPN tunnel using OPNSense A and C.



Office 1 has two ISP's.  OPNSense B is connected to the second ISP and there are some hosts in Office 1 that use OPNSense B as the default gateway.

Problem

I would like to make it so 192.168.0.50 can communicate with 192.168.8.2 and 192.168.8.2 can communicate with 192.168.0.50 without having to configure a static route on 192.168.0.50.

Attempts So Far

I created a static route entry on OPNSense B to send anything for 192.168.8.0/24 to 192.168.0.100



This does not work as expected.  The hosts can ping each other, but no network traffic other than that makes it through.  For example, HTTP does not work between them.

Firewall rules for LAN on all OPNSense's:



For the IPSec interface on OPNSense A and C:

Had a little catastrophe/accident on the storage level, but the system boots, but shows signs of corruption, then kernel panics.  I can access single user mode and see that the image appears to be damaged, conf files looks intact.  What is the best way to approach recovery?

This is a problem I have struggled with since the days of m0n0wall.  Here's a realistic (yet completely made up) scenario:

There are two OPNSense devices, each with there own public IP and private LAN's. Each have the LAN port connected to their own isolated network switch, and each have the WAN port connected to the ISP modems/routers.  Lets call them WORK and PLAY

- The LAN subnet for WORK is 172.16.22.0/24
- The LAN subnet for PLAY is 192.168.0.0/24
- There is a computer called OFFICE-PC connected to WORK LAN
- There is a computer called GAME-PC connected to PLAY LAN
- Both OPNSense devices have a (third) extra Ethernet port that is unused

What are my options and what steps must I take (assuming there are no firewalls) so that OFFICE-PC can talk to GAME-PC and vice a versa?

Unfortunately it's still not possible to do without a PPTP server.  It's simply not feasible to use OpenVPN, Wireguard, Tinc, etc without having to install additional software on OS's, devices, etc.

So I have resorted to setting up a FreeBSD VM that's running mpd5.

What do I need to do on the OPNSense Firewall to allow traffic to the PPTP server?  I already did the usual NAT port forwarding for TCP port 1723 and the GRE protocol.  While I can "connect" it's failing on LCP paramater negotiation "LCP: parameter negotiation failed".  I suspect the auto-created firewall rule is wrong?

This isn't a post complaining about the removal of PPTP.

I had to replace t1n1wall with a new device that uses OPNsense.  The PPTP VPN which was available in t1n1wall allowed me to easily setup the VPN server to provide access to all the networks, including IPSec tunnels, and WAN/Internet that were configured on it.  This was very useful in that I could VPN into my home network and access the Internet as if I was home, and access all my remote offices without having to do any additional setup on my part.

I'm having trouble replicating this behavior with IPSec mobile VPN (https://docs.opnsense.org/manual/how-tos/ipsec-rw.html#vpn-compatibility).  I can barely access my home network, let alone any remote offices.  I haven't even begun to think about "remote" Internet access. I'd prefer not to use OpenVPN since it requires installing extra stuff on the client side.

Are there any instructions on how to do this?

Should I be creating my own firewall rules when setting up Mobile VPN (as per https://docs.opnsense.org/manual/how-tos/ipsec-rw-srv-rsamschapv2.html) or should I rely on the rules that get automatically created?

It looks like OPNsense will automatically create the following rules (screenshot):



However, these appear to not work.  VPN clients are unable to connect unless I create the following rules myself:

Is the certificate manager UI (system_camanager.php) based on an existing PHP application/library or was it custom built for OPNsense?  It's got some good enough functionality for a standalone app and was wondering if I should copy that or go directly to the source.

I think there should be some indication that the option labeled AES128 is the same as if you where selecting AES.  This can help in cases when devices (like other m0n0wall derivatives) still label the option as "AES".  Especially now since the DES and 3DES options are no longer possible on newer OPNSense releases.

Is it just me or is IPSEC not reliable on the 23 series?  I have multiple virtual appliances and can without doubt see a difference between 21 and 23.   If I have two 23.x appliances hooked up using site-site IPSEC, the tunnel is always breaking and I have to manually restart it.  If I have two 21.x appliance hooked up the tunnel is always working.  Mixing the two results in the tunnel breaking.  This problem started with 22.x becuase I also have a 22.x instance currently having the same problem, and I remember very well that's exactly when I first saw this issue.

[Example]

I would like to advertise custom routers using DCHP, however I am not sure which options I should use.

Example

• The default router given by the DHCP server is 192.168.99.100
• I have a subnet 10.0.0.0/16 that I would like clients to route using 192.168.99.1

According to various sources I cam across online, I would create a "string" entry under "Additional Options" in the DHCP server settings, and use option 121.  The format of which is as follows:

• The subnet width 16 converts to 10 in HEX
• The destination network's significant octets 10.0 converts to 0A:00 in HEX
• The router 192.168.99.1 converts to C0:A8:63:01 in HEX

Therefor the value of the field is: 10:0A:00:C0:A8:63:01


I saved changes and restarted the DHCP service.  I then rebooted a client machine.  The machine did not receive the route.

Code Select Expand

netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.99.100     UGS        vmx0
127.0.0.1          link#2             UH          lo0
192.168.99.0/24    link#1             U          vmx0
192.168.99.86      link#1             UHS         lo0


Removing the extra 00's so that the value is "10:0A:C0:A8:63:01" as suggested by some sources does not change the behavior.

Using 249 as the option number also does nothing.  With this option, the 00 is required, otherwise it fails.

It's only when I use both 121 and 249 do I get the route (it has to be in that order, otherwise it doesn't work):


Unfortunately looses the default route:

Code Select Expand

netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
10.0.0.0/16        192.168.99.1       UGS        vmx0
127.0.0.1          link#2             UH          lo0
192.168.99.0/24    link#1             U          vmx0
192.168.99.86      link#1             UHS         lo0


I would like to change an existing firewall/router that is running Pfsense to OPNsense.  I'm not looking for automatic migration. 

I will be manually re-configuring the new firewall by having the two running simultaneously, but isolated from each other using VLAN's on my switch.  Then once everything is setup I do a cut-over by swapping VLAN's on the switch. 

I've done this many times, moving to and from other firewalls (m0n0wall, t1n1wall, smallwall, opnsense, and pfsense).  Usually moving between different firewall platform's isn't a major undertaking, even with the vast amount of rules, and configurations I have in place. 

In this specific instance I am using this firewall as an OpenVPN server.

How do I migrate the OpenVPN configuration from Pfsense to OPNsense without having to reconfigure each client?