Topics

[This setup only handles IPv4!]

There are lots of (mostly incomplete) instructions on how to setup NordVPN on OpnSense scattered all over the forum. Also, there are instructions from NordVPN themselves, but they do not focus on Wireguard.

Also, I found that my setup failed a few days ago, because NordVPN obviously cancelled one of their rented servers (de1019.nordvpn.com) which happened to be the one I used until then.

Therefore, I took the effort to create a setup with a failover for a fixed set of clients. This setup also will behave as a "kill switch", i.e. that no client targeted by it can bypass the NordVPN setup and expose itself accidentally.

1. You have to create a so-called "Access Token" in the NordVPN customer dashboard by clicking on NordVPN on the left. Then, the link to enter the access token menu will be at the bottom of the page. Create a token and be sure to note it, because it will not be accessible later on.

2. Once you have this token, you will need to create a private key from it. This can be done via 'curl -s -u token:<YOUR_TOKEN_HERE> https://api.nordvpn.com/v1/users/services/credentials | jq -r .nordlynx_private_key'.

3. For entry into WG instances, you will also need the corresponding public key, which you can get via 'echo <YOUR_PRIVATE_KEY_HERE> | wg pubkey'.

4. If you followed along, you should now be in possession of a private/public key pair pertaining to your NordVPN account.

5. Using this information, you can now add two Wireguard instances under VPN:Wireguard->Instances. Add these as "NORDVPN_INSTANCE1" and "NORDVPN_INSTANCE2", use different, free "Listen Ports" (these can be arbitrary, but must not be used anywhere else) and 10.5.0.2/16 as "Tunnel address" (yes, both are the same!). Be sure not to mix up the private and public keys. Both instances can, but do not strictly have to use the same key pair, should you have access to another one.

6. Obtain two NordVPN server names from their API via 'curl -s "https://api.nordvpn.com/v1/servers/recommendations?&filters\[servers_technologies\]\[identifier\]=wireguard_udp&limit=2" | jq -r ".[]|.hostname"'. This will give you two names like "xx999.nordvpn.com" for servers that are near you.

7. Next, create two Wireguard peers under VPN:Wireguard->Peers. Name them NORDVPN_PEER1 and NORDVPN_PEER2, and choose different NordVPN server names obtained in the previous step for both peers (e.g. 'deXXX.nordvpn.com'). Use "public key" = "m0tej5P6pYfBivkJc8yRV4KqQXmM81AChLlzlsOSjSs=" (this happens to be the public key for all NordVPN servers), "Allowed IPs" = "0.0.0.0/0", "Endpoint port" = 51820, "Keepalive Interval" = 30. Choose NORDVPN_INSTANCE1 as Wireguard instance for NORDVPN_PEER1 and NORDVPN_INSTANCE2 for NORDVPN_PEER2 - use only one per each peer!

8. Next, you can check VPN:WireGuard:Status. You should see both the peer and the instance come up as green for both instance/peer pairs if Wireguard is enabled.

9. Now, assign those two Wireguard interfaces names via Interfaces:Assignments. Name them NORDVPN1 and NORDVPN2. Select each of them under Interfaces and check the "Enable Interface" and "Prevent Interface Removal" boxes.

10. Create an alias with the definition of all clients, that should go through NordVPN only. Name that NORDVPN_CLIENTS. I prefer to use a network group alias that includes MAC aliases for individual machines, see also the hint about IPv6 at the end of this article.

11. Create two SNAT rules for each of the two NordVPN interfaces under Firewall:NAT:Source NAT with:

Code Select Expand

Description = "NAT for NordVPN"
Interface = NORDVPN1 / NORDVPN2
Version = IPv4
Protocol = any
Source Address = NORDVPN_CLIENTS
Translate Source IP = leave empty
12. Create two Gateways under System:Gateways:Configuration with

Code Select Expand

Name = NORDVPN_GW1
Interface = NORDVPN1
Address Family = IPv4
IP Address = 10.5.0.1
Far Gateway = checked
Failover States = checked
Disable Gateway Monitoring = unchecked
Monitor IP = 103.86.96.100
Description = IPv4 Gateway for NordVPN


Code Select Expand

Name = NORDVPN_GW2
Interface = NORDVPN2
Address Family = IPv4
IP Address = 103.86.99.100
Far Gateway = checked
Disable Gateway Monitoring = unchecked
Monitor IP = 103.86.99.100
Description = IPv4 Gateway for NordVPN

The "IP address" of 103.86.99.100 in the second entry is not an error. It is a workaround because you cannot use the same gateway IP twice. WireGuard does not actually use this IP for routing.
Leave everything else on default.

13. Create a gateway group under System:Gateways:Group named NORDVPN_FAILOVER with "Trigger Level" = "Member Down", having NORDVPN_GW1 as Tier 1 and NordVPN_GW2 as Tier 2, with anything else as tier "Never". After having them created, both gateways should be listed as active (green) here.

14. Finally, create a firewall rule under Firewall:Rules[new] (it should precede most other rules):

Code Select Expand

Description = Force gateway for NordVPN clients
Interface = any
Quick = checked
Action = Pass
Direction = In
Version = IPv4
Protocol = any
Invert Source = unchecked
Source = NORDVPN_CLIENTS
Source Port = any
Invert Destination = checked
Destination = RFC1918
Destination Port = any
Gateway = NORDVPN_FAILOVER

This assumes that you have an RFC1918 alias. You could also use an inverted firewall group containing all of your local networks. What this is supposed to mean is "internet traffic", with the exception of "local traffic", should you want to access your NORDVPN_CLIENTS from any of your local networks. By explicitely setting a gateway for this rule, you effectively implement policy-based routing (PBR).



If you want to try the failover, disable the first Wireguard peer. You will see the gateway failing and after a few seconds, the client traffic will go over the second connection. Re-enable the peer and soon afterwards, the traffic will be routed over the first connection again.
You can verify this when you check the change of your visible external IP with e.g. https://www.whatismyip.com/ from one of your NordVPN clients.

Note: This setup only handles IPv4! Currently, NordVPN does not support IPv6 anyways. If you want to make sure that your NordVPN clients will not expose themselves via IPv6, disable it completely on those clients and / or block IPv6 completely via a firewall rule for NORDVPN_CLIENTS (hint: you can use the MAC addresses instead of IPs to define the alias, which will work for bot IPv4 and IPv6!).

P.S.: By default, OPNsense can take up to 60 seconds to detect a gateway failure and switch over. To make the failover happen faster, apply this adjustment:

Go to System:Gateways:Configuration, edit both gateways (`NORDVPN_GW1` and `NORDVPN_GW2`), scroll down, click "Advanced" under *Weight / Intervals* and change the values for both gateways to:
  - "Ping Interval" = 1 (pings every second)
  - "Time Period" = 3 (calculates average over 3 seconds)
  - "Down Interval" = 3 (marks the gateway as down after 3 missed pings)

[This setup only handles IPv4!]

There are lots of (mostly incomplete) instructions on how to setup NordVPN on OpnSense scattered all over the forum. Also, there are instructions from NordVPN themselves, but they do not focus on Wireguard.

Also, I found that my setup failed a few days ago, because NordVPN obviously cancelled one of their rented servers (de1019.nordvpn.com) which happened to be the one I used until then.

Therefore, I took the effort to create a setup with a failover for a fixed set of clients. This setup also will behave as a "kill switch", i.e. that no client targeted by it can bypass the NordVPN setup and expose itself accidentally.

1. You have to create not one, but two so-called "Access Token" in the NordVPN customer dashboard by clicking on NordVPN on the left. Then, the link to enter the access token menu will be at the bottom of the page. Create two different tokens and be sure to note them, because they will not be accessible later on.

2. Once you have those tokens, you will need to create a private key for both of them. This can be done via 'curl -s -u token:<YOUR_TOKEN_HERE> https://api.nordvpn.com/v1/users/services/credentials | jq -r .nordlynx_private_key'.

3. For entry into WG instances, you will also need the corresponding public keys, which you can get via 'echo <YOUR_PRIVATE_KEY_HERE> | wg pubkey'.

4. If you followed along, you should now be in posession of two private/public key pairs pertaining to your NordVPN account.

5. Using these informations, you can now add two Wireguard instances under VPN:Wireguard->Instances. Add these as "NORDVPN_INSTANCE1" and "NORDVPN_INSTANCE2", use different, free "Listen Ports" (these can be arbitrary, but must not be used anywhere else) and 10.5.0.2/16 as "Tunnel address" (yes, both are the same!).

6. Obtain two NordVPN server names from their API via 'curl -s "https://api.nordvpn.com/v1/servers/recommendations?&filters\[servers_technologies\]\[identifier\]=wireguard_udp&limit=2" | jq -r ".[]|.hostname"'.

7. Next, create two Wireguard peers under VPN:Wireguard->Peers. Name them NORDVPN_PEER1 and NORDVPN_PEER2, and choose different NordVPN server names obtained in the previous step for bot peers (e.g. 'deXXXX.nordvpn.com'). Use "public key" = "m0tej5P6pYfBivkJc8yRV4KqQXmM81AChLlzlsOSjSs=", "Allowed IPs" = "0.0.0.0/0", "Endpoint port" = 51820, "Keepalive Interval" = 30. Choose NORDVPN_INSTANCE1 as Wireguard instance for NORDVPN_PEER1 and NORDVPN_INSTANCE2 for NORDVPN_PEER2 - use only one per each peer!

8. Next, you can check VPN:WireGuard:Status. You should see both the peer and the instance come up as green for both instance/peer pairs if Wireguard is enabled.

9. Now, assign those two Wireguard interfaces names via Interfaces:Assignments. Name them NORDVPN1 and NORDVPN12 Select each of them under Interfaces and check the "Enable Interface" and "Prevent Interface Removal" boxes.

10. Create a "Host" alias with the definition of all clients, that should go through NordVPN only. Name that NORDVPN_CLIENTS.

11. Create two SNAT rules for each of the two NordVPN interfaces under Firewall:NAT:Source NAT with:

Code Select Expand

Description = "NAT for NordVPN"
Interface = NORDVPN1 / NORDVPN2
Version = IPv4
Protocol = any
Source Addres = VPN_CLIENTS
Translate Source IP = leave empty
12. Create two Gateways under System:Gateways:Configuration with

Code Select Expand

Name = NORDVPN_GW1
Interface = NORDVPN1
Address Family = IPv4
IP Address = 10.5.0.1
Far Gateway = checked
Disable Gateway Monitoring = unchecked
Monitor IP = 103.86.96.100
Description = IPv4 Gateway for NordVPN


Code Select Expand

Name = NORDVPN_GW2
Interface = NORDVPN2
Address Family = IPv4
IP Address = 103.86.99.100 [i](Note: This is not an error, but a trick!)[/i]
Far Gateway = checked
Disable Gateway Monitoring = unchecked
Monitor IP = 103.86.99.100
Description = IPv4 Gateway for NordVPN
Leave everything else on default.

13. Create a gateway group under System:Gateways:Group named NORDVPN_FAILOVER with "Trigger Level" = "Member Down", having NORDVPN_GW1 as Tier 1 and NordVPN_GW2 as Tier 2, with anything else as tier "Never". After having them created, both gateways should be listed as active (green) here.

14. Finally, create a firewall rule under Firewall:Rules[new]:

Code Select Expand

Description = Force gateway for NordVPN clients
Interface = any
Quick = checked
Action = Pass
Direction = In
Version = IPv4
Protocol = any
Invert Source = unchecked
Source = NORDVPN_CLIENTS
Source Port = any
Invert Destination = checked
Destination = RFC1918
Destination Port = any
Gateway = NORDVPN_FAILOVER

This assumes that you have an RFC1918 alias. You could also use an inverted firewall group containing all of your local networks. What this is supposed to mean is "internet traffic", with the exception of "local traffic", should you want to access your NORDVPN_CLIENTS from any of your local networks.



If you want to try the failover, disable the first Wireguard peer. You will see the gateway failing and after a few seconds, the client traffic will go over the second connection. Re-enable the peer and soon afterwards, the traffic will be routed over the first connection again.
You can verify this when you check the change of your visible external IP with e.g. https://www.whatismyip.com/ from one of your NordVPN clients.

Note 1: This setup only handles IPv4! If you want to make sure that your NordVPN clients will not expose themselves via IPv6, disable it completely on those clients.

Note 2: Since the setup does not keep VPN clients to use any DNS service they like, you can still have DNS leaks. These may be due to DoT, DoH or even because the DHCP settings for your clients tell them to use OpnSense with any kind of DNS upstream.
 

[VPN: IPsec: Connections]

Hi everyone,


I usually prefer Wireguard for its simplicity, but I found that some ISPs block it using Deep Packet Inspection (even for the purpose of fighting copyright violations). IPsec, being the more "enterprise" VPN protocol, is less often blocked, so it is handy to have a fallback.

While setting up an IKEv2 EAP-MSCHAPv2 Roadwarrior connection using the modern VPN: IPsec: Connections module according to the official OPNsense Roadwarrior (swanctl) Documentation, you might run into situations where the connection seems established on the firewall (swanctl --list-sas), but 0 packets / 0 bytes are being transmitted.

To save you hours of structural troubleshooting on the FreeBSD kernel or routing layers, here is a definitive list of bullet points on what actually causes issues with modern iOS/macOS clients—and what you can safely ignore.

⚠️ The Real Problems (What you must avoid / fix)

• Avoid Manual Profile Configuration (The DNS Trap)
  
  • The Issue: Typing the VPN credentials directly into the native iOS VPN settings menu.
    
  • The Impact: iOS manually configured profiles strictly ignore the DNS Configuration Payload sent by the server. This happens regardless of whether you use Split Tunneling or Full Tunneling (0.0.0.0/0). Internal DNS resolution will be completely broken.
    
  • The Fix: You must deploy the configuration via a .mobileconfig file to explicitly inject the DNS structure into the Apple network stack.
    You can create such profiles with the free Imazing app.
    
• Avoid "long" Server Certificate Lifetimes
  
  • The Issue: Creating a server certificate with a really long validity period, like more than a year.
    
  • The Impact: The Apple crypto subsystem silently and rigidly rejects any server certificate with too long lifetimes exceeding.
    
  • Note: While this rule can technically be bypassed by backdating the "Not Before" timestamp years into the past via OpenSSL CLI commands, the OPNsense GUI does not allow backdating (it always forces creation date to time()). Therefore, stick to ACME/Let's Encrypt (90 days) or issue short-lived certificates.
    Note: This will not be any problem when using ACME-type certificates. Also, the System : Trust : Certificate dialogue will preset the certificate lifetime with 397 days for this very reason.
    
• Avoid Missing Server Authentication EKU
  
  • The Issue: Generating the server certificate as a generic or unconstrained type.
    
  • The Impact: The certificate must explicitly contain the Extended Key Usage (EKU) attribute TLS Web Server Authentication. Without it, iOS drops the connection instantly during the IKE_AUTH phase. This is also done automatically in System : Trust : Certificate if you select "Server Authentication".
    
• Avoid Non-RSA certificates
  
  • The Issue: Generating the server certificate as an EC certificate.
    
  • The Impact: The certificate must be an RSA certificate to be recognized. This is also done automatically in System : Trust : Certificate per default.
    

ℹ️ The Cosmetic Illusion (Do not judge the connection by this)

• The Missing VPN Icon
  
  • The Reality: In current iOS versions, the VPN icon in the status bar standardly disappears after a successful IKEv2 handshake. Do not assume a vanished icon means a routing failure, a bad configuration, or a silent disconnect. The tunnel remains fully active (ESTABLISHED / INSTALLED in swanctl), even when the icon is missing. The status display is not tied to a failing DNS check.
    

🚫 Mythbusting (What is NOT the problem)

If your tunnel is up but registers 0 packets on active SAs, do not waste your time troubleshooting the following theoretical network pitfalls, as iOS handles them perfectly fine:

• Overlapping / Supernet Routing: iOS handles broad local traffic selectors like 192.168.0.0/16 flawlessly alongside CGNAT or carrier cellular networks. However, use a disjoint network for your IP pools.
  
• MTU sizes: IP fragmentation is handled correctly, but AI agents will often misinterpret log messages as to try to make the
  
• Multiple LocalNets / Comma-separated Lists: The native Apple client parses multiple distinct local networks in the traffic selector correctly.
  
• Strict SAN / Wildcard Validation: Wildcard certificates (*.domain.tld) or strict Subject Alternative Name (SAN) match anomalies are not the cause of 0-byte transmission stalls.
  
• IPv6 problems: There are none. AI bots may also misinterpret mixed IP adressing for the connection itself and the tunnel IPs, but that is not a problem.
  

Summary for a working setup:
Follow the official documentation, make sure your certificate is short-lived with the correct Server-EKU, ignore the missing status bar icon, and deploy the client configuration exclusively via a tailored .mobileconfig profile to get proper DNS access.

[they caused themselves]

Background: I had a problem with a remote Plex access to my network earlier this month. This problem resurfaced today and now, I have found what eventually caused it.

The client was trying to access from a Deutsche Glasfaser account with IP 93.104.119.235 (nothing private to that, because it is CG-NAT).
As it turns out, at the time of writing, this IP is listed at Q-Feeds. It also was listed at Blocklist.de, because of IMAP attacks on 1st of May.

You cannot view this attachment.

Ironically, this was caused by a prolonged outage of Vodafone's IMAP services. It seems that they also notified some blacklisting services because of failed IMAP attempts that they caused themselves. I already reported this as a false positive, but the IP above is only one example, so in case you see problems, now you know... Q-Feeds should dismiss all reports on this IMAP problem.


Proof:

https://forum.vodafone.de/t5/MeinVodafone-E-Mail/Sammelthread-IMAP-Problem-mit-Vodafone-E-Mail-Konto/td-p/3329568


P.S.: There is no contradiction, as using a free e-mail account on one of Vodafone's domains but still having any other access provider is perfectly fine....

[TL;DR: The newer Realtek chips RTL8126 and RTL8127 do not run reliably on older hardware that "supports" PCIe 4.0.]

Well, you could say I should have known better - yet, I tried:

Recently, I saw an article about the new RealTek RTL8127 NIC. On paper, it looks fine: 10/5/2.5/1 Gbps, low power draw and PCIe 4.0 x1 interface, which makes it easy to fit into any mainboard - or so it seems.

Main main reason to use it were the shortcomings of other high speed RJ45 adapters, like the X550-T1. That one:

- uses much more power
- needs a PCIe 3.0 x4 slot
- cannot handle autonegotiation on some OSes

Because I have not-so-good CAT.5E cabling, I like to use 5 Gbps, but with newer Windows drivers, you cannot even fix speeds at NBase-T (5 or 2.5 Gobs), so you are mostly stuck with 1 or 10 Gbps only. Linux is a prominent example where the driver allows intermediate speeds, see this for details.

I already had a Realtek RTL8126, which can do 5 Gbps at most, but that one had a quirk: On cold boots, it was sometimes not detected at all. Only a power-cycle would help. The internet is full of discussions about that. So, I went for it and bought the RTL8127...

...only to find that it had the same quirk.

But now comes the hard part: My Windows 11 installation has gone awry in the last few weeks. Basically, I could not start Steam and Ollama any more. Nothing I did could repair it.

On I went with a new installation from scratch and restoration of my user files afterwards. Then, I had to install every application again.
System worked fine for ~24 hours, then I noticed strange things happening: My desktop icons flashed ever 0.5 seconds and I could no longer set the default browser. No repairs worked here, either.

Since I still had the profile backup, I decided to repeat the whole process. While I was doing that, I tried to relocate the RTL8127 into another PCIe slot. This time, the restoration of my user profile showed 3 CRC errors. I was quite sure there were no problems on the source.


So, the plot thickened: Obviously, in my X570 board, the PCIe 4.0 slots attached to the chipset have problems. The X570 chipset was the first to use PCIe 4.0, so the implementation may be flaky. At the time, nobody would have noticed, because most PCIe cards only supported PCIe 3.0 anyway.

Now, the problem manifests in two ways:

1. The NICs may sometimes go undetected during cold start PCIe training.
2. The data that is transferred to the mainboard can get corrupted. This became very obvious when I installed some 500 GByte of software and data over the network.

I am quite sure that this caused all the other problems, too.


TL;DR: The newer Realtek chips RTL8126 and RTL8127 do not run reliably on older hardware that "supports" PCIe 4.0.

Of course, this is independent of OS. You might also say that is not Realtek's fault, but at least, it does not happen with the Intel X550-T1, which only uses PCIe 3.0. BTW: It only works with either PCIe x4 or x1, so my x2 slot was a waste.

[Basic Requirements]

OPNsense – Accessing ONT / Modem Management Interfaces (Complete Guide)

I know that there are existing guides to do this, like this, yet those threads have become quite cluttered.
This guide does not want to take away from these efforts, rather expand them for a complex edge case that warrants a new guide, so here goes:


Basic Requirements

First off, there are some basic requirements for this to work:

• You must not block RFC1918 IPs on the OPNsense interface connected to the ONT for obvious reasons.
• Your ONT's (or modem's) management IP must be outside of any of your local networks, including remote VPN sites. This is why you should never use some well-known network ranges that ONTs often use (also covered here).
• Some ONTs shut off management access once they establish internet access (i.e., O5 state). Nokia ONTs only react to ARP, but neither ICMP nor IP access works once connected on the fiber side.
• You most probably need outbound NAT to access the ONT because it is statically configured and does not have a default gateway. By using NAT, the translated IP the ONT sees lies inside its own local network, so it can respond correctly. You could theoretically install a route via OPNsense instead, but that does not work for all ONTs. Outbound NAT is the more general solution.
• You need a firewall rule allowing access from your management client(s) to your ONT's management IP.

In what follows, I assume 192.168.100.1 is the ONT management IP and 192.168.100.2/24 is the OPNsense interface IP. Adjust as needed for your setup.

Important warning: Some providers (e.g., Deutsche Glasfaser) block your internet access for a few minutes if they detect traffic to RFC1918 or BOGON IPs outside your networks. Potentially, this may even be true for IPv6 ULAs, if you use those.

• This can happen if devices (Fritzboxes, IoT devices) attempt to reach hardcoded RFC1918 IPs that are not local.
• Such traffic goes out via your ISP router and triggers provider protections.

You cannot simply block RFC1918 on the WAN interface because it violates requirement a).

I like to have an "out" rule on the WAN interface to suppress traffic to RFC1918 destinations, but that is not so easy sometimes...


Two Main Cases

You have two separate cases if you want to enable ONT (or modem) access:

Easy Case: WAN interface ≠ ONT interface

You have the actual WAN interface on a separate interface than your ONT. 
This applies if your WAN uses either a VLAN, PPPoE, or both.

In this case, you can assign the ONT interface to the management subnet, because the interface is not being used otherwise.

So, you simply configure the ONT interface like this:

You cannot view this attachment.

This is the outbound NAT rule (you can use it via manual or hybrid NAT):

You cannot view this attachment.

Now for the question on how to block WAN traffic for RFC1918 destinations - this is easy in this case, because the rule applies to WAN only (and not to ONT):

You cannot view this attachment.

And lastly, the firewall rule to allow management clients to access your ONT (in case your LAN does not have an "allow any" rule):

You cannot view this attachment.

I got a log entry about a failed Github configuration backup, but I first thought this was because of a false positive in one of my blocklists blocking github.com. Now I found that when I started the configuration backup manually, I see an error on the web page reading:

Code Select Expand

Fatal error: Uncaught ValueError: The arguments array must contain 2 items, 1 given in /usr/local/opnsense/mvc/app/library/OPNsense/Core/Shell.php:55 Stack trace: #0 /usr/local/opnsense/mvc/app/library/OPNsense/Core/Shell.php(55): vsprintf('(/usr/local/bin...', Array) #1 /usr/local/opnsense/mvc/app/library/OPNsense/Core/Shell.php(109): OPNsense\Core\Shell::exec_safe(Array, Array) #2 /usr/local/opnsense/mvc/app/library/OPNsense/Backup/Git.php(182): OPNsense\Core\Shell::shell_safe(Array, 'master:main') #3 /usr/local/www/diag_backup.php(395): OPNsense\Backup\Git->backup() #4 {main} thrown in /usr/local/opnsense/mvc/app/library/OPNsense/Core/Shell.php on line 55

The stack trace in the crash reporter was:

Code Select Expand

[28-Jan-2026 01:08:12 Europe/Berlin] PHP Fatal error:  Uncaught ValueError: The arguments array must contain 2 items, 1 given in /usr/local/opnsense/mvc/app/library/OPNsense/Core/Shell.php:55
Stack trace:
#0 /usr/local/opnsense/mvc/app/library/OPNsense/Core/Shell.php(55): vsprintf('(/usr/local/bin...', Array)
#1 /usr/local/opnsense/mvc/app/library/OPNsense/Core/Shell.php(109): OPNsense\Core\Shell::exec_safe(Array, Array)
#2 /usr/local/opnsense/mvc/app/library/OPNsense/Backup/Git.php(182): OPNsense\Core\Shell::shell_safe(Array, 'master:main')
#3 /usr/local/opnsense/scripts/system/remote_backup.php(17): OPNsense\Backup\Git->backup()
#4 {main}
  thrown in /usr/local/opnsense/mvc/app/library/OPNsense/Core/Shell.php on line 55
[28-Jan-2026 13:11:24 Europe/Berlin] PHP Fatal error:  Uncaught ValueError: The arguments array must contain 2 items, 1 given in /usr/local/opnsense/mvc/app/library/OPNsense/Core/Shell.php:55
Stack trace:
#0 /usr/local/opnsense/mvc/app/library/OPNsense/Core/Shell.php(55): vsprintf('(/usr/local/bin...', Array)
#1 /usr/local/opnsense/mvc/app/library/OPNsense/Core/Shell.php(109): OPNsense\Core\Shell::exec_safe(Array, Array)
#2 /usr/local/opnsense/mvc/app/library/OPNsense/Backup/Git.php(182): OPNsense\Core\Shell::shell_safe(Array, 'master:main')
#3 /usr/local/www/diag_backup.php(395): OPNsense\Backup\Git->backup()
#4 {main}
  thrown in /usr/local/opnsense/mvc/app/library/OPNsense/Core/Shell.php on line 55
[28-Jan-2026 13:19:45 Europe/Berlin] PHP Fatal error:  Uncaught ValueError: The arguments array must contain 2 items, 1 given in /usr/local/opnsense/mvc/app/library/OPNsense/Core/Shell.php:55
Stack trace:
#0 /usr/local/opnsense/mvc/app/library/OPNsense/Core/Shell.php(55): vsprintf('(/usr/local/bin...', Array)
#1 /usr/local/opnsense/mvc/app/library/OPNsense/Core/Shell.php(109): OPNsense\Core\Shell::exec_safe(Array, Array)
#2 /usr/local/opnsense/mvc/app/library/OPNsense/Backup/Git.php(182): OPNsense\Core\Shell::shell_safe(Array, 'master:main')
#3 /usr/local/www/diag_backup.php(395): OPNsense\Backup\Git->backup()
#4 {main}
  thrown in /usr/local/opnsense/mvc/app/library/OPNsense/Core/Shell.php on line 55


...went fine, but I got a an error message after the "pkg" update saying "The release type "opnsense" is not available on this repository.".

Asking for an update again does the trick.

This was reproducible on at least three machines.

[you should do the same]

I urge every new OpnSense user to carefully plan their subnets - preferably in advance. While this seems obvious or "easy", the devil lies in the details, some of which you may not know or anticipate.

Of course, if you know anything about networks at all, you will have heard about RFC1918.
This standard defines IPv4 ranges that are to be used on local networks and will not be routed into the open internet, thus needing a NAT translation.

There are three available ranges, namely 192.168.0.0/16, 172.16.0.0/12 and 10.0.0.0/8. You might think, well fine, let's just use one of those, but wait...

By convention, the 192.168.0.0/16 range is almost always divided up into smaller /24 subnetworks - and you should do the same. There are some reasons for that:

• If you want to use larger ranges, because you need more than 254 clients within one subnet, think again:
  - More than about 200 clients in one subnet will cause a significant amount of broadcasts. Not only for ARP, but also for proprietary protocols that many IoT and smart devices use. While modern ethernet with full-duplex is nice, it does not help for broadcasts, let alone what broadcasts do on wireless networks (consider separating them from your (V)LANs).
  - Also, you should at least split up your network into several VLANs based on the trustworthiness of your clients by type: Do they "phone home"? Then put them into a separate VLAN! This alone will probably result in less than 200 devices per VLAN/subnet.
  
• If you want to use smaller ranges, take care. For example, you may think that for accessing your modem, which has an 192.168.100.1 address, you could use a very small network containing only two adresses (thus with a netmask of /31), you would be wrong to choose 192.168.100.2 on your WAN interface. Think about why (use a network calculator if needed).
  This example provides two insights:
  a) A netmask of /31 is always wrong, because usually, you have at least a "network", a "broadcast", a router and one client address, therefore you need a /30 netmask at least and
  b) anything apart from a /24 netmask is hard to visually get correct. /24 netmasks are easy to deal with for humans, because their 3rd octet is always the same, so you can see if two IPs belong to the same subnet (hint: 192.168.100.1 and 192.168.100.2 do not lie together in any /31 subnet).
  
• In fact, anything apart from a /24 subnet in the 192.168.0.0/16 range is considered "unconventional" - and if not explicitely stated, will not be implicitely assumed by anyone trying to help you. This may lead to much wasted time while troubleshooting.

If you really need to configure netmasks different from /24, please do so in the 172.16.0.0/12 or 10.0.0.0/8 range. Because these are more often used by businesses or big enterprises, there will be a lot less assumptions for those.

Here is a subnet calculator. On the right, you can also split and join networks, see this example.

Another hint:

Do not use 192.168.0.0/24, 192.168.1.0/24, 192.168.2.0/24, 192.168.88.0/24, 192.168.100.0/24, 192.168.123.0/24 or 192.168.178.0/24 for your (V)LAN subnets. This is why:

Those ranges are often the default for many routers, including OpnSense itself. Most people, who often know jack about networking, leave their routers at their default settings. At some point in the future, when you try to build a Wireguard site-2-site VPN to a friend, you will find that one of you has to restructure their whole network to be able to route anything at all. Depending on the complexity of the network, this may be a difficult task. Also, many ONTs, cable and DSL modems use IPs from these ranges. If you want them to be made accessible on the WAN interface, you must not have the same range in one of your own (V)LANs. And finally, those ranges are often used by devices in their default state, which may conflict with your network when you connect those.

Before filing a bug report, I rather ask here: Under System: Firmware, you can start a connectivity audit. For me this gives:

Code Select Expand

***GOT REQUEST TO AUDIT CONNECTIVITY***
Currently running OPNsense 25.1.3 (amd64) at Wed Mar 12 13:13:59 CET 2025
Checking connectivity for host: pkg.opnsense.org -> 89.149.222.99
PING 89.149.222.99 (89.149.222.99): 1500 data bytes

--- 89.149.222.99 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
Checking connectivity for repository (IPv4): https://pkg.opnsense.org/FreeBSD:14:amd64/25.1
Updating OPNsense repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: .......... done
Processing entries: .......... done
OPNsense repository update completed. 862 packages processed.
Updating mimugmail repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: ........ done
Processing entries: .......... done
mimugmail repository update completed. 193 packages processed.
All repositories are up to date.
Checking connectivity for host: pkg.opnsense.org -> 2001:1af8:5300:a010:1::1
PING(1548=40+8+1500 bytes) 2001:a61:47f:b942::42 --> 2001:1af8:5300:a010:1::1

--- 2001:1af8:5300:a010:1::1 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
Checking connectivity for repository (IPv6): https://pkg.opnsense.org/FreeBSD:14:amd64/25.1
Updating OPNsense repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: .......... done
Processing entries: .......... done
OPNsense repository update completed. 862 packages processed.
Updating mimugmail repository catalogue...
pkg: https://opn-repo.routerperformance.net/repo/FreeBSD:14:amd64/meta.txz: Unknown resolver error
repository mimugmail has no meta file, using default settings
pkg: https://opn-repo.routerperformance.net/repo/FreeBSD:14:amd64/packagesite.pkg: Unknown resolver error
pkg: https://opn-repo.routerperformance.net/repo/FreeBSD:14:amd64/packagesite.txz: Unknown resolver error
Unable to update repository mimugmail
Error updating repositories!
Checking server certificate for host: opn-repo.routerperformance.net
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = E5
verify return:1
depth=0 CN = opn-repo.routerperformance.net
verify return:1
DONE
Checking server certificate for host: pkg.opnsense.org
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root G2
verify return:1
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL TLS RSA CA G1
verify return:1
depth=0 CN = pkg.opnsense.org
verify return:1
DONE
***DONE***

I see a problem with this:

The ping being called for both IPv4 and IPv6 targets is almost impossible to succeed, because it is being called via "ping -c4 -s1500". Even with a normal ethernet connection without VPN, VLAN or PPPoE overhead, the maximum payload size that is feasible is 1472 for IPv4 and 1452 bytes, respectively, so these pings will almost never work. So, shouldn't the "-s" option be removed altogether?

The "unknown resolver" problem is just because the mimugmail repository does not have an IPv6 address, so we can disregard that.

There have been multiple threads about problems with IPv6 configuration lately.

While there are instructions in the documentation, they leave some vital parts as an exercise to the reader.

Preface:

Despite there being enough IPv6 address prefixes with the abundance of 2^128 possible IPs, many ISPs do not provide you with a static IPv6 prefix like the RFCs propose. Some of them will give you just one /64 range, which is awkward, because you usually need one /64 for each of your subnets/VLANs. Even if they provide a /56 or /48 prefix, which you can then subdivide, some ISPs change the ranges regularly or at least when the connection drops. I think this is to make it harder to host services like a business customer would and thus to differentiate business and consumer uplinks. You can try "prevent release" and setting the DUID under "Interfaces : Settings", but usually, it does not help.

With OpnSense, it is possible to create firewall aliases of type "Dynamic IPv6 Host", that specify only the EUI-64 part of the address and are provided with the 64 bits long prefix of the (V)LAN interface. However, anything else than static DHCP prefixes neccessitate dynamic DNS for IPv6 if you want to make services available from outside. This is sometimes difficult when there are different IP prefixes for your WAN (IA_NA) and (V)LAN (IA_PD) ranges.

So, mostly, you want to have inside-out IPv6 access first, potentially using IPv6 privacy extensions in order to hide your identity.

If you also want to expose services, I recommend to use reverse proxies like HAproxy, Caddy or Nginx - there are tutorials on how to use them here on the forum. The reverse proxies can also convert from incoming IPv6 connections to internal IPv4 services, which further reduces complexity for these scenarios. Also, you can (and should) use wildcard certificates if ever possible, because named certificates will be published and can be used to scan for IPv6-based services where a normal port scan is infeasible.

When you are behind CG-NAT, your only chance to expose services is via a reverse tunnel like Cloudflare or to use IPv6. In the latter case, a reverse proxy will work as well.

You can also expose individual (V)LAN clients via their EUI-64, but you will need both firewall aliases and a decent dynamic DNS to do it.


The way I do this is as follows:

1. I configure the WAN interface to request an IPv6 prefix via DHCPv6.

You cannot view this attachment.

Usually, this will be a /56 or a /48 prefix, depending on your ISP. It must be < /64 for the rest to work.
I also use "Request prefix only", because some ISPs do not hand out IA_NA (i.e. a /128 WAN IPv6 GUA) anyway and some even refuse to respond to DHVPv6 requests for it. That way, I will only get a IA_PD IPv6 GUA prefix and no WAN IPv6, but that can actually be turned into an advantage:

If I got an IA_NA, it would be from another range than the IA_PD. Thus, when I update a dynamic DNS entry from OpnSense, the IA_NA prefix will get registered and I cannot make LAN clients available, because they use the other (IA_PD) prefix. So, it is better to use the IA_PD prefix for the WAN interface, too. Of course, you have to be able to set or fix the lower 64(+) bits of the DNS entry to the EUI-64 of the clients if you want to expose them, because the used IPv6 will be the one from OpnSense's WAN interface, that differs in the lower 64 plus interface bits. With current OpnSense versions, you can also use the "Check ip method" "Interface [IPv6]" in the Dynamic DNS service to actually use an IP from one of your local (V)LAN interfaces independent from your actual WAN IPv6 - but in order to use it, your DynDNS provider needs to be capable of accepting an IPv6 from the supplied parameters rather than the source address (which will always be the WAN IPv6).

If you want to have an IPv6 assigned to your WAN despite using IA_PD only, it is essential to specify an "Optional prefix ID" that is different from all (V)LAN prefixes, because each interface has to have a different prefix ID. Consider a /56 prefix from the provider that is: 1111:2222:3333:44XX:YYYY:YYYY:YYYY:YYYY/56. Then, XX denotes the prefix ID, leaving the 64 lower bits (YYYY) for SLAAC assignments on the repective interface.

You can leave the "Optional interface ID" empty, in which case the EUI-64 that is derived from the MAC of the WAN interface will be taken as a default. This in turn leaks information about the manufacturer of your WAN NIC via its OUI, so you may want to specify something arbitrary (or memorable) here.

2. I use SLAAC only for my (V)LAN interfaces, not DHCPv6.

You cannot view this attachment.

That is for several reasons:

• DHCPv6 is not understood by many devices (like Android).
• DHCPv6 is unable to specify more than one prefix (it cannot assign ULA IPv6 on top of GUA IPv6, e.g.).
• DHCPv6 will keep your clients to use an outdated, non-working prefix in case your connection drops or your ISP changes GUA prefixes regularly.
• By using SLAAC, you also can use IPv6 Privacy Extensions (RFC 8981) on the clients, still leaving an EUI-64 based management IPv6 for predictable inbound use.

SLAAC is done via router advertisements (RA) and can also handle DNS server and gateway announcements (but see note 6 below).

Therefore, I use these settings:

• IPv6 Configuration Type = Identity Association
• Parent interface = WAN
• Assign prefix ID = <different for each interface>
• Optional Interface ID = empty

The prefix ID can be only N bits long, where N = 64 minus the ISP-provided prefix size, e.g. 8 bits when /56 is used or 16 bits with /48.
On (V)LAN interfaces, I usually do not specify an interface ID.

Under Services: Router Advertisements, I use these settings for any (V)LAN:

You cannot view this attachment.

• Router Advertisements = Unmanaged (meaning SLAAC only)
• Source Address = automatic
• Enable DNS = unchecked (meaning do not send DNS configuration at all - see note 6 below)
• DNS servers = empty
• Domain search list = whatever you need (or empty)
• Minimum Interval = 30
• Maximum Interval = 60

[li}Shutdown Advertisement = Automatic (or probably Off, if you experience problems when your ISP re-uses the same prefix again)[/li]
[/list]

I use rather short intervals to distribute RAs quite often and also, to have a short lifetime in case the WAN interface changes its GUA. I want to keep a potential use of deprecated prefixes as short as possible. This can have an averse effect on battery life for WLAN clients.

Mind you, there may be reasons why RAs do not work for your clients. You can always look at the resulting RA settings by inspecting /var/etc/radvd.conf and see if the GUA prefix was picked up and if your (V)LAN prefixes and other settings seem right. Then, you can look at your client if it really receives the RAs.

3. A neat trick: If you want (or need) to configure your OpnSense to announce itself for any service, like, as a gateway, NTP or DNS server, you can always use its link-local address - because you do not know the GUA beforehand with changing prefixes. You can even create a virtual IP like fe80::1/64 on each local interface and use that instead. This is also easier to memorize for static configurations.

4. If you use VLANs and want to block inter-VLAN traffic like you probably do for IPv4: Instead of the often suggested approach to first block destination "RFC1918" and after that, allow "any" (or just allow "!RFC1918"), I define an interface group for all local protected VLANs and then use one IPv4+IPv6 blocking rule with a target of "LOCAL_VLANS net". This would block local traffic destined for the firewall (as does using "RFC1918" there, but is restricted to IPv4), so any more specific allow rules should be placed into the floating rules or put before these blocks.

5. If you want to make a local client accessible from the outside or if you need specific firewall rules, you may face the problem of changing IPv6 prefixes, like many ISPs offer. You can use a "Dynamic IPv6 host" type alias, which only specifies the EUI-64, i.e. the lower 64 bits of the IP. If you go that route instead of offering access via a reverse proxy, keep in mind that such rules would best be place in the floating rules, because, if you allow outside access, then why should you block inter-VLAN traffic?

Also, such a "Dynamic IPv6 host" alias can also be used as the target of a NAT rule, thus, you can make OpnSense forward ports to your internal host, while still being accessible by OpnSense's WAN IPv6, thus making dynamic DNS maintainable by OpnSense.

However, because you will find that handling dynamic DNS for your internal services is sometimes complicated, I recommend using a reverse proxy, like Caddy, Nginx or (preferably) HAproxy. That way:

a. OpnSense can handle DynDNS for the services - it always fits because the IPv6 is then always the WAN IP of OpnSense and the internal servers can be addressed via IPv4.

b. the services can be addressed by DNS name, whereby you can also work with wildcards (e.g. *.domain.de) so that new services/hosts work without further DynDNS adjustments. The wildcard address is directed to the DynDNS name.

c. TLS termination and obtaining certificates are carried out by OpnSense, too, which can also be done via wildcard, so you only need one certificate for everything. This makes distributing names to internal servers super easy. With HAproxy, for example, the new service/name is then just a "real server", a "backend" and a line in the map file. Wildcard certificates are also more secure because you cannot easily see which names really exist (see https://crt.sh).

There is a guide on how to setup reverse proxies for HTTP(S), and here you can find how to do it for SMTP and IMAP.

6. A note about DNS: While you can distribute DNS servers via IPv6, the priority between IPv6 and IPv4 is undetermined. Also, supplying that via SLAAC is a late addition that not all clients understand (matter-of-fact, if you use DNSmasq for RA, it cannot even be supplied). Sometimes, DHCPv6 is only used to supply the DNS server address, while SLAAC provides IPs and gateways. Because I have dual-stack and provide an IPv4 DNS server (which can also reolve to IPv6 addresses), I do not provide any IPv6 DNS server at all.

[MTU - how do I set it?]

So, because still some people might have problems understanding what this is all about, despite there is already another good guide about this:

https://forum.opnsense.org/index.php?topic=21207.0


MTU - how do I set it?

First off, you must know that on you LAN, the default MTU size will most likely be 1500 Bytes.
This is not the same as the ethernet frame size of 1518 Bytes, which includes the MAC and frame check sequence.
Also, this is not the maximum data payload size, called MSS (maximum segment size), which is then 1460 Bytes.

The physical reality is shown here: https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/appneta/GA/analyze-results/network-performance-monitoring-delivery/delivery-results/delivery-diagnostics-messages/maximum-transmission-unit.html

Ignore VLANs on your own network for now, I will explain that later on.


Let's focus on MTU only. When a network packet is transmitted over your WAN interface, you can get lucky and it only gets converted to another medium by an NT (optical or DSL). This is often the case when your WAN configuration only has DHCP and nothing more.

However, there can be setups where your WAN interface is over a VLAN. The VLAN tag is 4 bytes in length, and because the interfaces are stacked on one another, you will see that under OpnSense, your WAN interface may have a VLAN name like "igc0_vlan40". Thus, the lower layer interface has to accommodate both the packet payload and the VLAN header. Since the default MTU of your igc0 ethernet interface is 1500 bytes, the VLAN interface's MTU can only accomodate 1496 bytes, and accordingly, the MTU of igc0_vlan40 will be 1496 bytes.

What happens now is that packets destined to a target on the internet may have to be split up, e.g. when they carry a payload of 1500 bytes, into one packet with 1496 bytes and one with 4 bytes.
Because of the inter-frame gaps and roundtrip times, the 4 byte packet is suboptimal and will cause a slowdown. It would be beneficial if your WAN MTU matches your LAN MTU.

There can also be the case that your internet connection uses PPPoE, which incurs an overhead of 8 bytes, giving you only a net payload MTU of 1492 Bytes. The worst case would be that the PPPoE connection goes through a VLAN, so have have a stack of WAN (PPPoE, 8 bytes overhead) over a VLAN (4 bytes overhead) on an ethernet port. In such cases, you only have 1500 -8 - 4 bytes MTU, leaving you at 1488 bytes.

Both the "normal" ethernet setup and the "worst case" setup are depicted in the image I appended.

You cannot view this attachment.

OpnSense tries to guess the resulting MTUs, but it sometimes fails.

It is essential that your MTU size is not set too high, because some internet sites cannot correctly train their TCP packets to match the maximum available MTU (this mechanism is called path MTU discovery).
When you try to contact such a site, packets may get lost on the way, such that you effectively cannot reliably work with them.

There is a Linux tool to find out what maximum MTU your WAN will accomplish here. I added a FreeBSD version of the script at the end, it can be run under OpnSense CLI.

You should set your WAN MTU to the value that the tool will tell you can achieve.

However, as I said, ideally, you will want to match your WAN MTU to your LAN MTU and sometimes, this is feasible.

Disclaimer: "Mini" jumbo frames do not work with german Telekom and their reseller ISPs, so the best you can get is a 1492 byte MTU. This is because in reality, the 4 bytes VLAN tags can be accomodated by modern network hardware, while the 8 Bytes PPPoE overhead cannot. Therefore, all you can use is 1500-8 = 1492 bytes.

It clearly depends on your hardware and that of your ISP if that works, because you will have to use a non-standard ethernet packet MTU, a kind of "jumbo" frame. BTW: A proxmox VirtIO interface (vtnetX) will need configuration for that under Proxmox itself.
For the worst-case scenario with both PPPoE and VLAN involved, you would theoretically need an MTU of 1512 on the ethernet port, 1508 on the VLAN created on it and then 1500 for PPPoE. This is the third case in the appended image.

As an example, if you have a WAN over PPPoE over VLAN, you should have to set WAN MTU = 1500, PPPOE VLAN = 1508, ethernet port = 1512, and it really only works for me with these tightly controlled MTUs:

pppoe0 MTU: 1500 (this must only be set in the advanced settings of pppoe0, not on the WAN interface itself, see screendumps). This will set the WAN MTU.
ONT MTU: (this means the physical ethernet port): 1512 if you have a VLAN for PPPoE, 1508 if not.
PPPOE VLAN MTU: 1508 (if needed in your setup).

You cannot view this attachment.You cannot view this attachment.


To be able to set these values, you will have to assign each of the underlying interfaces a name, which you would normally not need to.

Also, set the above values in the web UI and then reboot, if in doubt - the values sometimes cannot be successfully changed via UI manipulations, because the order of application seems to be wrong that way.

Afterwards, verify that the shown MTUs are as expected via "ifconfig". Also verify that the achievable MTU over your WAN connection works with the tool I linked above. Ideally, you should be able to do get ping replies this command:

"ping -4 -c4 -D -s 1472 1.1.1.1", resulting in an unfragmented 1500 byte packet.

But not to this:

"ping -4 -c4 -D -s 1474 1.1.1.1", resulting in one oversized packet that cannot be transmitted because of the DF bit set.

Usually, you will also get an answer, when you use larger packets, but only when you allow for fragmentation like this:

"ping -4 -c4 -s 1480 1.1.1.1", resulting in two fragmented packets (note the missing "-D" parameter!).


P.S.: As promised, about your local VLANs: Theoretically, the calculations above would also apply to local ethernet ports, but most switches can do jumbo frames on their switching fabric. Also, usually, you will fan out those VLANs on ports as untagged anyway. Therefore, OpnSense thinks it is best to set a VLAN's MTU also at 1500 bytes. That usually works, even when you use a trunk port.
Most network hardware is capable of doing jumbo frames (up to 9014 bytes) anyway. So, normally, you do not have to think about the theoretical difference for an untagged or a tagged VLAN and can use 1500 bytes MTU for either of them.

P.P.S.: If you use Proxmox or other types of virtualisation, you must set the larger MTU of 1512 or 1508 on the underlying host interface(s) as well. For Proxmox, this would be the bridge and ethernet interface for the WAN port for VTNET interfaces or only the ethernet interface with pass-through.

P.P.P.S.: If you cannot set your WAN MTU to 1500 and have problems with IPv6 router advertisements, see this: https://forum.opnsense.org/index.php?msg=261211

I saw this error in my wireguard log after upgrading to 25.1:

Code Select Expand

2024-11-02T12:41:43 Error wireguard /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: The command '/usr/bin/wg syncconf 'wg0' '/usr/local/etc/wireguard/wg0.conf'' returned exit code '1', the output was 'Name does not resolve: `xxx.yyy.de:6010' Configuration parsing error'

The corresponding line in the "Peer" section of /usr/local/etc/wireguard/wg0.conf is:

Code Select Expand

Endpoint = xxx.yyy.de:6010

This configuration has not been hand-edited, it was created via the web UI.

This is about a brand-new Unifi USW-Pro-HD-24-POE switch. I though I write this here to save others the time to find out themselves:

I recently bought one of those switches and wanted to use the LAGG feature with my OpnSense box with 4 I226V NICs.

My previous setup was to use two of those interfaces, one carrying my main (V)LAN and the other to carry all other VLANs, e.g. IoT. With that, I could have cross-VLAN traffic at 2.5 Gbps in both directions.

So I tried setting up a LAGG with LACP layer 4, in order to have TCP ports come into the mix for hashing and distributing different TCP streams over two physical links. I tested with "iperf3 -c x.x.x.x -P4" between two Linux VM hosts on different VLANs to have multiple source ports and to my great disbelief, got only 2.5 Gbps. The same thing happened when I tried against OpnSense itself.
I also tried two different iperf3 instances from the same machine to no avail.

After some more trial and error, I found that the Unifi switch can only handle layer 2 & 3 hashing, which is especially disappointing when you know that they did better years ago. Only when I had two different clients do independent runs, could I have > 4 Gbps in sum - and even this took a few tries before I found a counterpart machine with a matching MAC.

I then found this tidbit in the CLI of the switch:

Code Select Expand

BusyBox v1.25.1 () built-in shell (ash)


  ___ ___      .__________.__
 |   |   |____ |__\_  ____/__|
 |   |   /    \|  ||  __) |  |   (c) 2010-2024
 |   |  |   |  \  ||  \   |  |   Ubiquiti Inc.
 |______|___|  /__||__/   |__|
            |_/                  https://www.ui.com

      Welcome to UniFi USW-Pro-HD-24-PoE!

********************************* NOTICE **********************************
* By logging in to, accessing, or using any Ubiquiti product, you are     *
* signifying that you have read our Terms of Service (ToS) and End User   *
* License Agreement (EULA), understand their terms, and agree to be       *
* fully bound to them. The use of SSH (Secure Shell) can potentially      *
* harm Ubiquiti devices and result in lost access to them and their data. *
* By proceeding, you acknowledge that the use of SSH to modify device(s)  *
* outside of their normal operational scope, or in any manner             *
* inconsistent with the ToS or EULA, will permanently and irrevocably     *
* void any applicable warranty.                                           *
***************************************************************************

hagen-US3.7.1.33# cli
hagen# configure
hagen(config)# lag load-balance
Incomplete command
hagen(config)# lag load-balance ?
  src-dst-mac     LAG load balancing is based on source and destination MAC addr
                  ess.
  src-dst-mac-ip  LAG load balancing is based on source and destination of MAC a
                  nd IP addresses.
hagen(config)#

This shows that only layer 2 & 3 are being supported, as my tests confirmed.

Actually, for my scenario, LAGGs are worse without layer 4 hashing than splitting the VLANs across the interfaces, as I would have to be lucky (chances are 50:50) to have two arbitrary devices be put on different links via layer 2 or 3 hashing, even when I use multiple TCP streams with different ports.

I also posted this to the Unifi forum.

Oh, BTW: Today, I looked at my USW-Enterprise-24-PoE. It has a completely different CLI structure and obviously, this one does in fact support LACP layer 4:

Code Select Expand

BusyBox v1.25.1 () built-in shell (ash)


  ___ ___      .__________.__
 |   |   |____ |__\_  ____/__|
 |   |   /    \|  ||  __) |  |   (c) 2010-2024
 |   |  |   |  \  ||  \   |  |   Ubiquiti Inc.
 |______|___|  /__||__/   |__|
            |_/                  https://www.ui.com

      Welcome to UniFi USW-Enterprise-24-PoE!

********************************* NOTICE **********************************
* By logging in to, accessing, or using any Ubiquiti product, you are     *
* signifying that you have read our Terms of Service (ToS) and End User   *
* License Agreement (EULA), understand their terms, and agree to be       *
* fully bound to them. The use of SSH (Secure Shell) can potentially      *
* harm Ubiquiti devices and result in lost access to them and their data. *
* By proceeding, you acknowledge that the use of SSH to modify device(s)  *
* outside of their normal operational scope, or in any manner             *
* inconsistent with the ToS or EULA, will permanently and irrevocably     *
* void any applicable warranty.                                           *
***************************************************************************

edgar-US.7.1.26# cli

Entering character mode
Escape character is '^]'.

Warning!
The changes may break controller settings and only be effective until reboot.

(UBNT) >enable

(UBNT) #configure

(UBNT) (Config)#port-channel load-balance ?

1                        Src MAC, VLAN, EType, incoming port
2                        Dest MAC, VLAN, EType, incoming port
3                        Src/Dest MAC, VLAN, EType, incoming port
4                        Src IP and Src TCP/UDP Port fields
5                        Dest IP and Dest TCP/UDP Port fields
6                        Src/Dest IP and TCP/UDP Port fields

(UBNT) (Config)#port-channel load-balance

So, it seems model-dependend, probably based on product lines (i.e. Pro vs. Enterprise).

I just got the dreaded MOZILLA_PKIX_ERROR_REQUIRED_TLS_FEATURE_MISSING error message, because my preferred certificate has OCSP stapling enabled. I want to use that feature, because my domain is used for other purposes as well with a wildcard certificate.

I found https://forum.opnsense.org/index.php?topic=26812, so is OCSP stapling still infeasible with lighthttpd?

Ah, apparently, the feature is available now: https://redmine.lighttpd.net/projects/lighttpd/wiki/Docs_SSL#OCSP-Stapling, I'll create a ticket.

[RAM, CPU and system]

These days, there are many folks who use OpnSense under a virtualisation host, like Proxmox, for example.

This configuration has its own pitfalls, therefore I wanted to have this guide. The first part starts with common settings needed, the second part will deal with a setup where the virtualisation host is to be deployed remotely (e.g. in a datacenter) and holds other VMs besides OpnSense.

RAM, CPU and system

Use at least 8 GByte, better 16 GBytes of RAM and do not enable ballooning. Although OpnSense does not need that much RAM, it can be beneficial in case you put /var/log in RAM (see below). Only if you are really short on RAM should you use 4 GByte.
And please, do not complain if your Proxmox memory statistics show ~100% (or more) RAM being in use - it is expected. Also note, that ballooning does not work with the os-qemu-guest-agent (which you should install, anyway).

Obviously, you should use "host" CPU type in order not to sacrifice performance by emulation. However, you should not install the microcode update packages in OpnSense - they would be useless or even do harm anyway. Instead, install the appropriate microcode packages on the virtualisation host.

That being said, just for good measure, set tuneables "hw.ibrs_disable=1" and "vm.pmap.pti=0". This wil avoid performance bottlenecks because of Spectre and Meltdown mitigations. I trust the other VMs in my setup, but YMMV...

The system architecture is arbitrary, as OpnSense can boot both in legacy (BIOS) or UEFI mode.

Filesystem peculiarities

First off, when you create an OpnSense VM, what should you choose as file system? If you have Proxmox, it will likely use ZFS, so you need to choose between UFS and ZFS for OpnSense itself. Although it is often said that ZFS underneath ZFS is a little more overhead, I would use it regardless, just because UFS fails more often. Also, OpnSense does not stress the filesystem much, anyway (unless you use excessive logging, RRD or Netflow).

32 GBytes is a minimum I would recommend for disk size. You can increase the disk size later on when you use "touch /.probe.for.growfs" from the CLI, - see this post for an explanation.

After a while, you will notice, that the space you have allocated for the OpnSense disk will grow to use 100%, despite that within OpnSense, the disk may be mostly unused. That is a side-effect of the copy-on-write feature of ZFS: writing logs and RRD data and other statistics always writes new data and the old data does not get dismissed against the underlying (virtual) block device.

That is, if the ZFS "autotrim" feature is not set manually. You can either set this via the OpnSense CLI with "zpool set autotrim=on zroot" or, better, add a daily cron job to to this (System: Settings: Cron) with "zroot" as parameter.
You can trim your zpool once via CLI with "zpool trim zroot".

That being said, you should always avoid to fill up the space for the disk by having verbose logging. If you do not need to keep your logs, you can also put them on a RAM disk (System: Settings: Miscellaneous).

Network "hardware"

With modern FreeBSD, there should not be any more discussion about pass-through vs. emulated VTNET adapters: the latter are often faster. This is because Linux drivers are often more optimized than the FreeBSD ones. There are exceptions to the rule, but not many.

In some situations, you basically have no choice than to use vtnet anyway, e.g.:

• If FreeBSD has no driver for your NIC hardware
• If the adapter must be bridged, e.g. in a datacenter with a single NIC machine

Also, some FreeBSD drivers are known to have caused problems in the past, e.g. for RealTek NICs. By using vtnet, you rely on the often better Linux drivers for such chips.

With vtnet, you should make sure that hardware checksumming is off ("hw.vtnet.csum_disable=1", which is the default on new OpnSense installations anyway because of a FreeBSD interoperability bug with KVM). Note, however, that this setting will be slower than using hardware offloading, which you will notice at very high speeds, especially on weak hardware.

You can also enable multiqueue on the VM NIC interfaces, especially, if you have multiple threads active. There is no need for enabling this in OpnSense.

For some Broadcom adapters (and possibly others, as well), it is neccessary to disable GRO by using:

Code Select Expand

iface enp2s0f0np0 inet manual
    up ethtool --offload $IFACE generic-receive-offload off

See: https://forum.opnsense.org/index.php?msg=233131, https://help.ovhcloud.com/csm/en-dedicated-servers-proxmox-network-troubleshoot?id=kb_article_view&sysparm_article=KB0066095 and https://www.thomas-krenn.com/de/wiki/Broadcom_P2100G_schlechte_Netzwerk_Performance_innerhalb_Docker.

When you use bridging with vtnet, there is a known Linux bug with IPv6 multicasting, that breaks IPv6 after a few minutes. It can be avoided by disabling multicast snooping in /etc/network/interfaces of the Proxmox host like:

auto vmbr0
iface vmbr0 inet manual
    bridge-ports eth0
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094
    bridge-mcsnoop 0


If you plan to enlarge your MTU size on VirtIO network interfaces, note that you must do so on the Proxmox bridge device first.

Also, you probably should disable the firewall checkbox for the network interfaces in the OpnSense VM.

Guest utilities

In order to be able to control and monitor OpnSense from the VM host, you can install the os-qemu-guest-agent plugin.

Problems with rolling back

One of the main advantages of using a virtualisation platform is that you can roll back your installation.

There are two problems with this:

1. DHCP leases that have been handed since the time of last roll back are still known to the client devices, but not to the OpnSense VM. Usually, this will not cause IP conflicts, but DNS for affected devices may be off intermediately.

2. If you switch back and forth, you can cause problems with backups done via os-backup-git. This plugin keeps track on both the OpnSense VM and the backup repository. If both are of a different opinion about the correct revision of the backup, subsequent backups will fail. Basically, you will ned to setup the backup again with a new, empty repository.

If you want to avoid such problems, you can roll back single packages with opnsense-revert.


TL;DR

• Have at least 8 GByte of RAM, non-balooning
• Have a disk size of 32 GByte or more and "touch /.probe.for.growfs" via CLI
• Use "host" type CPU and disable Spectre and Meltdown mitigations
• Use ZFS, dummy
• Keep 20% free space
• Add a trim job to your zpool
• Use vtnet, unless you have a good reason not to
• Check if hardware checksumming is off on OpnSense
• Disable multicast snooping and Proxmox firewall
• Install os-qemu-guest-agent plugin

That is all for now, recommendations welcome!

I just upgraded 3 machines from 24.7.4_1 to 24.7.5. All of them completed the upgrade, showed a popup that they now reboot, but did not. After 2 minutes, the popup clears and the dashboard shows with 24.7.5, but a reboot has not occurred as the uptime shown by "w" still was a few days.

[READ THIS FIRST]

I always threatened that at some point I might be inclined to write something like this, because I find myself answering the same questions time and time again.

So, reflecting an old article of the great Dave Barry which starts:

READ THIS FIRST

Congratulations! You have purchased an extremely fine device that would give you thousands of years of trouble-free service, except that you will undoubtly will destroy it via some typical bonehead consumer maneuver.

Which is why we ask you to PLEASE FOR GOD'S SAKE READ THIS OWNER'S MANUAL CAREFULLY BEFORE YOU UNPACK THE DEVICE.

YOU ALREADY UNPACKED IT, DIDN'T YOU? YOU UNPACKED IT AND PLUGGED IT IN AND TURNED IT ON AND FIDDLED WITH THE KNOBS, AND NOW YOUR CHILD, THE SAME CHILD WHO ONCE SHOVED A POLISH SAUSAGE INTO YOUR VIDEOCASSETTE RECORDER AND SET IT ON "FAST FORWARD", THIS CHILD ALSO IS FIDDLING WITH THE KNOBS, RIGHT?

WE MIGHT AS WELL JUST BREAK THESE DEVICES RIGHT AT THE FACTORY BEFORE WE SHIP THEM OUT, YOU KNOW THAT?

Please, read this before you ask questions here:

1. Learn basic networking skills before trying to use OpnSense. OpnSense is a firewall, yes, but in the first place it is a  router (unless you use it as a transparent bridge). Know that you must have disjoint subnets on different interfaces in order to be able to control traffic at all. Many questions hover around topics like: "Why can't I filter traffic between 192.168.23.10 and 192.168.23.17?" (It will not work, that is basic networking 101). Also: get your subnets straight and non-overlapping, as this may have all kinds of strange effects.

2. This includes scenarios where you have multiple clients attached to different ports of your OpnSense. That would  be called a bridge, which is usually discouraged ("use a real switch"), but can be done, if done correctly ("read the docs" and follow them, stupid). In order to not get confused, do not even assign names to bridge member interfaces, because you will not use them in rules or anything but the bridge definition. Do not forget the tuneables ("Which tunables?" - read the docs again!).
 
3. You have created a VLAN but cannot get internet access? You have to create an outbound "allow all" firewall rule like the one that is created per default on your LAN. Remember? "Anything, that is not explicitely allowed, is forbidden." - from the great book: Purpose of a firewall, chapter 1. BTW: Do not mix tagged and untagged traffic on the same interface - it does not work in some cases, e.g. VLAN traffic is counted on both the VLAN and the (parent) untagged interface.
On a side note: Never, ever use VLAN 1 - many network vendors handle that internally as the equivalent of the untagged VLAN, which is technically not the same. In Proxmox, I use a restriction to only use VLANs 2-4095 for that very reason.
 
4. Try to avoid router-behind-router scenarios. You either would need double NAT, which is awkward, or be able to control local routing in your ISP router, which you probably are not.
That in turn may be more costly than you think, which I explained here (but in german). For german Fritzbox converts, there is a special article here. If you (like most people) have dynamic IPv6 prefixes, this will pose big problems, because prefix sub-delegation cannot be provided via SLAAC and DHCPv6 can only be pulled for new prefixes, such that OpnSense will be delayed in getting the new prefix after a change has occurred. This is different if OpnSense itself is the primary router and knows when the prefix changes.
It is similar with the WAN address: OpnSense does only know its local WAN address, not the real, internet-facing one. This makes it harder to do dynamic DNS.
Also, you might get problems with UDP-based protocols, like NTP, depending on the outer router's NAT handling.
 
5. Get to know your hardware. One of the more often asked questions is: "I cannot access my web UI after installation." The problem is probably either that you have no useable NICs (see next point), you plugged your LAN connection to the wrong spot or you left the USB stick in and boot the installation image instead of the configured system. If you absolutely want / must use OpnSense virtualized, read this.
 
6. Realtek NICs are badly supported under FreeBSD. That concerns performance, VLANs and more generally, basic operation.  There are OEM drivers which can be installed (System:Firmware:Plugins, os-realtek-re), but in some situations, this is difficult, because you cannot even load them from the internet ("use a USB stick instead"). Intel chips are supported much better, usually. If you have an I226-V, have problems and feel adventurous, take a look at this thread. Also, see point #23 below.

7. Do not try to use the WLAN adapter on your OpnSense box as a WiFi AP. FreeBSD / OpnSense is just not good (tm) at it - neither is it good at using USB adapters of any kind. Use dedicated AP(s).
 
8. If something does not work, do not jump to conclusions. Try to pinpoint your problem, e.g. do not say: "I cannot ping www.google.com". There are so many problems there:

- ping from where? Client or OpnSense itself?
- what client OS? (ping works differently on some)
- What exactly did not work? DNS, IPv4 or IPv6?

Possibly, you know - we don't. So there may be at least 7 separate potential problems that were subsumed in one statement. Use "nslookup www.google.com", "ping 8.8.8.8" and "ping 2600::" instead, from both clients and OpnSense itself. BTW: As far as ping options go: FreeBSD ≠ Linux ≠ Windows.

9. Only some sites are unreachable, like OpnSense.org? Even if basic DNS, routing and firewalling works for both IPv4 and IPv6, there may still be other problems lurking, like wrong MTU settings on WAN because of encapsulation via PPPoE, VLANs or IPoE, which you did not account for. In that case, some sites with defective PMTU discovery may fail.
There is a tool to check how big your WAN MTU may safely be set for a specific site.

10. You do not get close to your ISP's advertised speed? Some CPUs are not sufficient for fast internet connections. Look for ones with high single-thread performance - especially, if you need PPPoE, VPN and/or Zenarmor or other IDS (disable them for tests). For more than 1 GBps, you may need something along the range of an N100, for more than 2.5 Gbps you will look at CPUs with even more punch. There are many websites to compare CPU speeds or look in the hardware and performance section.
Also note this about RSS. That being said, hand-me-down systems often are sub-par w/r to being used for OpnSense, because desktops with older CPUs tend to use way more power such that the higher energy cost may soon outweigh the investment on a modern appliance in a 24/7 scenario.

Also: Disable any traffic shaper, especially when you just upgraded your plan to a higher speed. For > 1 Gbps speeds, check if the ethernet connection between modem/ONT and your firewall is actually negotiating at more than 1 Gbps.

That being said: How are you measuring? Single stream performance may not utilize your connection fully, so if you use iperf, try "-P4". Also, do not measure from OpnSense itself, but route the connection through it, in order not to put more strain by running the test client or server on it. You can use OOKLA speedtest in a browser, this is multithreaded.

11. If you aim to access your ONT or modem on the WAN side, you will need a (virtual) IP within the same subnet plus an outbound NAT rule for that, because the ONT or modem does not have a default gateway and does not know how to access anything beyond its own subnet. This is explained in more detail here.
 
12. If you think you can control on what websites your network clients can access with a firewall, notice there are three basic ways:
  a. you can block certain IPs or
  b. you can control specific domains via pre-manufactured DNS-based black- or whitelisting or
  c. you can inspect traffic by intercepting it with a web proxy (this would also allow a centralised virus scanner)
Now forget a. and c. Why? a. will not work if the afftected websites are hosted on a CDN, because you will block the whole CDN in this case. For c., note that for the most part, web traffic is encrypted these days. In order to inspect it, you will have to de- and then re-encrypt it.  Since the re-encryption is done by your firewall, you will have to create TLS certificates for the destination website on-the-fly with your own CA to present to your browser. These "faked" TLS certificates must then be trusted by the client devices, which is difficult to do on PCs and mostly infeasible on many smartphones and IoT devices. Such "man-in-the-middle-attacks" are deliberately made hard for good reasons. Thus, you would at least have to use the TLS "no bump" feature for banking sites and also, for many more domains that use CAA DNS records and will not work with TLS inspection. The amount of work to maintain that list is huge and only warranted in an enterprise setup.

13. I do not believe in IPSs like Zenarmor, Crowdsec or Suricata, but YMMV. At least do not use Suricata on WAN, unless you are willing to sacrifice IPv6 connectivity - BTW: Choosing the WAN interface is the wrong approach, anyway. This is a fine example for always having a tradeoff between (perceived) security and useability. Also: If you use IPS and experience any problems, please state that in your posting - or better, disable it and test again! The same goes for any kind of blocklists: check if they are the culprit.

14. Install OpnSense with ZFS, not UFS. You will get a "snapshot" feature that enables you to roll back system upgrades iff you use it before you update. Also, it is more robust against hard resets. If your storage space is small, keep in mind that the default swap size is 8 GBytes - you may need to lower that value during installation to have enough logging space. ZFS eats SSDs, though, especially when you enable Netflow and / or RRD, because they write huge amounts of data. Choose a drive with a large TBW rating.

Also, these days, there are more reports of no-name brand NVMEs in china boxes (probably because of higher storage cost) which give strange errors in logs (see this and this). FreeBSD does not care about circumventing NVME protocol problems and those manufacturers probably do not test under FreeBSD, either. More often than not, there is no remedy short of replacing the drive.

15. If something really strange happens, like you cannot log in, try things that should be obvious, but aren't:
  - verify that the system time and date is correct (e.g., certificates will not work if the date is too far off).
  - verify that the file system is not out of disk space (you may have set verbose logs that filled it all up). You can also have logging on a RAM disk if you do not need permanent logs. In that case, a reboot will clear the logs.

16. Whenever you have a question, please, use the forum search. It is highly likely your question has been asked (and answered) many times before. If you still have questions, do not add to or revive another thread if it does not handle your specific problem, but create your own thread with as much information pertaining to the problem, such as:
- OpnSense version
- network topology
- IP ranges
- rules or settings that you use
- hardware (also: if you are using a hypervisor, please tell us!)
If your problem was solved, mark the thead by prepending [SOLVED] in its title.

17. You found that the dashboard temperatures are higher than those reported by "sysctl dev.cpu | fgrep temperature" or you see regular CPU spikes when using the dashboard? Congratulations - but please, do not ask about it or tell us before having consulted one of these threads:

https://forum.opnsense.org/index.php?topic=44373.0
https://forum.opnsense.org/index.php?topic=42575
https://forum.opnsense.org/index.php?topic=36234
https://forum.opnsense.org/index.php?topic=34395
https://forum.opnsense.org/index.php?topic=41759.0
https://forum.opnsense.org/index.php?topic=30293

These turn up in the first 30 search results if you just use "temperature" as keyword. A fine example of what can be achieved by using the search, isn't it?

18. Your IPv6 setup does work work right out of the box? Try this HOWTO.

19. While feature requests and bugs can be discussed here on the forum, thee most likely will never be implemented unless someone (i.e. you!) does a feature request or bug report on Gibthub. If there is a corresponding thread already, please post a link to the feature request or bug report there.

20. Plan your subnets carefully in advance. Too much to discuss here, but I wrote a separate article about this. If you did not and have any problem, explicitely tell us up front!

21. You want to speed up your inter-VLAN traffic? If you do not have an enterprise network, forget link aggregation (LAGG) - use trunked interfaces instead. A LAGG does not work in practive for home setups. That is because any IP stream between two machines can only use one connection at a time. Most switches can only distribute packets by MAC, IP or port number, round robin exists only for Infiniband.
The best thing you could have is to have multiple streams between two machines, but since most switches cannot even do distribution based on port numbers, this will not help, either. And since most switches just distribute based on MAC (i.e. you are limited to a layer 2 hash), you can end up in situations where most or all of your machines communicate over the same link, while the other is scarcely used. LAGG works only for large installations, where statistics play in your favor. By dividing up the links into the probable sides of a communication, you can manually ensure that both physical links are used for inter-VLAN communications.

22. You chose to use OpnSense under virtualization, like with Proxmox? There may be some pitfalls you do not know yet.

23. I generally recommend to apply the appropriate CPU microcode updates, especially if you experience erratic behaviour. The exception to this rule is a virtualized setup, where the microcode updates must not be applied on OpnSense, but on the virtualisation host platform.
You use an N-series CPU from Intel or any Intel CPU from Alder Lake or Twin Lake series? See this. Also, these machine have frequent lockups on their network interfaces with ASPM enabled. You can use the tuneable "hw.pci.enable_aspm = 0" to disable ASPM if your BIOS does not support it. Also, see this.

BTW, if you look for a cooler system, take at look at this post (in german or here (translated)).

24. Stop worrying about the automatic firewall rules - they are better designed than you think:

The automatic rules are designed to facilitate practical function of the firewall, and were carefully designed to do so, while still allowing useful customization. OpnSense exposes these rules (under "Automatically generated rules" at the top of each ruleset), allowing users to examine them and understand their function. Logging options are also provided, under "Firewall: Settings: Advanced" -> "Logging". Many other firewalls are deficient in this respect: Local rules tend to be fully or partially hidden, and logging is often at least partially suppressed.

The most common rule of concern is "let out anything from firewall host itself":

Code Select Expand

pass out [log] all flags S/SA keep state allow-opts label "1232f88e5fac29a32501e3f051020cac"As you can see, this rule has no "quick" option defined, which means it is a last matching rule - it executes after your already existing ruleset, so you may override it with your own "quick" outbound rules. This is a choice; it is generally unnecessary. If you are concerned about the rule, you may enable logging and audit its operation.

The second most common rule of concern is the first "IPv6 RFC4890 requirements (ICMP)":

Code Select Expand

pass in [log] quick inet6 proto ipv6-icmp all icmp6-type unreach keep state label "e37fba608a73a92ed0a9d73645cdc15c"
pass in [log] quick inet6 proto ipv6-icmp all icmp6-type toobig keep state label "e37fba608a73a92ed0a9d73645cdc15c"
pass in [log] quick inet6 proto ipv6-icmp all icmp6-type neighbrsol keep state label "e37fba608a73a92ed0a9d73645cdc15c"
pass in [log] quick inet6 proto ipv6-icmp all icmp6-type neighbradv keep state label "e37fba608a73a92ed0a9d73645cdc15c"
While these rules do open a window that may not be closed, they are mandatory for proper IPv6 operation. Also, consider that the source must have access to the target address space, and the attack would be - at worst - a resource consumption attack on the target. An accessible target is rather likely to have more exploitable vectors than this.

Incidentally, since 25.7.6, you can disable many of the default rules from the firewall settings menu, if you absolutely want to.

26. How do I do traffic shaping (and why)?

With up/downstream speeds differing by a huge amount (as is usual for nowaday's consumer connections), you can easily outrun buffers, such that TCP algorithms tend to start "pumping", which is called bufferbloat.
By throttling the traffic at the source when it becomes too much, you avoid that pumping effect. In other words: If your router limits bandwidth closely before the actual limits are reached, your overall experience will be better.

Recently, some fixes to the traffic shaper have been implemented and we found that control plane shaping does the trick, which is now even in the docs.

So, in short:

1. Follow the instructions in the docs., but do not use up all the available bandwidth as the limit, since there will be no more headroom left for shaping.
2. Configure control plane shaping that gives a higher priority to traffic that needs it most. You can implement separate queues for ICMPv4 and ICMPv6.

27. Do not enable any type of hardware offloading for your network interfaces. Look at the documentation for why. Enabling hardware offloading will (at least) interfere with IPSs (e.g. zenarmor and suricata) and has virtually no benefits. Also see this - you have been warned.

28. You cannot find plugin "X"? On the top right of the plugins screen, check the "show community plugins" box.

29. If you want to redirect NTP or DNS requests to OpnSense, maybe in trying to use the local resolver only, you can create port-forwards to do so. However, while you can create a redirect to "localhost" (i.e. 127.0.0.1) with IPv4, the same does not work in IPv6, because of RFC4912, section 2.5.3. Essentially, the IPv6 loopback address ::1 will never be routed outside of the host. You will find one of the discussions about that starting here.

Also, for redirecting DNS requests, keep in mind that there are several types of DNS messages, like queries and updates. The latter will not be forwarded by Unbound, Bind or DNSmasq. Thus, if one of your LAN hosts tries to send an RFC2136 DNS update in order to change a DNS entry for DNS-01 validation, it would fail if all DNS requests are being port-forwarded to your internal DNS service. You will have to exclude such clients from the port-forward.

30. Your Wireguard connection will not start up after a reboot or WAN reconnection or it loses its connection to the other side, waiting forever? The reason in the first case that if your DNS is not ready during startup, Wireguard cannot resolve the name of your peer(s). The reason in the second case is probably, that your peer has changed its address, but Wireguard does not notice and never re-resolves the peer name. The remedy to both problems is to activate a periodic cron job "Renew DNS for Wireguard on stale connections".

31. You can make port-forwarding work for IPv6, but not for IPv4 or you cannot reach opened IPv4 ports on your OpnSense? Look at your WAN IPv4. Does it start with 100? Also, start this command from a CLI from your OpnSense box:

Code Select Expand

curl -4 ifconfig.me
If either your WAN IPv4 starts with 100 or the WAN IPv4 differs from what the command shows you (or both), you probably have a CG/NAT aka DS-Lite connection. With that, your ISP does not give you a routeable IPv4, but translates your WAN IPv4 via NAT to a routeable IPv4 that is used outside of the provider network.

Therefore, you share your "outside" IPv4 with many other customers. This is a case of double-NAT, where your ISP would have to create a port-forwarding rule for you, which he doesn't. So, if you find yourself in that situation, you cannot port-forward IPv4 and must find other means for outside-in access, like IPv6-only or create a tunnel connection (e.g. via Cloudflare or a VPN to a cloud instance that provides IPv4). Some ISPs also offer "real" IPv4 for a fee.

32. Never, ever, bind any service to a specific (set of) interface(s). To answer the "why": When you do that, the interface must be configured before the service is started. Often times, that is beyond your control, so it won't always work - and if it eventually fails, you wonder why. If instead you leave that out, the service will bind to "any", which includes interfaces that pop up later. You should restrict access via firewall rules instead.

This list will likely be expanded in the future...

See here.

I upgraded from 24.1.3_1 to 24.1.4 and found that on all of my machines, I needed to upgrade twice.

There was an error message during the first update and the upgrade did not catch all upgradeable packages:

pkg-static: Fail to rename /usr/local/etc/rc.d/.pkgtemp.squid.ddRLvpFwCL9y -> /usr/local/etc/rc.d/squid:No such file or directory

On the second try after that, those packages were only then upgraded:

Code Select Expand


cpu-microcode-intel 20231114 20240312 upgrade OPNsense
ruby31-gems 3.4.20 3.5.6 upgrade OPNsense
squid 6.7 6.8 upgrade OPNsense
squid-langpack 7.0.0.20231227 7.0.0.20240307 upgrade OPNsense