Topics

[you should do the same]

I urge every new OpnSense user to carefully plan their subnets - preferably in advance. While this seems obvious or "easy", the devil lies in the details, some of which you may not know or anticipate.

Of course, if you know anything about networks at all, you will have heard about RFC1918.
This standard defines IPv4 ranges that are to be used on local networks and will not be routed into the open internet, thus needing a NAT translation.

There are three available ranges, namely 192.168.0.0/16, 172.16.0.0/12 and 10.0.0.0/8. You might think, well fine, let's just use one of those, but wait...

By convention, the 192.168.0.0/16 range is almost always divided up into smaller /24 subnetworks - and you should do the same. There are some reasons for that:

• If you want to use larger ranges, because you need more than 254 clients within one subnet, think again:
  - More than about 200 clients in one subnet will cause a significant amount of broadcasts. Not only for ARP, but also for proprietary protocols that many IoT and smart devices use. While modern ethernet with full-duplex is nice, it does not help for broadcasts, let alone what broadcasts do on wireless networks (consider separating them from your (V)LANs).
  - Also, you should at least split up your network into several VLANs based on the trustworthiness of your clients by type: Do they "phone home"? Then put them into a separate VLAN! This alone will probably result in less than 200 devices per VLAN/subnet.
  
• If you want to use smaller ranges, take care. For example, you may think that for accessing your modem, which has an 192.168.100.1 address, you could use a very small network containing only two adresses (thus with a netmask of /31), you would be wrong to choose 192.168.100.2 on your WAN interface. Think about why (use a network calculator if needed).
  This example provides two insights:
  a) A netmask of /31 is always wrong, because usually, you have at least a "network", a "broadcast", a router and one client address, therefore you need a /30 netmask at least and
  b) anything apart from a /24 netmask is hard to visually get correct. /24 netmasks are easy to deal with for humans, because their 3rd octet is always the same, so you can see if two IPs belong to the same subnet (hint: 192.168.100.1 and 192.168.100.2 do not lie together in any /31 subnet).
  
• In fact, anything apart from a /24 subnet in the 192.168.0.0/16 range is considered "unconventional" - and if not explicitely stated, will not be implicitely assumed by anyone trying to help you. This may lead to much wasted time while troubleshooting.

If you really need to configure netmasks different from /24, please do so in the 172.16.0.0/12 or 10.0.0.0/8 range. Because these are more often used by businesses or big enterprises, there a lot less assumptions for those.

Another hint:

Do not use 192.168.0.0/24, 192.168.1.0/24, 192.168.2.0/24, 192.168.88.0/24, 192.168.100.0/24, 192.168.123.0/24 or 192.168.178.0/24 for your (V)LAN subnets. This is why:

Those ranges are often the default for many routers, including OpnSense itself. Most people, who often know jack about networking, leave their routers at their default settings. At some point in the future, when you try to build a Wireguard site-2-site VPN to a friend, you will find that one of you has to restructure their whole network to be able to route anything at all. Depending on the complexity of the network, this may be hard task. Also, many ONTs, cable and DSL modems use IPs from these range. If you want them to be made accessible on the WAN interface, you must not have the same range in one of your own (V)LANs. And finally, those ranges are often used by devices in their default state, which may conflict with your network when you connect those.

Before filing a bug report, I rather ask here: Under System: Firmware, you can start a connectivity audit. For me this gives:

Code Select Expand

***GOT REQUEST TO AUDIT CONNECTIVITY***
Currently running OPNsense 25.1.3 (amd64) at Wed Mar 12 13:13:59 CET 2025
Checking connectivity for host: pkg.opnsense.org -> 89.149.222.99
PING 89.149.222.99 (89.149.222.99): 1500 data bytes

--- 89.149.222.99 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
Checking connectivity for repository (IPv4): https://pkg.opnsense.org/FreeBSD:14:amd64/25.1
Updating OPNsense repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: .......... done
Processing entries: .......... done
OPNsense repository update completed. 862 packages processed.
Updating mimugmail repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: ........ done
Processing entries: .......... done
mimugmail repository update completed. 193 packages processed.
All repositories are up to date.
Checking connectivity for host: pkg.opnsense.org -> 2001:1af8:5300:a010:1::1
PING(1548=40+8+1500 bytes) 2001:a61:47f:b942::42 --> 2001:1af8:5300:a010:1::1

--- 2001:1af8:5300:a010:1::1 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
Checking connectivity for repository (IPv6): https://pkg.opnsense.org/FreeBSD:14:amd64/25.1
Updating OPNsense repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: .......... done
Processing entries: .......... done
OPNsense repository update completed. 862 packages processed.
Updating mimugmail repository catalogue...
pkg: https://opn-repo.routerperformance.net/repo/FreeBSD:14:amd64/meta.txz: Unknown resolver error
repository mimugmail has no meta file, using default settings
pkg: https://opn-repo.routerperformance.net/repo/FreeBSD:14:amd64/packagesite.pkg: Unknown resolver error
pkg: https://opn-repo.routerperformance.net/repo/FreeBSD:14:amd64/packagesite.txz: Unknown resolver error
Unable to update repository mimugmail
Error updating repositories!
Checking server certificate for host: opn-repo.routerperformance.net
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = E5
verify return:1
depth=0 CN = opn-repo.routerperformance.net
verify return:1
DONE
Checking server certificate for host: pkg.opnsense.org
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root G2
verify return:1
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL TLS RSA CA G1
verify return:1
depth=0 CN = pkg.opnsense.org
verify return:1
DONE
***DONE***

I see a problem with this:

The ping being called for both IPv4 and IPv6 targets is almost impossible to succeed, because it is being called via "ping -c4 -s1500". Even with a normal ethernet connection without VPN, VLAN or PPPoE overhead, the maximum payload size that is feasible is 1472 for IPv4 and 1452 bytes, respectively, so these pings will almost never work. So, shouldn't the "-s" option be removed altogether?

The "unknown resolver" problem is just because the mimugmail repository does not have an IPv6 address, so we can disregard that.

There have been multiple threads about problems with IPv6 configuration lately.

While there are instructions in the documentation, they leave some vital parts as an exercise to the reader.

Preface:

Despite there being enough IPv6 address prefixes with the abundance of 2^128 possible IPs, many ISPs do not provide you with a static IPv6 prefix like the RFCs propose. Some of them will give you just one /64 range, which is awkward, because you usually need one /64 for each of your subnets/VLANs. Even if they provide a /56 or /48 prefix, which you can then subdivide, some ISPs change the ranges regularly or at least when the connection drops. I think this is to make it harder to host services like a business customer would and thus to differentiate business and consumer uplinks. You can try "prevent release" and setting the DUID under "Interfaces : Settings", but usually, it does not help.

With OpnSense, it is possible to create firewall aliases of type "Dynamic IPv6 Host", that specify only the EUI-64 part of the address and are provided with the 64 bits long prefix of the (V)LAN interface. However, anything else than static DHCP prefixes neccessitate dynamic DNS for IPv6 if you want to make services available from outside. This is sometimes difficult when there are different IP prefixes for your WAN (IA_NA) and (V)LAN (IA_PD) ranges.

So, mostly, you want to have inside-out IPv6 access first, potentially using IPv6 privacy extensions for this in order to hide your identity.

If you also want to expose services, I recommend to use reverse proxies like HAproxy, Caddy or Nginx - there are tutorials on how to use them here on the forum. The reverse proxies can also convert from incoming IPv6 connections to internal IPv4 services, which further reduces complexity for these scenarios. Also, you can (and should) use wildcard certificates if ever possible, because named certificates will be published and can be used to scan for IPv6-based services where a normal port scan is infeasible.

When you are behind CG-NAT, your only chance to expose services is via a reverse tunnel like Cloudflare or to use IPv6. In the latter case, a reverse proxy will work as well.

You can also expose individual (V)LAN clients via their EUI-64, but you will need both firewall aliases and a decent dynamic DNS to do it.


The way I do this is as follows:

1. I configure the WAN interface to request an IPv6 prefix via DHCPv6.

You cannot view this attachment.

Usually, this will be a /56 or a /48 prefix, depending on your ISP. It must be < /64 for the rest to work.
I also use "Request prefix only", because some ISPs do not hand out IA_NA (i.e. a /128 WAN IPv6 GUA) anyway and some even refuse to respond to DHVPv6 requests for it. That way, I will only get a IA_PD IPv6 GUA prefix and no WAN IPv6, but that can actually be turned into an advantage:

If I got an IA_NA, it would be from another range than the IA_PD. Thus, when I update a dynamic DNS entry from OpnSense, the IA_NA prefix will get registered and I cannot make LAN clients available, because they use the other (IA_PD) prefix. So, it is better to use the IA_PD prefix for the WAN interface, too. Of course, you have to be able to set or fix the lower 64(+) bits of the DNS entry to the EUI-64 of the clients if you want to expose them, because the used IPv6 will be the one from OpnSense's WAN interface, that differs in the lower 64 plus interface bits.

It is essential to specify an "Optional prefix ID" that is different from all (V)LAN prefixes in this case, because each interface has to have a different prefix ID. Consider a /56 prefix from the provider that is: 1111:2222:3333:44XX:YYYY:YYYY:YYYY:YYYY/56. Then, XX denotes the prefix ID, leaving the 64 lower bits (YYYY) for SLAAC assignments on the repective interface.

You can leave the "Optional interface ID" empty, in which case the EUI-64 that is derived from the MAC of the WAN interface will be taken as a default. This in turn leaks information about the manufacturer of your WAN NIC via its OUI, so you may want to specify something arbitrary (or memorable) here.

2. I use SLAAC only for my (V)LAN interfaces, not DHCPv6.

You cannot view this attachment.

That is for several reasons:

• DHCPv6 is not understood by many devices (like Android).
• DHCPv6 is unable to specify more than one prefix (it cannot assign ULA IPv6 on top of GUA IPv6, e.g.).
• DHCPv6 will keep your clients to use an outdated, non-working prefix in case your connection drops or your ISP changes GUA prefixes regularly.
• By using SLAAC, you also can use IPv6 Privacy Extensions (RFC 8981) on the clients, still leaving an EUI-64 based management IPv6 for predictable inbound use.

SLAAC is done via router advertisements (RA) and can also handle DNS server and gateway announcements.

Therefore, I use these settings:

• IPv6 Configuration Type = Track interface
• Parent interface = WAN
• Assign prefix ID = <different for each interface>
• Optional Interface ID = empty
• Manual configuration = checked

The prefix ID can be only N bits long, where N = 64 minus the ISP-provided prefix size, e.g. 8 bits when /56 is used or 16 bits with /48.
On (V(LAN interfaces, I usually do not specify an interface ID.

Under Services: Router Advertisements, I use these settings for any (V)LAN:

You cannot view this attachment.

• Router Advertisements = Unmanaged (meaning SLAAC only)
• Router Priority = normal
• Source Address = automatic
• Advertise Default Gateway = checked
• DNS options = both unchecked (meaning use the system DNS servers
• DNS servers = empty
• Domain search list = whatever you need
• Minimum Interval = 30
• Maximum Interval = 60

I use rather short intervals to distribute RAs quite often and also, to have a short lifetime in case the WAN interface changes its GUA. I want to keep a potential use of deprecated prefixes as short as possible. This can have an averse effect on battery life for WLAN clients.

Mind you, there may be reasons why RAs do not work for your clients. You can always look at the resulting RA settings by inspecting /var/etc/radvd.conf and see if the GUA prefix was picked up and if your (V)LAN prefixes and other settings seem right. Then, you can look at your client if it really receives the RAs.

3. A neat trick: If you want (or need) to configure your OpnSense to announce itself for any service, like, as a gateway, NTP or DNS server, you can always use its link-local address - because you do not know the GUA beforehand with changing prefixes. You can even create a virtual IP like fe80::1/64 on each local interface and use that instead. This is also easier to memorize for static configurations.

4. If you use VLANs and want to block inter-VLAN traffic like you probably do for IPv4: Instead of the often suggested approach to first block destination "RFC1918" and after that, allow "any" (or just allow "!RFC1918"), I define an interface group for all local protected VLANs and then use one IPv4+IPv6 blocking rule with a target of "LOCAL_VLANS net". This would block local traffic destined for the firewall (as does using "RFC1918" there, but is restricted to IPv4), so any more specific allow rules should be placed into the floating rules or put before these blocks.

5. If you want to make a local client accessible from the outside or if you need specific firewall rules, you may face the problem of changing IPv6 prefixes, like many ISPs offer. You can use a "Dynamic IPv6 host" type alias, which only specifies the EUI-64, i.e. the lower 64 bits of the IP. If you go that route instead of offering access via a reverse proxy, keep in mind that such rules would best be place in the floating rules, because, if you allow outside access, then why should you block inter-VLAN traffic?

Also, such a "Dynamic IPv6 host" alias can also be used as the target of a NAT rule, thus, you can make OpnSense forward ports to your internal host, while still being accessible by OpnSense's WAN IPv6, thus making dynamic DNS maintainable by OpnSense.

However, because you will find that handling dynamic DNS for your internal services is sometimes complicated, I recommend using a reverse proxy, like Caddy, Nginx or (preferably) HAproxy. That way:

a. OpnSense can handle DynDNS for the services - it always fits because the IPv6 is then always the WAN IP of OpnSense and the internal servers can be addressed via IPv4.

b. the services can be addressed by DNS name, whereby you can also work with wildcards (e.g. *.domain.de) so that new services/hosts work without further DynDNS adjustments. The wildcard address is directed to the DynDNS name.

c. TLS termination and obtaining certificates are carried out by OpnSense, too, which can also be done via wildcard, so you only need one certificate for everything. This makes distributing names to internal servers super easy. With HAproxy, for example, the new service/name is then just a "real server", a "backend" and a line in the map file. Wildcard certificates are also more secure because you cannot easily see which names really exist (see https://crt.sh).

There is a guide on how to setup reverse proxies for HTTP(S), and here you can find how to do it for SMTP and IMAP.

[MTU - how do I set it?]

So, because still some people might have problems understanding what this is all about, despite there is already another good guide about this:

https://forum.opnsense.org/index.php?topic=21207.0


MTU - how do I set it?

First off, you must know that on you LAN, the default MTU size will most likely be 1500 Bytes.
This is not the same as the ethernet frame size of 1518 Bytes, which includes the MAC and frame check sequence.
Also, this is not the maximum data payload size, called MSS (maximum segment size), which is then 1460 Bytes.

The physical reality is shown here: https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/appneta/GA/analyze-results/network-performance-monitoring-delivery/delivery-results/delivery-diagnostics-messages/maximum-transmission-unit.html

Ignore VLANs on your own network for now, I will explain that later on.


Let's focus on MTU only. When a network packet is transmitted over your WAN interface, you can get lucky and it only gets converted to another medium by an NT (optical or DSL). This is often the case when your WAN configuration only has DHCP and nothing more.

However, there can be setups where your WAN interface is over a VLAN. The VLAN tag is 4 bytes in length, and because the interfaces are stacked on one another, you will see that under OpnSense, your WAN interface may have a VLAN name like "igc0_vlan40". Thus, the lower layer interface has to accommodate both the packet payload and the VLAN header. Since the default MTU of your igc0 ethernet interface is 1500 bytes, the VLAN interface's MTU can only accomodate 1496 bytes, and accordingly, the MTU of igc0_vlan40 will be 1496 bytes.

What happens now is that packets destined to a target on the internet may have to be split up, e.g. when they carry a payload of 1500 bytes, into one packet with 1496 bytes and one with 4 bytes.
Because of the inter-frame gaps and roundtrip times, the 4 byte packet is suboptimal and will cause a slowdown. It would be beneficial if your WAN MTU matches your LAN MTU.

There can also be the case that your internet connection uses PPPoE, which incurs an overhead of 8 bytes, giving you only a net payload MTU of 1492 Bytes. The worst case would be that the PPPoE connection goes through a VLAN, so have have a stack of WAN (PPPoE, 8 bytes overhead) over a VLAN (4 bytes overhead) on an ethernet port. In such cases, you only have 1500 -8 - 4 bytes MTU, leaving you at 1488 bytes.

Both the "normal" ethernet setup and the "worst case" setup are depicted in the image I appended.

OpnSense tries to guess the resulting MTUs, but it sometimes fails.

It is essential that your MTU size is not set too high, because some internet sites cannot correctly train their TCP packets to match the maximum available MTU (this mechanism is called path MTU discovery).
When you try to contact such a site, packets may get lost on the way, such that you effectively cannot reliably work with them.

There is a Linux tool to find out what maximum MTU your WAN will accomplish here. I added a FreeBSD version of the script at the end, it can be run under OpnSense CLI.

You should set your WAN MTU to the value that the tool will tell you can achieve.

However, as I said, ideally, you will want to match your WAN MTU to your LAN MTU and sometimes, this is feasible.
It clearly depends on your hardware and that of your ISP if that works, because you will have to use a non-standard ethernet packet MTU, a kind of "jumbo" frame. BTW: A proxmox VirtIO interface (vtnetX) will need configuration for that under Proxmox itself.
For the worst-case scenario with both PPPoE and VLAN involved, you would theoretically need an MTU of 1512 on the ethernet port, 1508 on the VLAN created on it and then 1500 for PPPoE. This is the third case in the appended image.

As an example, if you have a WAN over PPPoE over VLAN, you should have to set WAN MTU = 1500, PPPOE VLAN = 1508, ethernet port = 1512, and it really only works for me with these tightly controlled MTUs:

pppoe0 MTU: 1500 (this must only be set in the advanced settings of pppoe0, not on the WAN interface itself, see screendumps). This will set the WAN MTU.
ONT MTU: (this means the physical ethernet port): 1512 if you have a VLAN for PPPoE, 1508 if not.
PPPOE VLAN MTU: 1508 (if needed in your setup).

To be able to set these values, you will have to assign each of the underlying interfaces a name, which you would normally not need to.

Also, set the above values in the web UI and then reboot, if in doubt - the values sometimes cannot be successfully changed via UI manipulations, because the order of application seems to be wrong that way.

Afterwards, verify that the shown MTUs are as expected via "ifconfig". Also verify that the achievable MTU over your WAN connection works with the tool I linked above. Ideally, you should be able to do get ping replies this command:

"ping -4 -c4 -D -s 1472 1.1.1.1", resulting in an unfragmented 1500 byte packet.

But not to this:

"ping -4 -c4 -D -s 1474 1.1.1.1", resulting in one oversized packet that cannot be transmitted because of the DF bit set.

Usually, you will also get an answer, when you use larger packets, but only when you allow for fragmentation like this:

"ping -4 -c4 -s 1480 1.1.1.1", resulting in two fragmented packets (note the missing "-D" parameter!).


P.S.: As promised, about your local VLANs: Theoretically, the calculations above would also apply to local ethernet ports, but most switches can do jumbo frames on their switching fabric. Also, usually, you will fan out those VLANs on ports as untagged anyway. Therefore, OpnSense thinks it is best to set a VLAN's MTU also at 1500 bytes. That usually works, even when you use a trunk port.
Most network hardware is capable of doing jumbo frames (up to 9014 bytes) anyway. So, normally, you do not have to think about the theoretical difference for an untagged or a tagged VLAN and can use 1500 bytes MTU for either of them.

P.P.S.: If you use Proxmox or other types of virtualisation, you must set the larger MTU of 1512 or 1508 on the underlying host interface(s) as well. For Proxmox, this would be the bridge and ethernet interface for the WAN port for VTNET interfaces or only the ethernet interface with pass-through.

I saw this error in my wireguard log after upgrading to 25.1:

Code Select Expand

2024-11-02T12:41:43 Error wireguard /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: The command '/usr/bin/wg syncconf 'wg0' '/usr/local/etc/wireguard/wg0.conf'' returned exit code '1', the output was 'Name does not resolve: `xxx.yyy.de:6010' Configuration parsing error'

The corresponding line in the "Peer" section of /usr/local/etc/wireguard/wg0.conf is:

Code Select Expand

Endpoint = xxx.yyy.de:6010

This configuration has not been hand-edited, it was created via the web UI.

This is about a brand-new Unifi USW-Pro-HD-24-POE switch. I though I write this here to save others the time to find out themselves:

I recently bought one of those switches and wanted to use the LAGG feature with my OpnSense box with 4 I226V NICs.

My previous setup was to use two of those interfaces, one carrying my main (V)LAN and the other to carry all other VLANs, e.g. IoT. With that, I could have cross-VLAN traffic at 2.5 Gbps in both directions.

So I tried setting up a LAGG with LACP layer 4, in order to have TCP ports come into the mix for hashing and distributing different TCP streams over two physical links. I tested with "iperf3 -c x.x.x.x -P4" between two Linux VM hosts on different VLANs to have multiple source ports and to my great disbelief, got only 2.5 Gbps. The same thing happened when I tried against OpnSense itself.
I also tried two different iperf3 instances from the same machine to no avail.

After some more trial and error, I found that the Unifi switch can only handle layer 2 & 3 hashing, which is especially disappointing when you know that they did better years ago. Only when I had two different clients do independent runs, could I have > 4 Gbps in sum - and even this took a few tries before I found a counterpart machine with a matching MAC.

I then found this tidbit in the CLI of the switch:

Code Select Expand

BusyBox v1.25.1 () built-in shell (ash)


  ___ ___      .__________.__
 |   |   |____ |__\_  ____/__|
 |   |   /    \|  ||  __) |  |   (c) 2010-2024
 |   |  |   |  \  ||  \   |  |   Ubiquiti Inc.
 |______|___|  /__||__/   |__|
            |_/                  https://www.ui.com

      Welcome to UniFi USW-Pro-HD-24-PoE!

********************************* NOTICE **********************************
* By logging in to, accessing, or using any Ubiquiti product, you are     *
* signifying that you have read our Terms of Service (ToS) and End User   *
* License Agreement (EULA), understand their terms, and agree to be       *
* fully bound to them. The use of SSH (Secure Shell) can potentially      *
* harm Ubiquiti devices and result in lost access to them and their data. *
* By proceeding, you acknowledge that the use of SSH to modify device(s)  *
* outside of their normal operational scope, or in any manner             *
* inconsistent with the ToS or EULA, will permanently and irrevocably     *
* void any applicable warranty.                                           *
***************************************************************************

hagen-US3.7.1.33# cli
hagen# configure
hagen(config)# lag load-balance
Incomplete command
hagen(config)# lag load-balance ?
  src-dst-mac     LAG load balancing is based on source and destination MAC addr
                  ess.
  src-dst-mac-ip  LAG load balancing is based on source and destination of MAC a
                  nd IP addresses.
hagen(config)#

This shows that only layer 2 & 3 are being supported, as my tests confirmed.

Actually, for my scenario, LAGGs are worse without layer 4 hashing than splitting the VLANs across the interfaces, as I would have to be lucky (chances are 50:50) to have two arbitrary devices be put on different links via layer 2 or 3 hashing, even when I use multiple TCP streams with different ports.

I also posted this to the Unifi forum.

Oh, BTW: Today, I looked at my USW-Enterprise-24-PoE. It has a completely different CLI structure and obviously, this one does in fact support LACP layer 4:

Code Select Expand

BusyBox v1.25.1 () built-in shell (ash)


  ___ ___      .__________.__
 |   |   |____ |__\_  ____/__|
 |   |   /    \|  ||  __) |  |   (c) 2010-2024
 |   |  |   |  \  ||  \   |  |   Ubiquiti Inc.
 |______|___|  /__||__/   |__|
            |_/                  https://www.ui.com

      Welcome to UniFi USW-Enterprise-24-PoE!

********************************* NOTICE **********************************
* By logging in to, accessing, or using any Ubiquiti product, you are     *
* signifying that you have read our Terms of Service (ToS) and End User   *
* License Agreement (EULA), understand their terms, and agree to be       *
* fully bound to them. The use of SSH (Secure Shell) can potentially      *
* harm Ubiquiti devices and result in lost access to them and their data. *
* By proceeding, you acknowledge that the use of SSH to modify device(s)  *
* outside of their normal operational scope, or in any manner             *
* inconsistent with the ToS or EULA, will permanently and irrevocably     *
* void any applicable warranty.                                           *
***************************************************************************

edgar-US.7.1.26# cli

Entering character mode
Escape character is '^]'.

Warning!
The changes may break controller settings and only be effective until reboot.

(UBNT) >enable

(UBNT) #configure

(UBNT) (Config)#port-channel load-balance ?

1                        Src MAC, VLAN, EType, incoming port
2                        Dest MAC, VLAN, EType, incoming port
3                        Src/Dest MAC, VLAN, EType, incoming port
4                        Src IP and Src TCP/UDP Port fields
5                        Dest IP and Dest TCP/UDP Port fields
6                        Src/Dest IP and TCP/UDP Port fields

(UBNT) (Config)#port-channel load-balance

So, it seems model-dependend, probably based on product lines (i.e. Pro vs. Enterprise).

I just got the dreaded MOZILLA_PKIX_ERROR_REQUIRED_TLS_FEATURE_MISSING error message, because my preferred certificate has OCSP stapling enabled. I want to use that feature, because my domain is used for other purposes as well with a wildcard certificate.

I found https://forum.opnsense.org/index.php?topic=26812, so is OCSP stapling still infeasible with lighthttpd?

Ah, apparently, the feature is available now: https://redmine.lighttpd.net/projects/lighttpd/wiki/Docs_SSL#OCSP-Stapling, I'll create a ticket.

[RAM, CPU and system]

These days, there are many folks who use OpnSense under a virtualisation host, like Proxmox, for example.

This configuration has its own pitfalls, therefore I wanted to have this guide. The first part starts with common settings needed, the second part will deal with a setup where the virtualisation host is to be deployed remotely (e.g. in a datacenter) and holds other VMs besides OpnSense.

RAM, CPU and system

Use at least 8 GByte, better 16 GBytes of RAM and do not enable ballooning. Although OpnSense does not need that much RAM, it can be beneficial in case you put /var/log in RAM (see below).

Obviously, you should use "host" CPU type in order not to sacrifice performance by emulation. However, you should not install the microcode update packages in OpnSense - they would be useless anyway. Instead, install the appropriate microcode packages on the virtualisation host.

That being said, just for good measure, set tuneables "hw.ibrs_disable=1" and "vm.pmap.pti=0". This wil avoid performance bottlenecks because of Spectre and Meltdown mitigations. I trust the other VMs in my setup, but YMMV...

The system architecture is arbitrary, as OpnSense can boot both in legacy (BIOS) or UEFI mode.

Filesystem peculiarities

First off, when you create an OpnSense VM, what should you choose as file system? If you have Proxmox, it will likely use ZFS, so you need to choose between UFS and ZFS for OpnSense itself. Although it is often said that ZFS underneath ZFS is a little more overhead, I would use it regardless, just because UFS fails more often. Also, OpnSense does not stress the filesystem much, anyway (unless you use excessive logging, RRD or Netflow).

32 GBytes is a minimum I would recommend for disk size. It may be difficult to increase the size later on.

After a while, you will notice, that the space you have allocated for the OpnSense disk will grow to use 100%, despite that within OpnSense, the disk may be mostly unused. That is a side-effect of the copy-on-write feature of ZFS: writing logs and RRD data and other statistics always writes new data and the old data does not get dismissed against the underlying (virtual) block device.

That is, if the ZFS "autotrim" feature is not set manually. You can either set this via the OpnSense CLI with "zpool set autotrim=on zroot" or, better, add a daily cron job to to this (System: Settings: Cron) with "zroot" as parameter.
You can trim your zpool once via CLI with "zpool trim zroot".

That being said, you should always avoid to fill up the space for the disk by having verbose logging. If you do not need to keep your logs, you can also put them on a RAM disk (System: Settings: Miscellaneous).

Network "hardware"

With modern FreeBSD, there should not be any more discussion about pass-through vs. emulated VTNET adapters: the latter are often faster. This is because Linux drivers are often more optimized than the FreeBSD ones. There are exceptions to the rule, but not many.

In some situations, you basically have no choice than to use vtnet anyway, e.g.:

• If FreeBSD has no driver for your NIC hardware
• If the adapter must be bridged, e.g. in a datacenter with a single NIC machine

Also, some FreeBSD drivers are known to have caused problems in the past, e.g. for RealTek NICs. By using vtnet, you rely on the often better Linux drivers for such chips.

With vtnet, you should make sure that hardware checksumming is off ("hw.vtnet.csum_disable=1", which is the default on new OpnSense installations anyway because of a FreeBSD interoperability bug with KVM). Note, however, that this setting will be slower than using hardware offloading, which you will notice at very high speeds, especially on weak hardware.

You can also enable multiqueue on the VM NIC interfaces, especially, if you have multiple threads active. There is no need for enabling this in OpnSense.

For some Broadcom adapters (and possibly other, too), it is neccessary to disable GRO by using:

Code Select Expand

iface enp2s0f0np0 inet manual
    up ethtool --offload $IFACE generic-receive-offload off

See: https://forum.opnsense.org/index.php?msg=233131, https://help.ovhcloud.com/csm/en-dedicated-servers-proxmox-network-troubleshoot?id=kb_article_view&sysparm_article=KB0066095 and https://www.thomas-krenn.com/de/wiki/Broadcom_P2100G_schlechte_Netzwerk_Performance_innerhalb_Docker.

When you use bridging with vtnet, there is a known Linux bug with IPv6 multicasting, that breaks IPv6 after a few minutes. It can be avoided by disabling multicast snooping in /etc/network/interfaces of the Proxmox host like:

auto vmbr0
iface vmbr0 inet manual
    bridge-ports eth0
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094
    bridge-mcsnoop 0


If you plan to enlarge your MTU size on VirtIO network interfaces, note that you must do so on the Proxmox bridge device first.

Also, you probably should disable the firewall checkbox for the network interfaces in the OpnSense VM.

Guest utilities

In order to be able to control and monitor OpnSense from the VM host, you can install the os-qemu-guest-agent plugin.


Problems with rolling back

One of the main advantages of using a virtualisation platform is that you can roll back your installation.

There are two problems with this:

1. DHCP leases that have been handed since the time of last roll back are still known to the client devices, but not to the OpnSense VM. Usually, this will not cause IP conflicts, but DNS for affected devices may be off intermediately.

2. If you switch back and forth, you can cause problems with backups done via os-backup-git. This plugin keeps track on both the OpnSense VM and the backup repository. If both are of a different opinion about the correct revision of the backup, subsequent backups will fail. Basically, you will ned to setup the backup again with a new, empty repository.

If you want to avoid such problems, you can roll back single packages with opnsense-revert.


TL;DR

• Have at least 8 GByte of RAM, non-balooning
• Use "host" type CPU and disable Spectre and Meltdown mitigations
• Use ZFS, dummy
• Keep 20% free space
• Add a trim job to your zpool
• Use vtnet, unless you have a good reason not to
• Check if hardware checksumming is off on OpnSense
• Disable multicast snooping and Proxmox firewall
• Install os-qemu-guest-agent plugin

That is all for now, recommendations welcome!

I just upgraded 3 machines from 24.7.4_1 to 24.7.5. All of them completed the upgrade, showed a popup that they now reboot, but did not. After 2 minutes, the popup clears and the dashboard shows with 24.7.5, but a reboot has not occurred as the uptime shown by "w" still was a few days.

[READ THIS FIRST]

I always threatened that at some point I might be inclined to write something like this, because I find myself answering the same questions time and time again.

So, reflecting an old article of the great Dave Barry which starts:

READ THIS FIRST

Congratulations! You have purchased an extremely fine device that would give you thousands of years of trouble-free service, except that you will undoubtly will destroy it via some typical bonehead consumer maneuver.

Which is why we ask you to PLEASE FOR GOD'S SAKE READ THIS OWNER'S MANUAL CAREFULLY BEFORE YOU UNPACK THE DEVICE.

YOU ALREADY UNPACKED IT, DIDN'T YOU? YOU UNPACKED IT AND PLUGGED IT IN AND TURNED IT ON AND FIDDLED WITH THE KNOBS, AND NOW YOUR CHILD, THE SAME CHILD WHO ONCE SHOVED A POLISH SAUSAGE INTO YOUR VIDEOCASSETTE RECORDER AND SET IT ON "FAST FORWARD", THIS CHILD ALSO IS FIDDLING WITH THE KNOBS, RIGHT?

WE MIGHT AS WELL JUST BREAK THESE DEVICES RIGHT AT THE FACTORY BEFORE WE SHIP THEM OUT, YOU KNOW THAT?

Please, read this before you ask questions here:

1. Learn basic networking skills before trying to use OpnSense. OpnSense is a firewall, yes, but in the first place it is a  router (unless you use it as a transparent bridge). Know that you must have disjoint subnets on different interfaces in order to be able to control traffic at all. Many questions hover around topics like: "Why can't I filter traffic between 192.168.23.10 and 192.168.23.17?" (It will not work, that is basic networking 101). Also: get your subnets straight and non-overlapping, as this may have all kinds of strange effects.

2. This includes scenarios where you have multiple clients attached to different ports of your OpnSense. That would  be called a bridge, which is usually discouraged ("use a real switch"), but can be done, if done correctly ("read the docs" and follow them, stupid). In order to not get confused, do not even assign names to bridge member interfaces, because you will not use them in rules or anything but the bridge definition. Do not forget the tuneables ("Which tunables?" - read the docs again!).
 
3. You have created a VLAN but cannot get internet access? You have to create an outbound "allow all" firewall rule like the one that is created per default on your LAN. Remember? "Anything, that is not explicitely allowed, is forbidden." - from the great book: Purpose of a firewall, chapter 1
 
4. Try to avoid router-behind-router scenarios. You either would need double NAT, which is awkward, or be able to control local routing in your ISP router, which you probably are not.
That in turn may be more costly than you think, which I explained here (but in german). For german Fritzbox converts, there is a special article here.
 
5. Get to know your hardware. One of the more often asked questions is: "I cannot access my web UI after installation." The problem is probably either that you have no useable NICs (see next point), you plugged your LAN connection to the wrong spot or you left the USB stick in and boot the installation image instead of the configured system. If you absolutely want / must use OpnSense virtualized, read this.
 
6. Realtek NICs are badly supported under FreeBSD. That concerns performance, VLANs and more generally, basic operation.  There are OEM drivers which can be installed (System:Firmware:Plugins, os-realtek-re), but in some situations, this is difficult, because you cannot even load them from the internet ("use a USB stick instead").

7. Do not try to use the WLAN adapter on your OpnSense box as a WiFi AP. FreeBSD / OpnSense is just not good (tm) at it. Use dedicated AP(s) for that.
 
8. If something does not work, do not jump to conclusions. Try to pinpoint your problem, e.g. do not say: "ping www.google.com does not work". There are so many problems there:

- ping from where? Client or OpnSense itself?
- What exactly did not work? DNS, IPv4 or IPv6?

Possibly, you know - we don't. So there may be at least 6 separate potential problems that were subsumed in one statement. Use "nslookup www.google.com", "ping 8.8.8.8" and "ping 2600::" instead, from both clients and OpnSense itself. BTW: As far as ping options go: FreeBSD ≠ Linux ≠ Windows.

9. Only some sites are unreachable, like OpnSense.org? Even if basic DNS, routing and firewalling works for both IPv4 and IPv6, there may still be other problems lurking, like wrong MTU settings on WAN because of encapsulation via PPPoE, VLANs or IPoE, which you did not account for. In that case, some sites with defective PMTU discovery may fail.
There is a tool to check how big your WAN MTU may safely be set for a specific site.

10. You do not get close to your ISP's advertised speed? Some CPUs are not sufficient for fast internet connections. Look for ones with high single-thread performance, especially, if you need PPPoE, VPN and/or Zenarmor. For more than 1 GBps, you may need something along the range of an N100, for more than 2.5 Gbps you will look at CPUs with even more punch. There are many websites to compare CPU speeds or look in the hardware and performance section. That being said, hand-me-down systems often are sub-par w/r to being used for OpnSense, because older CPUs tend to use way more power such that the energy cost may soon outweigh the savings in a 24/7 scenario.
 
11. If you aim to access your ONT on the WAN side, you will need a virtual IP within the same subnet plus an outbound NAT rule for that, because the ONT does not have a default gateway and does not know how to access anything beyond its own subnet.
 
12. If you think you can control on what enters your network with a firewall, notice there are two basic ways:
  a. you can control specific domains via pre-manufactured DNS-based black- or whitelisting or
  b. you can inspect traffic by intercepting it with a web proxy (this would also allow a centralised virus scanner)
Now forget b. Why? Because for the most part, web traffic is encrypted these days. In order to inspect it, you will have to de- and then re-encrypt it.  Since the re-encryption is done by your firewall, its "faked" TLS certificates must then be trusted by the client devices, which is difficult to do on PCs and mostly infeasible on many smartphones and IoT devices. Such "man-in-the-middle-attacks" are deliberately made hard for good reasons. Thus, you would at least have to use TLS "no bump" for banking sites and also, for many Microsoft URLs.

13. I do not believe in IPSs like Zenarmor, Crowdsec or Suricata, but YMMV. At least do not use Suricata on WAN, unless you are willing to sacrifice IPv6 connectivity. This is a fine example for always having a tradeoff between (perceived) security and useability. Also: If you use IPS and experience any problems, please state that in your posting - or better, disable it and test again! The same goes for any kind of blocklists: check if they are the culprit.

14. Install OpnSense with ZFS, not UFS. You will get a "snapshot" feature that enables you to roll back system upgrades iff you use it before you update. Also, it is more robust against hard resets. If your storage space is small, keep in mind that the default swap size is 8 GBytes - you may need to lower that value during installation to have enough logging space.

15. If something really strange happens, like you cannot log in, try things that should be obvious, but aren't:
  - verify that the system time and date is correct.
  - verify that the file system is not out of disk space (you may have set verbose logs that filled it all up). You can also have logging on a RAM disk if you do not need permanent logs. In that case, a reboot will clear the logs.

16. Whenever you have a question, please, use the forum search. It is highly likely your question has been asked (and answered) many times before. If you still have questions, do not add to or revive another thread if it does not handle your specific problem, but create your own thread with as much information pertaining to the problem, such as:
- OpnSense version
- network topology
- IP ranges
- rules or settings that you use
- hardware (also: if you are using a hypervisor, please tell us!)
If your problem was solved, mark the thead by prepending [SOLVED] in its title. If someone has really helped you, thank him by pressing the [applaud] button under his name on the left.

17. You found that the dashboard temperatures are higher than those reported by "sysctl dev.cpu | fgrep temperature"? Congratulations - but please, do not ask about it or tell us before having consulted one of these threads:

https://forum.opnsense.org/index.php?topic=44373.0
https://forum.opnsense.org/index.php?topic=42575
https://forum.opnsense.org/index.php?topic=36234
https://forum.opnsense.org/index.php?topic=34395
https://forum.opnsense.org/index.php?topic=41759.0
https://forum.opnsense.org/index.php?topic=30293

These turn up in the first 30 search results if you just use "temperature" as keyword. A fine example of what can be achieved by using the search, isn't it?

18. Your IPv6 setup does work work right out of the box? Try this HOWTO.

19. While feature requests and bugs can be discussed here on the forum, thee most likely will never be implemented unless someone (i.e. you!) does a feature request or bug report on Gibthub. If there is a corresponding thread already, please post a link to the feature request or bug report there.

20. Plan your subnets carefully in advance. Too much to discuss here, but I wrote a separate article about this. If you did not and have any problem, explicitely tell us up front!



This list will likely be expanded in the future...

See here.

I upgraded from 24.1.3_1 to 24.1.4 and found that on all of my machines, I needed to upgrade twice.

There was an error message during the first update and the upgrade did not catch all upgradeable packages:

pkg-static: Fail to rename /usr/local/etc/rc.d/.pkgtemp.squid.ddRLvpFwCL9y -> /usr/local/etc/rc.d/squid:No such file or directory

On the second try after that, those packages were only then upgraded:

Code Select Expand


cpu-microcode-intel 20231114 20240312 upgrade OPNsense
ruby31-gems 3.4.20 3.5.6 upgrade OPNsense
squid 6.7 6.8 upgrade OPNsense
squid-langpack 7.0.0.20231227 7.0.0.20240307 upgrade OPNsense


[Ein paar Grundlagen, die man haben sollte]

In letzter Zeit habe ich im OpnSense-Forum immer wieder Fragen auftauchen sehen, die weniger OpnSense speziell, sondern eher Basis-Netzwerkwissen tangierten. Die Fragen waren immer wieder dieselben, oft motiviert von irrigen Annahmen und Missverständnissen, wenn jemand versucht, von einer vollintegrierten Lösung wie einer Fritzbox auf eine anspruchsvollere Lösung wie OpnSense umzusteigen.

Dieser Artikel ist nicht herablassend gemeint, sondern soll vorab klarmachen, worauf man sich dabei einlässt.

Achtung: der Artikel ist lang - wie gewohnt von mir...


Ein paar Grundlagen, die man haben sollte

Zunächst müssen wir uns klarmachen, was das Ziel eines Einsatzes von OpnSense ist: Normalerweise wollen wir unser eigenes Netzwerk (Local Area Network = LAN) mit dem Internet verbinden. Wir beschränken uns zunächst auf das dazu meist verwendete IPv4-Protokoll.

Die IPv4-Adressen von Systemen im Internet bestehen aus 4 Zahlen von 0-255, durch Punkte getrennt, z. B. 1.2.3.4. Damit ist ein bestimmtes System im Internet eindeutig bestimmt. Dar diese Adressraum auf insgesamt ca. 4 Milliarden (=2^32) Addressen beschränkt ist, stehen inzwischen nicht mehr genug IPv4-Adressen zur Verfügung. Insbesondere kann nicht jedes Gerät in unserem LAN eine eigene, öffentlich erreichbare IPv4 erhalten.

Für diese lokalen Adressen wurde deswegen ein eigener, privater Adressraum geschaffen, der im RFC1918 beschrieben ist und diese Bereiche umfasst:

Code Select Expand

10.0.0.0        -   10.255.255.255  (10/8 Präfix)
172.16.0.0      -   172.31.255.255  (172.16/12 Präfix)
192.168.0.0     -   192.168.255.255 (192.168/16 Präfix)

Damit der Zugriff auf ,,öffentliche" Adressen von diesen privaten IPs aus ermöglicht wird, benötigt man einen Router, der in der Lage ist, verschiedene Netzwerkbereiche miteinander zu verbinden. Zum einen hat ein solcher Router mindestens zwei Anschlüsse für verschiedene Netzwerke (beispielsweise 192.168.1.1/24 und 192.168.2.1/24). Dabei hat er im ersten Netz selbst die Adresse 192.168.1.1 und kann somit direkt mit allen anderen Geräten in diesem Netzwerk sprechen (192.168.1.2-255), im zweiten Netzwerk hat er die IP 192.168.2.1 und kann dort mit allen anderen Geräten sprechen. Er weiß auch, dass er Pakete, deren Quell-Adresse im ersten Netzwerk und deren Ziel-Adressen im zweiten Netzwerk liegen, weiterleiten soll (und umgekehrt). Übernimmt dieser Router nebenbei auch noch die Aufgaben einer Firewall, kann er diese Pakete nach bestimmten Regeln auch ausfiltern oder modifizieren.

Innerhalb jedes Netzwerks können sich die Geräte gegenseitig erreichen, weil sie sich per Address Resolution Protocol (ARP) finden können - sie erfahren ihre jeweiligen Kommunikationspartner per Broadcast, also Rundruf. Ein Rundruf wird aber nicht über die Grenzen eines lokalen Netzwerks hinaus gesendet.
Damit die Pakete über den Router geleitet werden, müssen die Geräte im ersten Netzwerk deswegen eine Route kennenlernen, die ihnen vorgibt, dass Pakete für das zweite Netzwerk an den Router mit der IP 192.168.1.1 in ihrem eigenen Netzwerk geschickt werden sollen, denn diese IP können sie ja direkt erreichen.

Diese Route würde also lauten:

Versende 192.168.2.0/24 (d. h. 192.168.2.0-255) über 192.168.1.1

Es gibt auch eine vereinfachte Form, eine sogenannte Standard- oder Default-Route dieser Form:

Versende 0.0.0.0/0 (also: ,,alles andere") über 192.168.1.1

Damit würden alle Pakete über den Router mit der IP 192.168.1.1 geleitet, also zunächst an diesen adressiert.
Nun könnte man erwarten, dass ein solcher Router einfach mit dem Internet verbunden werden muss und an diesem Anschluss keine RFC1918-IP, sondern eine öffentliche IP (z. B. 1.2.3.4) erhält, damit alle Geräte im LAN jedes Ziel im Internet erreichen können. Es ist aber so, dass Adressen aus unserem LAN, die dem RFC1918-Adressraum entstammen, also z. B. die IP 192.168.1.88, im Internet nicht weitergeleitet werden. Man bedenke: Dieselbe IP könnte potentiell von sehr vielen Geräten in den verschiedensten LANs auf der ganzen Welt genutzt werden, sie ist somit nicht eindeutig.

Aus diesem Grund muss ein mit dem Internet verbundener Router normalerweise eine Übersetzung dieser privaten LAN-Adressen vornehmen (Network Address Translation = NAT). Dabei schreibt er die privaten RFC1918-Quelladressen aus dem LAN auf seine eigene öffentliche IP-Adresse um – diese eine IPv4 wird ja korrekt weiter geroutet, da sie weltweit eindeutig ist.

Wenn die Antwort-Datenpakete zurückkommen, hat sich der Router diese Verbindung gemerkt und schreibt die Zieladresse von seiner öffentlichen IP wieder auf die RFC1918-IP des Zielgeräts im privaten LAN um.

Das geht mit einer Einschränkung einher: Um die Zuordnung von Datenpaketen von außen nach innen machen zu können, muss zunächst eine Verbindung von innen nach außen geöffnet worden sein. Es ist also nicht möglich, von außen eine Verbindung zu einem Gerät im LAN zu initiieren.

Diese Form von NAT wird deshalb oft als ,,Outbound NAT" oder SNAT (für Source NAT) bezeichnet.

Um einen umgekehrten Verbindungsaufbau von außen nach innen zu ermöglichen, muss im Router eine sogenannte Port-Weiterleitung eingerichtet werden, durch die Datenpakete, die eine bestimmte Port-Nummer (sinngemäß: Anschlussnummer) zum Ziel haben, immer an eine bestimmte private IP im LAN weitergeleitet werden. Dabei wird die öffentliche Ziel-IP des Routers auf die private IP des internen Geräts umgeschrieben, weshalb diese Form als DNAT (Destination NAT) bezeichnet wird.

Damit lässt es sich z. B. erreichen, dass jede Verbindung zur öffentlichen IP 1.2.3.4 des Routers auf der Portnummer 443 (für HTTPS) an ein Gerät im LAN (z. B. 192.168.1.77) weitergeleitet wird.

Alles schon bekannt? Gut, Auffrischung kann nicht schaden.

Wo kommt OpnSense ins Spiel?

Nun, da die Grundlagen gelegt sind, stellen wir uns die Frage, was OpnSense leistet?

OpnSense kann routen und beherrscht SNAT und DNAT. OpnSense kann darüber hinaus auch zur automatischen Adressvergabe per DHCP (Dynamic Host Configuration Protocol) verwendet werden, das die privaten IPs an LAN-Geräte zuweist und diesen Geräten auch sich selbst als Default-Router bekannt macht. Außerdem kann OpnSense auch Namensdienste (Domain Name System = DNS) bereitstellen und sich den LAN-Geräten als DNS-Server bekannt geben.

OpnSense bietet darüber hinaus weitaus mehr Möglichkeiten, Netzwerkverkehr zu regulieren und zu filtern, als das der übliche ,,Baumarkt-Router" oder eine Fritzbox können – deswegen will man sie ja einsetzen.

Zugangstechnik

Was kann OpnSense denn also nicht, bzw: Was fehlt?

Tatsache ist, dass Zugangsanbieter (Internet Service Provider = ISP) heute die verschiedensten Zugangstechniken anbieten, von DSL (Digital Subscriber Line) über Telefonkabel über TV-Kabelanschlüsse bis hin zu Glasfaseranschlüssen. Es gibt auch weitere Techniken, wie LTE (Mobilfunk) oder Satelliten-Internet.

Das bedeutet, dass in der eigenen Wohnung nicht einfach ein Netzwerkkabel liegt, über das der Router eine öffentliche IP bezieht. Das gilt insbesondere für Deutschland, wo ISPs verpflichtet sind, wahlweise einen rein passiven Netzwerkanschluss zur Verfügung zu stellen.

Nehmen wir als Beispiel DSL:

Der ISP betreibt (meist am Ende der Straße) einen sogenannten DSLAM, der hochfrequente Signale in die vorhandenen Telefonkabel einspeist. Diese landen in einer Anschlussdose in der Wohnung. Der erste Schritt der Umwandlung in das gewünschte Ziel-,,Format", nämlich IP-Adressen über Ethernet ist also eine Medienkonvertierung von DSL nach Ethernet. Dies übernimmt ein sogenanntes DSL-Modem (Modulator/Demodulator). Das Ausgangssignal des DSL-Modems ist zwar bereits Ethernet, meist wird hier aber noch kein IP ,,gesprochen". Das liegt daran, dass der ISP eine Zugangskontrolle realisieren will.

Deswegen kommt – zumindest in Deutschland – oft das PPPoE (Point-to-Point Protocol over Ethernet) zum Einsatz, bei dem man Zugangsdaten (Benutzername und Passwort) benötigt, um eine öffentliche IP zu erhalten. In anderen Ländern und bei manchen Providern ist das unnötig und der Router erhält seine IP direkt per DHCP (diesmal aber nicht in der Rolle als Server, sondern als Client) oder wird statisch konfiguriert (z. B. an Business-Anschlüssen).

Oft wird hierbei auch noch ein Trick verwendet, um einerseits über diese Ethernet-Verbindung den Internet-Anschluss, aber gleichzeitig den Zugriff auf das Modem zu erhalten. Letzterer kann interessant sein, um bestimmte Parameter abzurufen, z. B. Spektralanalysen, Leitungslänge und -qualität usw. - ich erkläre gleich, wie man das ermöglicht.

Ethernet kann entweder normale Datenpakete übertragen oder mittels vorgeschalteten Präfixen (sogenannten ,,Tags") logisch getrennte Kanäle bedienen, die sogenannten VLANs (Virtual LANs). Der Anschluss des Modems läuft dann oft ohne Tags (,,untagged VLAN"), die PPPoE-Verbindung wird dagegen über ein VLAN aufgebaut. Die Telekom verwendet z. B. VLAN 7, M-Net VLAN 40, EweTel VLAN 2011,  Willi Tel VLAN 2511, NetCologne VLAN 10 usw.

OpnSense kann natürlich PPPoE, wahlweise auch per VLAN, sprechen, auch DHCP als Client ist möglich – es beinhaltet aber kein DSL-Modem. Für den Betrieb an einem DSL-Anschluss ist also ein externes DSL-Modem notwendig.

Bei Glasfaser-Anschlüssen, die in Deutschland oft mittels GPON-Technologie ausgeführt werden, tritt an die Stelle des DSL-Modems als ,,Brücke" zum Ethernet ein sogenannter ONT (Optical Network Termination). Auch bei Glasfaser- ISPs wird oft PPPoE verwendet und meist auch VLANs, aus den selben Gründen wie bei DSL-Modems. Auch hier lässt sich am ONT der Glasfaser-Status abfragen, zusätzlich sind sogar weitere Zugangsdaten (Seriennummern, PLOAM-Passworte usw.) darin abgelegt bzw. administrierbar.

Somit lautet die Antwort auf ,,Was fehlt bei OpnSense?" einfach: Der Medienkonverter auf das bereitgestellte Zugangsmedium (Glasfaser, DSL, Kabel o. ä).

Randthema: Wie kommt man über die OpnSense an die Web-Oberfläche des ONT bzw. des DSL-Modems?

Dazu muss man zunächst wissen, welche IP-Adresse das Gerät hat. Es ist meist auch eine RFC1918-Adresse wie 192.168.100.1 oder 10.0.0.1. Man konfiguriert dann für die WAN-Netzwerkschnittstelle an der OpnSense eine andere Adresse aus demselben Adressbereich, also z. B. 192.168.100.2 oder 10.0.0.7 und gibt dieser eine Netzmaske (meist /24).
Hinweis: das private LAN muss dann zwingend einen anderen IP-Bereich nutzen, sonst funktioniert das Routing nicht!

Damit ist die Kommunikation zwischen der OpnSense und dem Modem sofort möglich, aus dem privaten LAN heraus lässt sich das Modem aber noch nicht erreichen. Das liegt am Rückwärts-Routing: Das Modem kennt ja die privaten LAN-Adressen nicht, man müsste also erst auf dem Modem eine Route dorthin konfigurieren, damit die Antworten ihren Rückweg finden. Da dies meist nicht möglich ist, nutzt man wiederum NAT: Auf der OpnSense wird also eine ,,Outbound-NAT"-Regel eingerichtet, die den Verkehr mit Quelladressen aus dem LAN (z. B. 192.168.2.0/24) über die konfigurierte IP am Modem-Port der OpnSense (also z. B. 192.168.100.2 oder 10.0.0.7) umschreibt. Diese IP ist für das Modem ja direkt erreichbar, so dass die Antwort-Datenpakete ihren Weg an den Fragesteller im LAN auch ohne explizite Route zurückfinden.

Man beachte: Die IP am nicht-getaggten Modem-Port ist nicht die WAN-IP, die per DHCP oder PPPoE vom ISP über das verwendete VLAN zugewiesen wurde!

Wieso haben so viele Leute Probleme, die OpnSense einzurichten?

Oft genug ist es so, dass ihnen die Mühe, all das oben dargestellte verstehen zu müssen, bislang mit einer vollintegrierten Lösung abgenommen wurde. Viele glauben aber, sie könnten mit Einsatz einer OpnSense ihr Netzwerk ,,irgendwie sicherer machen".

Wenn man z. B. aktuell eine Fritzbox im Einsatz hat, sind die obigen Differenzierungen komplett egal, weil diese sehr viele Details ganz selbstverständlich regelt, denn sie bietet gleichzeitig diese Funktionen:

• DSL-Modem oder ONT oder Kabel-Brücke (je nach Modell)
• PPPoE- oder DHCP-Router mit SNAT und DNAT
• DHCP- und DNS-Server im LAN
• WLAN-Accesspoint
• Switch mit mehreren Anschlüssen
• Gast-Zugang ohne Zugriff auf das interne Netzwerk

Das ist für normale Nutzer ein Segen – wird beim Umstieg auf OpnSense jedoch zum vielschichtigen Problem, weil jeder der sonst enthaltenen Funktionen separat abgedeckt werden muss.

Was sind die Fallen?

Hier in absteigender Reihenfolge nach ,,gefühlter" Häufigkeit die auftretenden Probleme bzw. Irrtümer:

• Man möchte die OpnSense hinter die Fritzbox stellen, um sein Netzwerk abzusichern. Teils, weil man es sich zuerst (scheinbar) einfach machen will, teils, weil der ISP die Endgerätefreiheit mit Füßen tritt oder zumindest absichtlich erschwert und die Fritzbox ,,netterweise" stellt und eventuell selbst fernadministriert. Letzteres ist oft sogar der Grund,  warum man den Einsatz der OpnSense anstrebt.
  
  Das sieht dann ungefähr so aus:
  
  Internet ----> Fritzbox <---- Zwischen-LAN (192.168.178.0/24) ---> OpnSense <--- echtes LAN (192.168.2.0/24) ----> Endgerät(e)
  
  Bei dieser Betriebsart bleibt die Fritzbox zunächst der Eingangsrouter mit NAT zum ,,Zwischen-LAN". Wenn man die OpnSense als Firewall einsetzen will, muss sie dafür zwingend zwei unterschiedliche Netzwerke trennen, eins ist das ,,Zwischen-LAN" der Fritzbox, das andere das dahinterliegende ,,echte" LAN der OpnSense.
  
  Am Rande: Es hat schon Anwender gegeben, die eine OpnSense ausschließlich mit einem LAN-Anschluss konfiguriert hatten und sich wunderten, warum denn die eingestellten Firewall-Regeln nicht griffen? Falls sich noch jemand darüber wundert, lese nochmals den Abschnitt über Grundlagen.
  
  Diese ,,Lösung" bringt folgende Probleme mit sich:
  
  a. Man kann den WLAN-Accesspoint der Fritzbox nicht richtig nutzen, zumindest nicht in dem Sinn, dass man mit der OpnSense Firewall-Regeln für WLAN-Geräte einstellen kann, da die Pakete die OpnSense ja überhaupt nicht passieren, wenn die WLAN-Geräte im Zwischen-LAN (192.168.178.0/24) angesiedelt sind.
  
  b. Da die Fritzbox den Addressbereich des echten LANs (192.168.2.0/24) hinter der OpnSense nicht kennt, würden Pakete auch nicht dorthin geroutet werden. Entweder muss man in der Fritzbox eine Route auf das echte LAN mit der Zwischen-LAN-IP der OpnSense (z. B. 192.168.178.2) als Gateway eintragen oder man verwendet wiederum NAT auf der OpnSense, um das echte LAN hinter der Zwischen-LAN-IP der OpnSense zu verbergen. Diese Konfiguration nennt sich Double-NAT und sollte, wenn möglich, vermieden werden. Will man nämlich dabei Endgeräte im echten LAN für den Zugriff aus dem Internet freigeben, müssen die entsprechenden DNAT-Eintragungen sowohl in der Fritzbox als auch in der OpnSense gemacht werden. In der Praxis funktioniert das meist nicht bzw. scheitert an diversen NAT-Details wie statischen vs. dynamischen Ports usw.
  
  
• Man glaubt, die Alternative dazu gefunden zu haben, indem man die Fritzbox nicht ,,vor", sondern ,,hinter" die OpnSense positioniert, etwa so:
  
  Internet ----> OpnSense <----LAN (192.168.2.0/24) --+--> Fritzbox
                                                                                               +--> Endgerät(e)
  
  
  In dieser Konfiguration ist die Fritzbox selbst ein reines LAN-Endgerät ohne Routing-Funktionalität. Man benötigt dann kein Doppel-NAT und die Funktionalität der Fritzbox als WLAN-Accesspoint bleibt erhalten – allerdings mit der Einschränkung, dass in diesem Betriebsmodus kein Gast-(W)LAN möglich ist. Für den Betrieb als VoIP-Telefonanlage müssen entsprechende NAT-Regeln auf der OpnSense eingerichtet werden, um die Fritzbox erreichbar zu machen.
  
  Ein weiteres technisches Detail ist, dass hierbei die (DSL-)Modem-Funktionalität der Fritzbox komplett abgeschaltet wird. Man benötigt also ein anderes Modem oder einen ONT:
  
  [tt]Internet ----> Modem/ONT <---- PPPoE oder DHCP ----> OpnSense <----LAN -----> Endgerät
  [/tt]
  
  Als Modem bzw. ONT lassen sich Fritzboxen übrigens nur bedingt nutzen, da erstens neuere Fritzboxen den sogenannten ,,Bridge-Modus" nicht mehr anbieten und zweitens dann sämtliche anderen Funktionen (Router/WLAN-AP/Telefonie) ungenutzt bleiben müssen – wenn überhaupt, wäre die Fritzbox also nur ein (sehr teures) Modem.
  
  Merke: Man kann die Fritzbox eben nicht in Einzelkomponenten zerlegen und Teile (z. B. das DSL-Modem) ,,vor" und andere Teile (z. B. den WLAN-Accesspoint) ,,hinter" die OpnSense positionieren.
  
  
• Man glaubt, man könne sich einen Switch sparen, weil die OpnSense ja mehrere LAN-Anschlüsse hat. Prinzipiell richtig, aber:
  
  Obwohl sich mehrere LAN-Ports ,,brücken" lassen, erfordert dies eine recht aufwendige Konfiguration, zudem werden dann Pakete, die von Anschluss zu Anschluss (also im LAN) übertragen werden müssen, auch über die OpnSense vermittelt und weitergeleitet (nicht aber: gefiltert) und belasten damit deren CPU. Meist steht ohnehin nur eine überschaubare Anzahl an Anschlüssen zur Verfügung.
  
  
• Man meint, man könne mit günstiger OpnSense-Hardware auskommen und man könne z. B. Anschlüsse mit niedriger Geschwindigkeit per Anschluss-Aggregation bündeln (Link Aggregation = LAGG). Das funktioniert nicht so, wie man denkt, denn jede konkrete IP-Verbindung zwischen zwei Partnern kann jeweils nur über eine physische Leitung laufen. Link Aggregation hilft also nur rein statistisch, wenn z. B. ein Server viele Clients bedient und unterschiedliche Clients unterschiedliche Anschlüsse nutzen. Nimmt man aber als Beispiel einen Server, der Daten für einem bestimmten PC bereitstellt, wird die Verbindung nicht schneller als 1 Gbit/s, selbst wenn beide Partner jeweils mit 2 Anschlüssen per LAGG an den Switch oder die OpnSense angeschlossen werden.
  
  Was man jedoch sinnvoll tun kann, ist eine Aufteilung von lokalen (V-)LANs auf verschiedene Anschlüsse, weil dann beispielsweise der Netzverkehr vom LAN ins IoT-Netzwerk nicht zweimal (rein und raus) denselben Anschluss nutzen muss.
  
  Grundsätzlich gilt hier: Es gibt inzwischen günstige OpnSense-Hardware, die z. B. 2.5 Gbit/s pro Anschluss unterstützt, entsprechende Switches sind auch bezahlbar und viele PCs haben schon 2.5 Gbit/s onboard.
  
  P.S.: Umgekehrt gilt: Ein ,,router on a stick", d. h. ein Router, der nur eine einzige Netzwerkkarte hat, funktioniert prinzipiell, setzt aber voraus, dass man einen VLAN-fähigen Switch hat, um (mindestens) WAN und LAN zu trennen. Außerdem passiert der Quer-Verkehr denselben physischen Anschluss dann mehrfach, was die Bandbreite entsprechend vermindert.
  
  
• Man meint, ein uralter X86-Router oder ein Billigst-Mini-PC mit Realtek-Netzwerkkarte(n) würde ausreichen oder man könne einen alten ausrangierten PC dazu hernehmen.
  
  Problem Nummer 1: Alte CPUs haben oft nicht die Befehlssatz-Erweiterungen bzw. überhaupt die CPU-Leistung um erweiterte Funktionen wie VPN oder Traffic-Introspektion mit hohen Netzwerkgeschwindigkeiten erfüllen zu können. Leider ist die FreeBSD-Implementierung für PPPoE auch nicht sehr performant, so dass dies eventuell zusätzlich den Durchsatz hemmt.
  
  Problem Nummer 2: FreeBSD als Grundlage für OpnSense unterstützt Realtek-Netzwerkchips nicht besonders gut, es kommt dabei immer wieder zu Kompatibilitätsproblemen. Teilweise funktionieren die proprietären Realtek-Treiber besser, aber wenn immer möglich, sollte man diese Hardware vermeiden.
  
  Problem Nummer 3: Ein Router läuft normalerweise 24/7. Fall es ein Uralt-PC ist, kann der Stromverbrauch schmerzhaft ins Gewicht fallen.
  
  

Fazit

Zusammengefasst lässt sich also festhalten, dass der Einsatz einer OpnSense im Grunde bewirkt, dass:

• Man einen zusätzlichen Medienkonverter (DSL-Modem, Kabel-Modem, Glasfaser-ONT) benötigt. Eine vorhandene Fritzbox ohne "Bridge-Modus" ist hierfür ungeeignet.
  
• Man meistens einen Switch benötigt, damit man nicht die vorhandenen Ethernet-Ports der OpnSense ,,brücken" muss. Dieser Switch sollte vorzugsweise managebar sein und VLANs beherrschen, damit man eine Netztrennung für Gäste oder IoT-Geräte vornehmen kann. Wenn man das nicht braucht: Wozu dann überhaupt eine OpnSense?
  
• Eine vorhandene Fritzbox sich allenfalls noch sinnvoll als Telefonanlage einsetzen lässt, indem sie als LAN-Client konfiguriert wird.
  
• Eine Fritzbox sich nur dann parallel auch als WLAN-AP einsetzen lässt, wenn man auf eine Netztrennung verzichten kann, weil die Fritzbox das nicht unterstützt. Benötigt man diese Netztrennung aber, beispielsweise, weil viele IoT-Clients nur per WLAN anschließbar sind, sind eigentlich auch separate, VLAN-fähige WLAN-Accesspoints (z. B. Unifi) zwingend.
  

Eine komplette ,,Zielkonfiguration" als Ersatz für eine Fritzbox umfasst also:

• Einen (externen) Medienkonverter (d. h. ONT, DSL-Modem, Kabelmodem o. ä.) im Bridge-Modus
  
• Eine OpnSense mit einigermaßen moderner CPU – insbesondere für breitbandige Anschlüsse oder wenn VPN und/oder speziellere Firewall-Funktionen benötigt werden, die Packet-Introspektion erfordern. Ports mit mehr als Gigabit-Geschwindigkeit sind hilfreich.
  
• Einen nachgelagerten Switch, um die CPU der OpnSense nicht durch Bridging zu belasten, vorzugsweise mit VLANs und managebar. Auch hier helfen schnellere Ports, z. B. SFP+ oder 2.5 Gbit/s. Port Security über IEEE 802.1X ist auch interessant, da die OpnSense einen FreeRADIUS-Server enthält.
  
• Ein oder mehrere WLAN-Accesspoints, die mehrere VLANs auf WLANs abbilden können.
  
• Eine VoIP-Telefonanlage (hierfür kann eine Fritzbox genutzt werden).
  

Neben den Kosten für die notwendigen Komponenten muss man antizipieren, dass man sich mit recht komplexer Netzwerktechnologie auseinandersetzen muss, um nicht das Gegenteil von dem zu erreichen, was man durch den Einsatz von OpnSense eigentlich erzielen möchte, nämlich: mehr Sicherheit.

Jeder sollte sich daher die Frage stellen: Brauche ich das wirklich? Bzw.: Kann ich das wirklich? Oder bleibe ich lieber bei einer vorkonfektionierten, integrierten Lösung wie einer Fritzbox?

Abschließend: Ich habe hier bewusst IPv6 ausgespart, was insofern eine vereinfachende Annahme ist, da inzwischen einige Provider nur noch Dual-Stack-Lite (DS-Lite) mit Carrier-Grade NAT (CG-NAT) anbieten (beispielsweise Deutsche Glasfaser). Bei dieser Anschlussvariante bekommt schon der Kundenrouter aufgrund der IP-Verknappung erst gar keine öffentlich routebare IPv4 mehr, sondern eine aus einem bestimmten IP-Bereich (100.64.0.0/10, also 100.64.0.0- 100.127.255.255), genauso wie beim LAN mit RFC1918. Effektiv betreibt also der ISP einen vorgeschalteten NAT-Router, bei dem insoweit immer ,,Doppel-NAT" vorliegt.

Dies hat den Haken, dass der Kunde natürlich keine Port-Freigaben auf dem vorgeschalteten Router des ISP vornehmen kann und somit keine Dienste in seinem Netz von außen verfügbar machen kann, außer, er nutzt dazu IPv6 oder komplizierte Tunnellösungen, bei denen zuerst ein Tunnel von innen nach außen geöffnet wird und eingehende Verbindungen dann auf dem Tunnelende entgegengenommen werden (beispielsweise Cloudflare, eigener VPS-Server im Internet oder andere Tunnelprovider).

Aber wie eingangs angedeutet: IPv6 und Tunneling sind eigene Themen, die in separaten Beiträgen beleuchtet werden sollten. Das Gleiche gilt für Dynamisches DNS, TLS-Terminierung mit HAproxy, Caddy oder Nginx usw., wobei ich dazu hier schon etwas geschrieben habe – das bewegt sich allerdings auf einem deutlich anderen Anspruchsniveau als dieser Artikel.

In der Tutorial-Sektion des OpnSense-Forums gibt es neben diesen Themen zusätzlich noch viel Stoff für diverse Spezialthemen wie Dynamisches DNS, Traffic-Shaping, VoIP-Konfiguration u.v.a.m.

I have noticed that under Firewall: Settings: Normalization, the text blocks disappear when hovered in some themes (namely Tukan and Dracula), but not in others. It seems the text color is changed to something that does not contrast above the background.

Since Tukan and Dracula do not come from the same developers and since similar entries under Firewall: Settings: Advanced do no show the same issues, I suspect that the specific structure of the page elements is different. Probably there are tags set that once existed and now are outdated, but are still contained in the styles?

I have looked at the CSS, but I have no experience in that.

I love the Tukan theme for its readability and this glitch has always bothered me.

Having followed some potential ZFS corruption issues that have been found with the ongoing OpenZFS 2.2 release but really were lurking since 2.0 already, I noticed that Proxmox has just issued new kernels and ZFS utilities with fixed Open ZFS 2.2.2.

I now saw a video by Lawrence Systems about pfSense having fixed those issues as well.

@Franco: Will those FreeBSD fixes make it into 23.7.10 or a hotfix version?

P.S.: There are also some security-relevant fixes in the latest pfSense release which probably triggered this comment by Tom...

[XX]

So now I am fed up with all these discussions about IPv6 and DynDNS shortcomings... beware, this is long!  :P


First, let's talk about IPv4 and how you did it back then.

In the old days (tm) of IPv4, you wanted to expose services on your firewall or your internal machines
(you will later see why this distinction is relevant) to the internet, but alas, your ISP did not give you a static IPv4.

Thus, you set up one or more dynamic DNS name(s) which your router updated via a DynDNS (for "dynamic DNS") provider and open up ports for your services via (D)NAT and be done with it.
Well, you probably had the problem that each service had to use a different port, but you could port-translate
to the destination machine. Thus, every machine could have another port even if the internal service always used the same one.

For IPv4, you also used NAT reflection, such that your clients can use the same external IP (and hence DNS name) from within your own network(s).


But now, it's 2023 and we have IPv6 (and sometimes even only IPv6).


Alas, despite of the abundance of IPv6 addresses, most ISPs only offer dynamic IPv6. So we probably
need DynDNS for IPv6 as well. But many DynDNS providers still do not support IPv6.
Even if they do, we still face problems:

• Potentially, we want to update both the IPv4 and the IPv6 for a given service. Some DynDNS providers cannot handle both for the same DNS entry and even if they can, you cannot split both updates into two separate requests, because "the last one wins".
  
• 
  On the other hand, your firewall probably has no means to specifiy both IPv4 and IPv6 addresses in one request.
  
• Protentially, your firewall cannot reliably determine the correct IPv6 anyway, because:
  
  
  
  • 
    It cannot distinguish between routable and non-routable IPv6 addresses.
    
  • 
    Even if it can, or if you just call an external service or use the WAN IPv6 "implicitly", the routeable IPv6 of the WAN interface is potentially useless, because it is the IA_NA of the firewall itself and not the IA_PD prefix you would like because you really wanted to update the IPv6 for a local client. With IPv6, that distinction becomes vital, because NATv6 is essentially non-existent (and it should be).
    

Special case: You only have a routeable IPv6, because your ISP only offers CG-NAT for IPv4. Well, forget about the idea of exposing IPv4 services by pure firewall means anyway. You need to rely on a service to translate IPv4 to IPv6 for you or you do it yourself via a cloud VPS.

For what follows, I assume you have both inbound IPv4 and IPv6.

You will have to find ways around these DynDNS problems. We will get to this point later on.

Before that, let us discuss types of services. Roughly, we have these:

• HTTP or HTTPS, i.e. web-frontends of any kind or some APIs, like the admin UI of a switch, your local installation
  of Unifi controller, a Plex media server or a Proxmox host.
  
• TCP-based services like SSH

For type 1, I personally prefer to use HAproxy (There is a great tutorial on this by TheHellSite for OpnSense,
see https://forum.opnsense.org/index.php?topic=23339.0).
Because of termination at the firewall, the main advantages to this are:

• You can easily use both IPv4 and IPv6 for the same service.
  
• You can reach any internal service, regardless if it can handle IPv6 itself.
  
• You can translate ports, even with IPv6.
  
• You need only one IPv4 and one IPv6, so your DynDNS has to handle all the DNS names, but only two IPs.
  
• This is the most important: You can route requests based on DNS names, not via ports or IPs!
  
• You can centrally manage certificates and encryption for all of your internal sites.
  

Notes:

• c. can be a security plus, because IPv4 port scanners will find it harder to identify services on non-standard ports.
  
• e. is more secure as well, because your end services become exposed only to someone also knowing the name of the service (side-note: use a fallback with a fake certificate in order not to expose all of your service names with the real certificate, but be aware of certificate transparency!).
  
• f. is obviously more secure because it allows TLS even for services that do not have encryption built-in.
  
• Preferably, whenever it is possible, do not expose services at all. For example, I just switched from exposing a centralized Unifi network controller to using it over a Wireguard VPN.
  

For type 2, you can (and probably should) still use HAproxy, but (mostly) you cannot have e. and f.
You can also expose type 2 services directly, but there are some catches:

• Obviously, your internal service must be able to handle IPv6 in the first place (advantage b.). If it cannot, but you still do not want to use HAproxy, limit the DynDNS for this client to IPv4 (i.e. do not use a DNS entry that resolves to both IPv4 and IPv6).
  
• If you want a service to work over IPv4 and IPv6, it must have the same port over both. Since port translation over IPv6 is difficult at best, you expose the "real" port of the local machine, so you cannot have advantage c. from above.
  
• You also lose advantage d. as you must do DynDNS for your local IPv6 that is derived from the IA_PD prefix, not the firewall's IN_NA. This is usually harder.
  

Now how do we do that type of "dual" DynDNS?

As said earlier, with IPv6, every device gets their own IP, so now the distinction between your firewall and
your other devices becomes relevant (with IPv4, all used the WAN IP).
So, first thing to note: if have have multiple devices, you will need multiple DNS names.

As with IPv4, it is probably your router who does the dynamic DNS updates for you. With IPv6, it can usually do these updates only for itself. There are at least two reasons for this:

• 
  Most ISPs give you more than one routable IPv6 (globally unique adress or GUA). One is the "network address" (IA_NA for "network address"), which is assigned to the WAN interface of your router and the other one is a prefix (IA_PD for "prefix delegation"), from which you can delegate IPv6 addresses to your local devices.
  This prefix determines between 48 to 64 most-significant bits of the final IPv6 (most commonly, 56 bits are used).
  
  The rest of the address can be up to 16 bits that are interface-specific (that is, you can assign each internal VLAN a different interface prefix) plus 64 bits of EUI-64, which is derived from the MAC of the client device.
  
  
• 
  Now, where is the problem? The thing to note here is: IA_NA and IA_PD IPv6 ranges are different!
  In order for your firewall to update the dynamic DNS entry for any other local client, it would have to generate  exactly this client's IPv6. But with typical DynDNS update tools like ddclient, all it can usually do is:
  
  
  
  
  • call an external URL to find its own IPv6 - this would normally be the WAN IPv6, and this is not what we want.
    
  • find its IPv6 locally - which would be the same in the best case, but wait: since any device and even interface can have multiple IPv6 addresses, which one should it even choose?
    
  • not use a specific IPv6 at all but rely on the DynDNS provider to detect which IPv6 the request originates from - this is essentially equivalent to variant a, unless you use some trickery.
    
  
  

OpnSense has an (partial) answer to this because it can find either the IPv4 or the IPv6 of a specific interface (via "Check ip methods" "Interface [IPv4]" and "Interface [IPv6]") and provide the result via the macro __MYIP__. This is a "partial solution" because for DynDNS providers that can only accept both IP types in the same request, this will not work without a small trick.


The remedy to these problems is:

• 
  You can force your firewall to use an IPv6 from the IA_PD range by specifiying "Request only an IPv6 prefix" on the WAN interface. By doing that, your firewall WAN interface does not get a routeable IPv6 (i.e. no IA_NA).
  However, by using "Track Interface" as IPv6 configuration type for your local network interfaces, at least one
  of them will get an IPv6 GUA from the prefix delegation range (IA_PD).
  One of those local interface IPv6s will then be chosen for your firewall's outbound connections. The trick is that the delegated prefix of this IPv6 will be the same for all of your routeable IPv6s. This works even if the client is on another (V)LAN that the outbound (V)LAN interface of your OpnSense.
  
  
• 
  You need a DynDNS provider who can detect the originating IPv6 from the API request plus he is able to cut the most significant N bits from it and have the rest be configured manually for a specific DynDNS name. Preferably, he offers an update API endpoint that is IPv6 only to make sure that the update is done via IPv6 even when "prefer IPv4" is configured.
  
  

So, as an example, let us assume there are these devices:

• Your OpnSense firewall with a MAC of AA:BB:CC:DD:EE:FF, giving an EUI-64 of ::a8bb:ccff:fedd:eeff
  
• A LAN server with a LAN MAC of 11:11:11:11:11:11, with an EUI-64 of ::1311:11ff:fe11:1111
  
• An IoT device with a VLAN1 MAC of 22:22:22:22:22:22, with an EUI-64 of ::2022:22ff:fe22:2222
  

Let us further assume that your ISP fives you a /56 IPv6 dynamic public delegation prefix (say dead:beef:feed:fa00::0/56) and you used the 8 network bits as "00" for your LAN and "ce" for your IoT VLAN1 (Be aware that OpnSense wants the network bits in decimal, so this would be 0 and 206, respectively).

In that case, you could configure three DynDNS names like so:

opnsense.dyndns-x.com	dead:beef:feed:faXX:a8bb:ccff:fedd:eeff
lanserver.dyndns-x.com	dead:beef:feed:fa00:1311:11ff:fe11:1111
iotdevice.dyndns-x.com	dead:beef:feed:face:2022:22ff:fe22:2222

Note that you will have to find which interface is being chosen for outbound connections, thus the XX in the OpnSense IPv6.

Whenever your OpnSense gets another dynamic prefix (say cafe:babe:bedd:ab00::0/56), only the first 56 bits on all of these DynDNS entries get updated, because the DynDNS provider uses the new requesting IPv6 (cafe:babe:bedd:abXX:a8bb:ccff:fedd:eeff).

Of course, you have to set up firewall rules to access these devices. But for that, OpnSense has a firewall alias type of "Dynamic IPv6 Host" where you can specifiy the EUI-64 like given above plus the interface to which it is connected. The "Dynamic IPv6 Host" alias will get updated when the IPv6 prefix for the interface changes.

BTW: This assumes you use SLAAC for your internal clients and that they can handle IPv6 at all. For DHCPv6, you would have to use static reservations and use the lower 64 bits for the device instead of the EUI-64. If your clients cannot handle IPv6, many times you could still use HAproxy (there is a tutorial how to set this up) as an application-level gateway between IPv6 and IPv4.


Apart from the shortfalls I gave above, there are several weaknesses in current implementations:

• 
  ddclient tries to accumulate DynDNS entries (hostname & IP) for the same DynDNS provider into one API request. This is a problem if you want to have both an IPv4 and an IPv6 for the same name in case you create two update rules with different parameters. You can sometimes circumvent that by using different API endpoint names, if available from your DynDNS provider. In OpnSense, there is now a "native" backend which is much better suited than ddclient.
  
• 
  Many DynDNS providers handle both IPv4 and IPv6, but the last updates overwrites whatever was there before, so you cannot split updates between IPv4 and IPv6 (e.g. https://www.ddnss.de).
  
• 
  At this time, OpnSense cannot determine both IPv4 and IPv6 interface adresses to wrap into one API call (like "...&myip=__MYIPV4__&myip2=__MYIPV6__").
  
• 
  Also, OpnSense cannot determine IPv6 addresses "in lieu" of a client, like use their "Dynamic IPv6 Host" firewall alias as a variable (this could not really work like that because the content of that variable can be an array).
  It would be very helpful to directly update DynDNS entries regardless of the API connections's IP protocol and not having to rely on the DynDNS provider to strip something off the IPv6.
  Something like this would even work if the IA_NA of OpnSense was used for the API request.
  

@AdSchellevis: Fixing weaknesses 3 and 4 would be pure luxury!


Step-by-step instructions for the current, somewhat broken implementation:

• 
  Register an account at https://dynv6.com
  
  
• 
  Create a zone of your choice, e.g. "testit99.dns.army" and leave the IPv4 and IPv6 fields empty for now
  
  
• 
  Configure your OpnSense to use DHCPv6 on the WAN interface. Specify how many bits your prefix has,  most often it will be 56. Because of another trick introduced later, you can even leave IA_NA on the WAN    interface for outbound connections for this specific DynDNS provider in place.
  
  
• 
  Configure your LAN interface to "Track Interface" WAN for IPv6 and assign an "IPv6 Prefix ID".  Check that it gets an IPv6 GUA from the correct range. If you do this for multiple (V)LANs, find out which of them is being used for outgoing connections by entering "curl v6.ident.me" on the OpnSense CLI. You will see the IPv6 Prefix ID number directly preceeding the EUI-64 bits, so you know the corresponding  outbound (V)LAN interface.
  Take a note of the EUI-64 and preceeding IPv6 Prefix ID number that your OpnSense uses.
  
  
• 
  Optional: If you have a local client / service that you want to address as well, check that Router Advertisements (aka SLAAC)  is configured under "Services->Router Advertisements". Use "Unmanaged" mode and a "Minimum" / "Maximum Interval" of 200 / 600. Check that your client gets an IPv6 GUA as well and also, if it has IPv6 connectivity - if not, check your firewall rules. Take a note of its EUI-64 and which (V)LAN interface it is on.
  
  
• 
  Looking at the DynV6 zone's "Current status", you still have nothing configured for the zone itself.
  
  
• 
  Lookup the Dynv6 HTTP Token in your Dynv6 account settings (under "keys").
  
  
• 
  Set the following under your OpnSense Services->Dynamic DNS->Settings "General settings" tab (enable advanced mode"):
  
      Enable: checked
      Verbose: checked
      Allow IPv6: checked
      Interval: 300 (or as you like)
      Backend: native
  
  Apply these settings.
  
  
• 
  Create a new account under your OpnSense Services->Dynamic DNS->Settings:
  
      Enabled: checked
      Description: whatever you like
      Service: custom
      Protocol: Custom GET
      Server: https://ipv4.dynv6.com/api/update?zone=<YOUR_ZONE_HERE>&token=<YOUR_TOKEN_HERE>&ipv6=__MYIP__/56&ipv4=auto
      Username: x (because technically, you need it)
      Password: x
      Wildcard: unchecked
      Hostname(s): <YOUR_ZONE_HERE>
      Check ip method: Interface [IPv6]
      Interface to monitor: <YOUR_OPNSENSE_OUTBOUND_(V)LAN_INTERFACE>
      Check ip timeout: 10
      Force SSL: checked
  
  
  (Note: Yes, we use the IPv4-only API endpoint, which sound weird. That is because we want to have both our
  IPv4 and our IPv6 updated in one go. We can only detect one of these addresses locally and use it via the
  macro "__MYIP__" in the request URI, the other one will be autodetected by the API.
  Also, we want to specify the prefix length (/56) together with the IPv6. Because this cannot be done automatically, it determines which is which. So, we must use IPv4 auto-detection via "ipv4=auto" and for this to work, we have to connect via IPv4 to the API. By using this trick, we could even differentiate between IA_NA and IA_PD, just  because we are able to send the corresponding prefix explicitely, but we would need different zones for this.)
  
  After that, do not forget to apply the settings. Check the Dynv6 administration interface and verify that your zone now has both an IPv4 and an IPv6 prefix. The IPv6 prefix should have all zeros for the "IPv6 Prefix ID" bits. This enables the use of any VLAN for the final DNS names.
  
  
• 
  Go to the Dynv6 administration console and choose your zone. Switch to the "Records" tab and create a new AAAA record. Leave the name empty and paste the EUI-64 of your chosen outbound OpnSense interface, preceeded by the "IPv6 Prefix ID" bits into the data field and save the entry. It should look something like "::00:a8bb:ccff:fedd:eeff".
  This record will result in the DNS name "testit99.dns.army" from our example being resolvable.
  Note that when you look at the DNS records, they will automatically expand to show the full resulting IPv6 GUA by prepending the zone's IPv6 prefix. You can switch between both views by clicking on the small icon ("expand") above the data column.
  Make sure not to specify the full IPv6, because then it will not be changed by a zone update later on!
  
  
• 
  Optional: If you want your local client(s) to be accessible, create another AAAA record, but this time, assign a name (e.g. "lanserver" from our example, resulting in "lanserver.testit99.dns.army") and  the client's EUI-64 plus prepended "IPv6 Prefix ID" bits for its (V)LAN. If you like, you can also create an additional entry for your OpnSense itself as a subdomain (e.g. "opnsense", resulting in "opnsense.testit99.dns.army").
  You may use multiple zones if you do not like subdomains and prefer "opnsense.dns.army", but you need an    DynDNS update account for each zone. In addition to this, Dynv6 allows you to handle your own, custom DNS domains.
  
  

Your OpnSense now has both IPv4 and IPv6 DNS addressability as "testit99.dns.army". Your other LAN client(s) can be addressed via the subdomain name(s) you have created (e.g. "name.testit99.dns.army". You can install IPv6 firewall rules to allow traffic as needed, but be careful.

[Note that these settings and installation of packages will not be part of a configuration backup, so after a hardware migration, you have to repeat most of it.]

As many firewall appliances suffer from neglect by they manufacturers, they often lack current microcode updates.

For example, Intel Alder Lake based CPUs (like N100, N200, N300 and I1215U have a bug that cause instabilities.

As of OpnSense 24.7.2, microcode updates can be enabled for AMD and Intel CPUs by installing the corresponding plugin os-cpu-microcode-amd or os-cpu-microcode-intel (not both!). The update will be done automatically upon next reboot.

The old post is left here for reference:



Since 23.7.9, there should be microcode updates available, which, when enabled, should cure those problems.

Though OpnSense does not have those enabled per default (e.g. it obviously is not applicable for VM installations), the neccessary tools are provided. Here is how:

1. Install the package cpu-microcode - this is a newer variant of the old devcpu-data package. From a shell, call:

Code Select Expand

echo y | pkg install cpu-microcode


Alongside with this package, the RC scripts and firmwares for both Intel and AMD CPUs will be installed. If you had the devcpu-data package installed before, it will get replaced.

2. Use the web UI to create these two tuneables in /boot/loader.conf:

Code Select Expand

cpu_microcode_load="YES"
cpu_microcode_name="/boot/firmware/intel-ucode.bin"


Please double-check that the entires exist. Note that this is only for Intel CPUs, because this addresses only the early boot stage, which is not available for AMD CPUs. This can be fixed in the next step.

3. Add an entry to /etc/rc.conf:

Code Select Expand

echo 'microcode_update_enable="YES"' >> /etc/rc.conf


This will enable a RC script which handles both Intel and AMD CPUs during boot.

4. (Optional and only available from 23.7.6 on)

Code Select Expand


pkg install x86info
rehash
x86info -a | fgrep -i microcode


The last command will show you the current microcode version.

5. Reboot or call the CPU firmware update script once with:

Code Select Expand

service microcode_update start


You will see an error if you skipped step 3.

6. (Optional and only available from 23.7.6 on) Call:

Code Select Expand

kldload -q cpuctl; x86info -a | fgrep -i microcode

again and you will probably see a different version than in step 4.

Note that these settings and installation of packages will not be part of a configuration backup, so after a hardware migration, you have to repeat most of it.

These instructions are provided as-is, so no warranties whatsoever.

I know this was a thing with 22.7 before (there is an archived thread about this here):

On one of my OpnSense boxes, radvd stops advertising the prefix after a while. That is to say, the RAs lack the prefix info, it is not radvd stops working completely. Restarting radvd immediately helps, after the restart, the prefix is announced again. In the old thread, a cron job to restart radvd every hour was suggested, but I thought that this problem was fixed long ago?

There are two other boxes which do not have this behaviour (with the same ISP!) and I do not see any configuration difference. Also, to my knowledge, the prefix also does not even change...

Until the latest OpnSense release, I used dnsmasq instead of unbound because of two reasons:

1. It is much faster.
2. It handles local domains better IMHO, because you can define a default domain like "ttt" and have both "host" and "host.ttt" resolve to the same name. This helps a lot for devices that cannot pick up the default domain via DHCP correctly.

With 23.1.5_4, dnsmasq seems to start too early (or whatever), at least after a reboot, it does not forward requests to upstream servers any more. Thus I started to try unbound.

Now for #2 in my list: I found that I need to define a domain for each host override, such that I do not even have an option to define both "host" and "host.ttt" - strangely enough, I can skip the domain for aliases, but then they will not work either...

So, is there a way or a best practice to map queries with hostnames without a domain to a "default" domain for unbound?

I have a routing problem after update from 23.1.4_1 -> 23.1.5_4. First thing was that I had an exception (which I could not read before it was replaced by the reboot notice). The reboot did not occur, I had to manually restart the box.

After coming up again, router advertisements were being handed out, I could see the addresses, but no IPv6 routes. After a manual restart of radvd, the routes re-appeared.

Could have something to do with resequencing of events during interface startup. In my case, the WAN interface does not have a routeable IPv6, since I get only a prefix range via DHCPv6.