Messages

In a strict sense, you would need a separate VLAN for any specific device. Most people only separate an "IoT" VLAN.
There are smart switches and networking equipment, where you can even keep devices on layer 2 separate. Unifi can do that on their WiFi networks ("Client Device Isolation", but it inhibits Airplay, Chromecast and others).

If you do not want network access for a single network device, you can block that specifically on OpnSense, even without having VLANs. Note, however, that this would strip a smart TV from most of its prominent features, like streaming.

As long as your device has internet access, it can spy on you even when it cannot reach its neighbors one way or another. As the video shows, the webcams and microphones are accessible from outside.

Please see reply #7 and provide the information requested there. Otherwise, there is little point in further speculation.

This appears to be the already identified caddy-l4 routing bug described in PR #455, rather than merely another insufficient prefetch-buffer size.

If a TLS matcher needs more data because the ClientHello arrived in multiple TCP segments, a subsequent HTTP route can reach a definitive non-match and inadvertently suppress the pending TLS match. Post-quantum ClientHellos make this much more reproducible because they commonly span multiple TCP segments.

This fits your observations exactly: curl and Firefox without Kyber send a smaller ClientHello, whereas current Firefox and Chromium trigger the bug.

As a workaround, try placing all Layer 4 HTTP routes before all TLS/SNI routes. PR #455 specifically identifies that route ordering as a workaround and states that caddy-l4 0.1.2 is affected. IDK if that is possible via the GUI, though... otherwise, Deciso would have to either include the PR or change configuration ordering.

[did]

As for Q1: I do not know. But that does not really matter. Even if DNS is broken immediately after OPNsense boots, the cron job should eventually heal that condition. As I already stated, this should not be your main concern; you can shorten the recovery time simply by running the cron job more frequently.

What your experiments did establish quite convincingly is my earlier point about FQDN resolution causing problems with S2S connections.

Alas, you did not provide any of the diagnostic information I asked for that could actually shed light on what goes wrong when a C2S client cannot resolve DNS.

Thanks, that rules out both of my DNS-related guesses.

In particular, the fact that restarting Unbound does not help, while restarting the inbound WireGuard instance does, strongly suggests that the problem is not DNS itself but the runtime state of the WireGuard interface after boot.

Also, I would not necessarily assume that your two observations have the same cause:

1. "Configuring WireGuard" taking a long time during boot may well be caused by S2S peers whose endpoints have to be resolved. That is to be expected, because the DNS of the endpoints is not yet resolvable and only gets corrected on first run of the cron job.

2. The C2S clients being unable to use the firewall itself as DNS afterwards is probably a separate problem with the C2S WireGuard interface not being initialized correctly.

So I would compare the actual WireGuard state before and after restarting the inbound instance.

While the problem exists, please show:

Code Select Expand

ifconfig wgX
wg show wgX
netstat -rn -f inet

Then restart only the affected inbound WireGuard instance and show the same three commands again.

In particular, check whether the configured tunnel IP address of the firewall is actually present on `wgX` in the broken state. If that address is missing or wrong, it would explain the symptoms very nicely: routed traffic through the tunnel may still work, while DNS to the firewall's own WG address cannot.

Another useful test while it is broken would be:

Code Select Expand

tcpdump -ni wgX port 53
and then try a DNS lookup from the client.

If the DNS packets arrive on `wgX`, we know that the tunnel itself transports them and can concentrate on local delivery/interface configuration instead of DNS.

At this point I would first look at the before/after difference rather than try another workaround.

[do not bind any services to specific interfaces]

Well, now I got you. I suspect this might be one of two cases:

1. You have bound your DNSmasq or Unbound to specific interfaces. In that case, when Unbound starts up, the WG interface does not yet exist and cannot be bound. You can test if this is the case by "sockstat -4 -6 -l | grep :53" and looking at which interfaces WG listens on when your error condition exists. If that is the case, you can avoid it by using "ALL" (nor no) interfaces to have the DNS service bind to 0.0.0.0. This is explained often throughout the forum: do not bind any services to specific interfaces. Besides, this is here: https://forum.opnsense.org/index.php?topic=42985.0, point 32.

2. If you are using Unbound, there is another known possibility: the automatically generated ACL for the WireGuard network may be missing after boot because the WG interface does not yet exist when the Unbound configuration is generated.

This has been reported before:
https://github.com/opnsense/core/issues/4142

Check /var/unbound/access_lists.conf while the problem exists and see whether your WG subnet is present. If it only appears after restarting Unbound, that is the cause.

Unlike the interface binding problem, the simple workaround here is to add an explicit ACL for the WireGuard client network under Services -> Unbound DNS -> Access Lists instead of relying on the automatically generated interface ACL.

This second case should only matter if your Unbound ACL default action is set to Deny or Refuse. With the default action set to Allow, the automatically generated interface ACL should be irrelevant (see: https://forum.opnsense.org/index.php?topic=42985.0, point 0). The old ticket was created when there where specific interface rules created (namely, in 2020).

For S2S connections, the cron job helps. For C2S, there should be nothing keeping your Wireguard server from starting, because the peer does not have to have an endpoint, so there is no DNS to be resolved that could keep the instance from coming up?

The "Renew DNS for Wireguard on stale connections" cron job had a problem with that specific DNS outage on startup, but it has been fixed recently, now taking care of it correctly.

This fix is in the CE version, but can manually be applied to the BE as well.

Outbound NAT is legacy, its rules should be migrated to SNAT and it will eventually go away completely (use the search):

https://forum.opnsense.org/index.php?topic=52261.0

The Outbound NAT page is essentially only there because the setting for Automatic, Hybrid, Manual and Disable NAT was still there, which is admittedly (and obviously) confusing. The last changes duplicated the mode setting to the SNAT page as well, so you can ignore Outbound NAT and have no rules left there (i.e.: migrate them to SNAT).

That is the same. With one of the last updates, "Outbound NAT" has been renamed to the more common "Source NAT". If you still see Outbound NAT in your firewall menu, you either use BE or are way behind with your updates.

You are using the wrong documentation. Please read the first paragraph of the initial post in this thread and follow the linked swanctl Roadwarrior documentation. The guide you linked describes the old/legacy IPsec configuration, which is why those menus do not exist in your installation.

So you now expect OpnSense CE (which is free) to invest in interoperability for Surfshark (which you presumably pay for)?

You are assuming that "hardware" must be faster than "software". That is not generally true.

If OPNsense itself runs as a VM on that host, the traffic has to pass through a VM anyway, so adding physical NICs and an external switch only adds another network path.

Only if OPNsense is a separate physical firewall does the traffic need to leave the host in the first place.

In either case, I would start with virtio + a Linux bridge and only change that if measurements show an actual problem.

I can only tell you how I am doing that kind of thing with PVE (which also uses KVM, so this should be similar enough):

I have one bridge vmbr0 connected to a physical ethernet adapter that connects to the "real" network and it also has the IP of the PVE host.
A second bridge interface vmbr1, is VLAN-aware and is purely virtual, in that it has no physical NICs attached to it at all.
For each VM, I create a separate VLAN X with a an associated subnet, say 10.0.X.0/24. The VM X itself has 10.0.X.2/24 as its IP address and 10.0.x.1 as gateway and DNS server.

I use one OpnSense VM that has its virtio "WAN" attached to vmbr0 and thus gets an IP on my LAN. It also has a virtio "VLANBRIDGE" interface, off of which I define "VLANX" interfaces accordingly with VLAN X and IP address 10.0.x.1/24.

Each of the VLANXs is allowed to access the internet, but not RFC1918, so it cannot access either LAN devices or other VMs.

On the other hand, I can create rules that allow access from the "WAN" (aka LAN) to each VM (or the whole 10.0.0.0/16 range).

You can use such a setup in a datacenter or at home, like described here: https://forum.opnsense.org/index.php?topic=44159.0

[you]

@talespin: Or you should update more often to keep up with patches for known vulnerabilities, like most people do. That way, you will only have to do small. incremental steps in case something changes w/r to functionality. This is the reason why you cannot expect an automatic mechanism that spans more than one version in one big leap.

There is also the possibility to backup your configuration, install fresh and import your configuration. However, that may fail for the same reason.