Messages

[Outgoing Network Interface]

Good. So instead of going through hoops for automatic NAT set WireGuard straight, assign interface for gateway, add NAT outbound and done?

It shouldn't be more work?


Cheers,
Franco

I can confirm that removing the static IPv4 config from tunnel interfaces "just works". VIPs also aren't needed.

However, with tunnel interfaces not being statically configured, I cannot use them as Outgoing Network Interface for Unbound DNS anymore. Using VIPs didn't do the trick. The Web GUI docs for valid outgoing interfaces state:

Utilize different network interfaces that Unbound will use to send queries to authoritative servers and receive their replies. By default all interfaces are used. Note that setting explicit outgoing interfaces only works when they are statically configured.

Does that mean this option is incompatible with tunnel interfaces going forward? Is that an issue I should raise on GitHub?

[wireguard-kmod]

Just a quick semi off-topic question here.

Before update to 23.1
- WireGuard Go plugin installed (os-wireguard)
- WireGuard kmod package installed manually from cli

Are we supposed to uninstall the kmod package before applying the update to 23.1?
Or will the update replace/remove the WireGuard Go plugin and update the kmod package?

Correct me if I'm wrong, but doesn't the OPNsense plugin system "track" / ignore manually installed packages? Maybe that was TrueNAS CORE?

I'm already on 23.1, but previously installed the wireguard-kmod package via pkg install.

Before following my tests blindly, please create a backup!

I just tested uninstalling the plugin. When doing so, the following packages were uninstalled:

• bash
• os-wireguard
• wireguard-tools

I then had to manually cleanup wireguard-kmod via pkg remove. Re-installing the os-wireguard plugin now adds the following packages:

• bash
• os-wireguard
• wireguard-tools
• -> wireguard-kmod <-

Uninstalling the os-wireguard plugin again now properly cleans up the wireguard-kmod package.

My conclusion is that it's cleaner to uninstall the os-wireguard plugin and wireguard-kmod package first, and then re-install the plugin.

My WireGuard config was untouched by this (please create a backup).

A PR has already been merged to fix the issue. So all we have to do is wait for a new haproxy release.

I upgraded to 22.7. HAProxy spits out some deprecation warnings, but my config seems to be working fine.

edit: a PR with a fix has already been merged, so we just have to wait for a new haproxy plugin release

Upgrading to 22.7 went smoothly 8)

I follwed the popular HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating tutorial.

Some PHP errors are reported by OPNsense:

Code Select Expand


[28-Jul-2022 16:47:15 Etc/UTC] PHP Deprecated:  Required parameter $expression follows optional parameter $description in /usr/local/opnsense/mvc/app/models/OPNsense/HAProxy/HAProxy.php on line 147
[28-Jul-2022 16:47:15 Etc/UTC] PHP Deprecated:  Required parameter $testType follows optional parameter $description in /usr/local/opnsense/mvc/app/models/OPNsense/HAProxy/HAProxy.php on line 176
[28-Jul-2022 16:47:15 Etc/UTC] PHP Deprecated:  Required parameter $type follows optional parameter $description in /usr/local/opnsense/mvc/app/models/OPNsense/HAProxy/HAProxy.php on line 176
[28-Jul-2022 16:47:15 Etc/UTC] PHP Deprecated:  Required parameter $address follows optional parameter $description in /usr/local/opnsense/mvc/app/models/OPNsense/HAProxy/HAProxy.php on line 204
[28-Jul-2022 16:47:15 Etc/UTC] PHP Deprecated:  Required parameter $port follows optional parameter $description in /usr/local/opnsense/mvc/app/models/OPNsense/HAProxy/HAProxy.php on line 204
[28-Jul-2022 16:47:15 Etc/UTC] PHP Deprecated:  Required parameter $mode follows optional parameter $description in /usr/local/opnsense/mvc/app/models/OPNsense/HAProxy/HAProxy.php on line 204
[28-Jul-2022 16:47:15 Etc/UTC] PHP Deprecated:  Required parameter $name follows optional parameter $enabled in /usr/local/opnsense/mvc/app/models/OPNsense/HAProxy/HAProxy.php on line 230
[28-Jul-2022 16:47:15 Etc/UTC] PHP Deprecated:  Required parameter $mode follows optional parameter $enabled in /usr/local/opnsense/mvc/app/models/OPNsense/HAProxy/HAProxy.php on line 230
[28-Jul-2022 16:47:15 Etc/UTC] PHP Deprecated:  Required parameter $algorithm follows optional parameter $enabled in /usr/local/opnsense/mvc/app/models/OPNsense/HAProxy/HAProxy.php on line 230


Those probably are the same errors you're seeing. But it still works!

[Traffic is SSL (TCP request content inspection)]

I've been following this excellent guide to a tee, but I get the following warning:

[WARNING] (51339) : Proxy '1_HTTP_frontend': L6 sample fetches ignored on HTTP proxies (declared at /usr/local/etc/haproxy.conf.staging:69).
Warnings were found.
Configuration file is valid

This is the affected part of the HAProxy config (the last line is 69):

Code Select Expand


# Frontend: 1_HTTP_frontend (Listening on 192.168.64.1:80)
frontend 1_HTTP_frontend
    bind 192.168.64.1:80 name 192.168.64.1:80 accept-proxy
    mode http
    option http-keep-alive
    option forwardfor
    # tuning options
    timeout client 30s

    # logging options
    # ACL: NoSSL_condition
    acl acl_629f48c6073c95.86527303 req.ssl_ver gt 0

    # ACTION: HTTPtoHTTPS_rule
    http-request redirect scheme https code 301 if !acl_629f48c6073c95.86527303


For reference, other people also encountered this issue:

• Reply #191 and after in this thread
• HAProxy warnung nach update auf 22.1 - L6 sample fetches ignored on HTTP proxies
• https://redmine.pfsense.org/issues/9261

The issue seems to be the NoSSL_condition. The suggested solution is to replace the condition type from Traffic is SSL (TCP request content inspection)  to Traffic is SSL (locally deciphered):

Code Select Expand


     # logging options
     # ACL: NoSSL_condition
-    acl acl_629f48c6073c95.86527303 req.ssl_ver gt 0
+    acl acl_629f48c6073c95.86527303 ssl_fc

     # ACTION: HTTPtoHTTPS_rule
     http-request redirect scheme https code 301 if !acl_629f48c6073c95.86527303


When doing so, the warning is gone. However, this is the first time I'm using HAProxy and I don't really know what I'm doing, so I wanted to check-in with you guys to ensure my solution is correct?

If so, I'd appreciate if you updated the screenshot in step 5.7 @TheHellSite

Thank you very much for this very detailed beginners guide, it helped me a lot getting things set up. Everything works but the wireguard interfaces. I even bought a month of mullvard to be sure that the vpn provider is not the problem but the wireguard service doesn't want to stay on and the vpn gateways  keep showing offline (probably as a cause of that). I did get a handshake in the wireguard config though so no idea what's the problem.
Maybe the cause is that I shouldn't have upgraded to 22.1 beta but I hope someone can help me getting it solved anyway.

Thanks in advance

edit: I've reverted to 21.7.5 now, the wireguard service is running now but the wan_vpn interfaces are still down

Adding this here also for completeness:

It turns out the solution to the problem was not to use a server adress in form of de4-wg.socks5.mullvad.net as listed on the mullvad website under servers, but to use a server adress like de4-wireguard.mullvad.net.

So I installed `wireguard-kmod` an have been testing failover gateway groups with WireGuard. It seems to be working.

However, load balancing doesn't work. I wouldn't know how to find out why.

Over the past few weeks, I created the OPNsense Baseline Guide with Mullvad VPN, Guest, and VLAN Support. It's a beginner-friendly, comprehensive step-by-step guide that replicates the popular pfSense baseline guide setup that many of you might know.

I skip over hardware selection and installation instructions as I was fortunate enough to be able to support Deciso's open-source mission by buying the DEC630 like a year ago. The only thing I regret about the purchase is that I now can't afford the sexier-looking successor model, the DEC690.  ;D

The guide covers the following topics:

• ISP and WireGuard Mullvad VPN WAN
• "Clearnet", VPN, and Guest VLAN configuration
• Simultaneous use of DNS resolver (Unbound) and forwarder (Dnsmasq) to satisfy the requirements of VLANs

I revised this guide many times as I configured and learned about the OPNsense platform. I probably clean installed my appliance more than 20 times. Publishing this guide has been on my agenda for a like a year and I'm really happy to share it with you. Any feedback is greatly appreciated and I hope you like it.

The only issue I'm having is that I can't get WireGuard multi-WAN to work. Someone commented that `wireguard-kmod` makes it possible, so I'm gonna give this a try soon.

[Outgoing Network Interfaces]

The Outgoing Network Interfaces Unbound option is what I was looking for. I got confused due to a bug that was unknown at the time.

[Dynamic gateway policy]

I successfully setup selective routing with WireGuard over one tunnel as per the tutorial from the docs for outbound internet traffic. I setup multiple tunnels and as long as I'm using only one tunnel / gateway, everything works fine. As soon as I use a gateway group to load balance traffic over all the tunnels, things stop working properly.

The docs mention this:

When assigning interfaces we can also add gateways to them. This would offer you the chance to balance traffic via different VPN providers or do more complex routing scenarios.

... and this:

When assigning interfaces, gateways can be added to them. This is useful if balancing traffic across multiple tunnels is required or in more complex routing scenarios. To do this, go to System ‣ Gateways ‣ Single and add a new gateway. Choose the relevant WireGuard interface and set the Gateway to dynamic. These scenarios are otherwise beyond the scope of this how-to

Does anyone have a link on where I can read up on the topic "beyond this how-to"? Can anyone shed some light on what the Dynamic gateway policy would do here?

In this post it's mentioned that:

But true HA / LB is not possible with WG (yet...). So all connection states will be dropped when having a failover-event.

Can anyone confirm this? Does anybody have a working multi-tunnel load balance configuration?

[remote]

Thanks you so much guys, I finally figured it out and successfully configured the static route. I already thought that I was going to lose my mind, but turns out I'm just a networking newbie and didn't understand what I had to route where ;D

<your wireguard peer>/32 --> <your ISP upstream gateway>

In hindsight, it's obvious that you mean the remote wireguard peer's IP address and not the IP of the local peer. I only understood this after looking at the routes that WG installs automatically when Disable Routes on the local peer is disabled.

I tested around some more with the Outgoing Network Interfaces option, but it has no effect. The info of the settings states:

Note that setting explicit outgoing interfaces only works when they are statically configured.

Since the WG interface has no IP configuration, might this be the reason this setting is ignored?

My tests show that Unbound only cares about the routing table. Next, I'm gonna try to configure multiple VPN gateways.

If new questions regarding multi-WAN come up, I'm gonna start a new topic.

Again, thanks so much guys, I couldn't have done it without you. I learned a lot  :-*

Restarted Unbound, nothing changed. Have a look at those minimal NAT rules... it's late... am I missing the obvious here?

[how do I embed an attachment?]

Yes, I use Mullvad, but the info is kinda outdated. You simply have to generate the WG keys with the API that the Mullvad app uses. It's undocumented but easy enough to find in the app code on GitHub. I have verified this by using my generated keys with a custom DNS with another client. The custom DNS servers I configured in the client successfully leaked through the tunnel.

Yes, more often than not I restart the whole system just to make sure everything is applied properly. As I said, I factory reset the whole system and use as little configuration as possible to understand.