Messages

Regularly through its native UI. I follow their RSS feed for releases, and as soon as something new comes out, I update locally.

Weirdly, since the update, DNS rewrites have been flaky.  I have had to disconnect and reconnect network clients to ensure they get rewritten responses.  It may be related to IPv6 somehow, so, will look at that. 

I'd been holding off installing 23.1.6 till the AdGuard plugin was updated - once that happened, and this thread indicated things were working, I went for it.  I had a few issues (still do) that I thought I'd share.  I fully accept this may Just Be Me.

I am using AdGuard on 53, I am not using Unbound, I am not using any NAT rules to redirect 53 to some other port where AdGuard is listening.  It's a very basic setup with regards to ports, but, for my specific DNS requirements, it's been working fine.

Upon installing the 1.9 plugin for AdGuard, I enabled the tickbox for listening on 53 (Primary DNS).

Everything reported green and up and running, but, after about a minute, AdGuard stopped responding and the Dashboard said the service was not running.  I restarted it, and everything worked fine.

This morning, I rebooted my system and the same thing happened.  Everything seemed to start up just fine, but, a minute or two later, AdGuard ceased replying (and I could not reach it on the local port/dashboard for AdGuard).  The dashboard was "green" and implied the service was running, and, when the system first booted, DNS worked.

I clicked on restart again, and it's been up ever since.  I haven't been able to locate anything in the OPNSense logs that seems salient, but will keep hunting.  Again, may Just Be Me, but, thought I'd mention it.

Prior to 23.1, the ACME plugin seemed to work fine, and I had automatically renewed certificates for several months.

Somewhere around the change to 23.1, however, it no longer works via OPNSense, even though I can use Gandi's LiveDNS and API key from "letsencrypt" on a Pi just fine (so the issue is not Gandi, and not the API key).

My logs appear as such (with debug logging enabled for the ACME Settings):

Code Select Expand

2023-04-10T14:02:33 Error   opnsense    AcmeClient: validation for certificate failed: host.mydomain.com   
2023-04-10T14:02:33 Error   opnsense    AcmeClient: domain validation failed (dns01)   
2023-04-10T14:02:25 Notice  opnsense    AcmeClient: running acme.sh command: /usr/local/sbin/acme.sh --issue --syslog 7 --debug --server 'letsencrypt' --dns 'dns_gandi_livedns' --dnssleep '90' --home '/var/etc/acme-client/home' --certpath '/var/etc/acme-client/certs/whatever.07307279/cert.pem' --keypath '/var/etc/acme-client/keys/whatever.07307279/private.key' --capath '/var/etc/acme-client/certs/whatever.07307279/chain.pem' --fullchainpath '/var/etc/acme-client/certs/whatever.07307279/fullchain.pem' --domain 'host.mydomain.com' --days '1' --force --keylength '4096' --accountconf '/var/etc/acme-client/accounts/whatever.40506586_prod/account.conf'   
2023-04-10T14:02:25 Notice  opnsense    AcmeClient: using challenge type: GandiV5   
2023-04-10T14:02:25 Notice  opnsense    AcmeClient: account is registered: Let's Encrypt   
2023-04-10T14:02:25 Notice  opnsense    AcmeClient: using CA: letsencrypt   
2023-04-10T14:02:25 Notice  opnsense    AcmeClient: issue certificate: host.mydomain.com   
2023-04-10T14:02:25 Notice  opnsense    AcmeClient: certificate must be issued/renewed: host.mydomain.com

Obviously, this is in reverse chronological order.

I've obfuscated a few things, but, I do not think they are relevant to the issue.  The domain has the Gandi API enabled, the key works fine, etc etc.

What I do notice, however, is that the "dnssleep" option passed to the ACME shell script is being ignored.  I've tried various values here, 120 seconds, 240, 0 (default) - however, as you can see from the logs, within 2 seconds OPNSense records the attempt as a failure, and gives up.

Interestingly, even with "0" set as the value, the OPNSense plugin does not seem to re-try as per the on-screen note of:

The time in seconds to wait for all the TXT records to take effect after adding them to the DNS API. Defaults to 0 seconds, which causes Acme Client to check public DNS services every 10 seconds for up to 20 minutes. If set to a non-zero value, a fixed DNS sleep time will be used and the local DNS servers will be queried instead. A DNS sleep time of 120 seconds or more is recommended for some DNS APIs.

Does anyone have ACME working with 23.1 series and Gandi LiveDNS?

This won't help you in any way, but, I just signed on to the forum now to report that ACME + Gandi as a plugin haven't been working after the 23.1 upgrades.  It worked fine prior to 23.1.x, but, since then, when my 90 days have all come and gone, I've been unable to get certificates renewed or issued using DNS01 via Gandi.

I was able to get it working for HTTP challenges, but that's not what I need.

Even with extremely verbose logging turned on, and doing it from the command line, it wasn't clear why it was failing, particularly given that the configuration on my end had not changed.  I also tried re-issuing my Gandi API key, but, that had no effect. 

Like I said, doesn't directly help you, but, you're not alone...

Just a thought, but...could it be that you're able to get to the OPNsense web service due to the anti-lockout firewall rules, but, whatever system you're trying to connect to AdGuard's admin interface from does not have an appropriate firewall rule to permit the traffic?

Since you're seeing the socket is open on the OPNsense box itself, presumably if you tried the wget/curl/telnet ON the OPNSense machine TO the OPNsense machine, it would work?

It wasn't clear from your earlier posts if you were trying the wget from the local OPNsense system or not.  The sockstat was clearly the OPNsense system.

Perhaps create a custom firewall rule that logs traffic to 192.168.0.1:3000 and try to trace things from there.

What works for me (and your kilometerage may vary), is:

VPN -> WG -> Local -> Proton -> Tunnel Address = 10.2.0.2 (this was in the .conf from Proton)
VPN -> WG -> Local -> Proton -> Gateway Address = 10.2.0.1
VPN -> WG -> Local -> Proton -> Peers = Whatever endpoint you selected in the .conf generator at Proton - note that you can't fill this field out until you first go to VPN -> WG -> Endpoints and define one.
VPN -> WG -> Local -> Proton -> Disable Routes = ON

For the Endpoint for Proton, I'm permitting 0.0.0.0/0 as AllowedIPs.

Once you've done the above, you will have to make sure you've clicked "Apply" a few times, as well as stopped and restarted WireGuard.  I've found the best way to do that is NOT from the Dashboard (has not worked for me reliably), but, turning of WG via VPN -> WG -> General, untick, apply, tick, apply.

Before you can set the Single Gateway, you also have to make sure you have enabled the wg Interface for the new connection, and it's not enough just to assign it, you have to make sure it is enabled.

You do this in Interfaces -> Assignments, and from the dropdown select your new WireGuard connection, then click on the Plus symbol to the right, then Save.

Then go BACK to Interfaces and select the new Interface, make sure you've got a good description for it, enable it, and click on Save.  In my case, I make sure that the blocking of private/bogon addresses is disabled.

Then you can go over to System -> Gateways -> Single and define your GW_Proton (or whatever you want to call it).

In my case, it's set to:

Interface: Name of interface from the just defined list in the previous step
Address Family: IPv4
IP address: 10.2.0.1
Disable gateway monitoring:  OFF
Far Gateway: ON
Upstream Gateway: OFF
Monitor IP:  9.9.9.9  Bad choice, makes it seem highly latent where I live, but, it works)

And then click on Save.

Now, with this configuration, the way I understand it, the "disable routes" option in the definition of the Local endpoint means that it won't install the 0.0.0.0/0 as a route in the routing table for effectively the default.  This means that you will HAVE to use policy based routing to use the Proton VPN tunnel.  You may not want it configured this way, but I did.

From there, you still need to configure outbound NAT rules for this interface, as well as any policy based rules which decide which of your LAN hosts you want to route traffic over the GW_Proton gateway.

Of course, you also have to make sure those routing rules are placed in the right spot in your list of rules...

Hope that helps get you a bit closer!

I was able to get OPNsense and ProtonVPN working, but, it wasn't as easy as Mullvad, or any other WireGuard setup.

The primary differences that I encountered -- basing my setup off of the Mullvad example in the Docs for OPNsense, including https://docs.opnsense.org/manual/how-tos/wireguard-client-mullvad.html and https://docs.opnsense.org/manual/how-tos/wireguard-selective-routing.html were:

• Setting the local IP address of my Gateway to 10.2.0.1 - but note that this breaks some things as ProtonVPNs setup uses this IP address for the DNS server.  If you need their DNS for your WG setup, you may want to set it to 10.2.0.250 or something else not likely to cause an issue
• ProtonVPN, unlike Mullvad or other WG implementations, does not give you your public key.  It does give you the private key in the web UI briefly, but, you WILL need to use the "wg pubkey" command (cli, on your OPNsense or wherever else you have WireGuard installed, to obtain the public key from the private key and enter BOTH into OPNsense

It's the setting of the Gateway IP (on your OPNsense box) in a few places that is important to get right, as well as all the other steps outlined in the Mullvad example.   It's a lot of steps, and read carefully as there are a few gotchas.

In the end, I was able to get it working just fine, but it was more picky than other setups.

Thanks.  Forgot about that.

I added a new alias which has the MAC address of a device I care about, and ensured that alias was included in the policy based routing rules.  Easy fix!  Ta!

I realise there may be no answer to this, but, I have a few iOS devices on the LAN side of my OPNsense controlled network that I'd like to have a static IPv6 address for. 

I want this because I have some policy based routing that I'd like to apply to both IPv4 and IPv6. 

My rules all work fine, but, iOS devices tend to rotate IPv6 addresses, which means I frequently have to update the rules for the NEW IPv6 address.

Assuming it's even possible (it may not be), what's the best/right technology to use on my OPNsense box to help?

This may not help you, but, I ran into this issue on my nascent OPNsense setup within my LAN as well.  A lot of tcpdumping later I ended up resolving the issue with a split DNS configuration.  Essentially, the OPNsense device was replying to packets with a different IP address on the IMAPS/HTTPS server than the client device was requesting them from.  I couldn't get the NAT reflection to work as I was expecting, and the split DNS solution was the most simple/elegant way to resolve it in my case.

Would strongly recommend tcpdump on the client and server - see what arrives, see what replies, and with what IPs, it may provide a clue.

Solution picked from Adguard bug thread:


- SSH in your opnsense box
- Send this command, change USERNAME:PASSWORD to your Login/Pass of Adguard
- Specify port if you changed the default

Code Select Expand

curl -H 'Content-Type: application/json' -X POST -v 'http://USERNAME:PASSWORD@127.0.0.1/control/update'

Can confirm this worked - but, in my case, I had to use the IP address of my AGH box on my LAN, localhost didn't work.

Updated to .b17 successfully, no apparent loss of data.  Thank you!

Next up - load balancing across two simultaneous WireGuard links to Mullvad!

Mission accomplished.

I may write up a Tutorial on this, if there is interest.

I have now configured my OPNsense device to have two different VPN links running, with full load balancing between them. 

In my case I'm using Mullvad, but, I'll likely throw in another VPN provider I use.

It was relatively simple to sort out, once I wrapped my head around both the Multi-WAN and Multi-VPN concepts (NAT, routing rules, gateways).

[franco]

Hi
it is certainly better to wait for @franco, but if I understand the code correctly, the signature file is downloaded only if the checksum of the changelog.txz has changed (that is, if at some point the missigned changelog file was already downloaded, then after creating a new signature file it may not be downloaded if the checksum of the changelog.txz has not changed). you can try deleting changelog.txz and  changelog.txz.sig files  in /usr/local/opnsense/changelog dir and try checking for updates again

For what it is worth, this worked for me.

In my case, I blamed the fact that I sidegraded from the Business Edition to the Community Edition -- but that may not have been the reason for the error.  I made a backup of the changelog files just in case, then removed them, tried updates, and no longer have the error.

Thanks for the reminder, Bob.Dig - it was actually enough to help me solve this by thinking about the problem differently.

I was trying to solve it using only the tips in the Guide for selective routing on WireGuard/Mullvad, and, as you rightfully point out, simply looking at my problem as a "rules" problem helped.

So, the solution, for anyone who looks into this in the future, was for me to create an Alias for Networks which represented the networks in my wg2 AllowedIps.  Sadly, this does mean that if I change the IPs I have to change it in two places, but, that's not a common scenario.

Once the Alias existed, I made a new Firewall -> Rules -> LAN rule which permitted with a Quick Pass on In traffic from the "permitted" Alias to reach the new NetworkAlias which represents the AllowedIPs from the wg2.

As this rule is high enough in the list, it's matched first, and traffic goes out wg2 as needed.

For anything else, the rules processing proceeds to the wg1 interface for Mullvad, and everything works as I wanted.

Next up - load balancing across two simultaneous WireGuard links to Mullvad!

Hi,

I have several WG interfaces defined on my OPNsense box.  One (wg1) is for Mullvad VPN, with 0.0.0.0/0 as the AllowedIPs.

On another WG interface (wg2), I have a series of IP addresses or networks (100+) defined as the Allowed IPs.

I would like a specific host on my LAN to be able to use either, depending upon what the destination is.

In other words:  If the destination network IP address is one of the AllowedIPs on wg2, send the traffic to that WG gateway.  If it is NOT one of those IPs, then, use the Mullvad VPN on wg1.

I can get either situation to work separately, but, not both simultaneously.

So, if I put the host in the Alias for the wg2 network, and only in that Alias, traffic will go out wg2 for those AllowedIPs, and out my WAN for anything else.  If I put it in the wg1 permitted alias, it goes out Mullvad just fine, but, can't reach any of the hosts in the AllowedIPs for wg2.

I've followed the guide at: https://docs.opnsense.org/manual/how-tos/wireguard-selective-routing.html

And that's helped considerably, but, I'm missing something in creating the Firewall rules (probably) that would permit this.  I may also be mucking up the destination network definition of RFC1918 addresses as per that guide.

I have included ONLY my LAN IP network in that definition, as some of the IP address that are behind wg2 are in other RFC1918 space, so it seemed unwise to include those.  I also wasn't sure if I should include local WireGuard IP addresses or networks in this excluded RFC1918 alias definition.

My reading of the point of Step 8 was to "permit inbound traffic to the LAN interface from the hosts in the defined alias and NOT intended for the local RFC1918 used networks to go out through the defined WireGuard gateway".

I'm probably just getting a bit blind to the Interface and Floating rules, NAT outbound rules, and Aliases, to make this all work, but, would appreciate any insight into the best way to accomplish my goal.

Thank you!