Messages

jprenken solved this issue: https://gist.github.com/jprenken/18ca7bf14ddae547ae0fdf6f56d72573

I will try to check what happens if you use the nano edition and inject a config.xml directly into the image. It should actually boot properly.

That would to for us: As long as a user can just insert a USB-stick to get it running, we're fine.

Hi everyone.

We're currently evaluating OPNsense as our gateway software of choice for a large-scale rollout (potentially up to 100 devices in multple countries).

One usecase we're a bit struggling on currently is an automated setup / provisioning routine: We would like to get a new appliance (hardware) online as easily as possible.

We're aware that we can simply put the nano version on a USB-stick and boot from there. But how can we inject an initial config.xml into that image so that this gets applied on boot?

It's ok if an employee only has to configure the BIOS (remotely via a videocall) and plugin a USB-stick. But we don't want that these employees have to coupe with network configuration tasks.

Are there any howtos or recommended procedures to accomplish that?

Best

Thomas

Well due to limitations of Wireguard we were never able to have two appliances with an active tunnel on each of them.

Instead - we have a master and slave configuration which is in-sync. We can then trigger enabling / disabling Wireguard with the CARP-events. Check the CARP-scripts to accomplish this.

But true HA / LB is not possible with WG (yet...). So all connection states will be dropped when having a failover-event.

The described setup was only set up in a lab-environment. We decided that hardware-failures are very rare and that we will fail-over manually when our master gateway crashes due to hardware issues.

HA / LB brings high complexity and cost. If our master gateway runs 5-10 years uninterrupted, it's hard to justify these costs to avoid a 20-30' downtime once or twice in that lifetime.

We'll integrate HA / LB when it's natively supported by WG / OPNsense.

haha, yeah we just invested about 2 hours together to reverse-engineer this syshook mechanic. We failed.

First, we did the following: https://github.com/opnsense/plugins/issues/2049#issuecomment-705397846

Secondly, we tried to simply put a bash script inside /usr/local/etc/rc.syshook.d/config:

/usr/local/etc/rc.syshook.d/config/10-bash-git-backup

Code Select Expand

#!/usr/local/bin/bash
set -ux
git clone git@ssh.dev.com/v3/opnsense-config /conf/backup/git
cd /conf/backup/git/
mkdir -p $HOST
cp  $1 $HOST/config.xml
git add $HOST/config.xml
git commit -m "Config-Change of $HOST"
git push
cd .. && rm -rf /conf/backup/git

Third, we triggered rc.syshook manually and supplied two variables: config /conf/backup/file.xml

This triggered the script - but we never managed to trigger it from the actual event.

Basically: What is necessary so this syshook triggers properly? https://docs.opnsense.org/development/backend/autorun.html

Best

Hi everyone.

I just tried to implement the following: https://docs.opnsense.org/manual/git-backup.html

-> Unfortunately, there is no such plugin.

Further more, the documentation of a linked page is deprecated too: https://docs.opnsense.org/development/backend/autorun.html

-> Unfortunately, there is no folder "config" to embed scripts which are run upon configuration changes.

Maybe it makes sense to update these HowTos or delete them altogether.

Then only one question remains: How can I trigger a script upon configuration changes? Any help is appreciated.

Hello; me again.
We tried to get a disfunctional SFP working with these commands. Yet we failed to install these ports as the kernel sources were not available:

Code Select Expand

root@gw-zf-01:/usr/ports/net/intel-ix-kmod # make install
===>  intel-ix-kmod-3.3.14_1 requires kernel source files in SRC_BASE=/usr/src.
*** Error code 1

Stop.

Yet this seems simple: Just install the kernel sources. I just found information on this for FreeBSD: https://unix.stackexchange.com/a/205026

But I'm pretty much unfamiliar with FreeBSD / OPNsense. How can we properly build this kernel module?
Thank you very much for clarifying.

Yeah, thank you very much. I'll have a look at this.

Regarding Wireguard: There seems to be a big misunderstanding. A lot of people try to get HA-setups with Wireguard following the standards (active-active tunnels, Policy-based routing, etc...).

What I try to accomplish is not a proper HA-setup from a networking perspective: There will be downtime and there will be packets getting lost. Yet it will still fulfill our requirements to a failover: Getting the system up and running as fast as possible (basically the time it takes to initiate a tunnel and adjust the routing tables).

You can try it yourself: Create a HA-setup and activate the Wireguard tunnel on the active node. Then, disable wireguard on the active node, pull the plug and simply start the Wireguard tunnel on the secondary node: The system will work as expected and the tunnel will establish just fine.

Then, reactivate the disabled node, disable Wireguard on the failover node and reactivate the Wireguard on the primary node: The system is successfully failed back and everything works as expected.

Did I understand everything properly? What I try to accomplish is simply to automate these processes. As both our nodes are clients to a Wireguard server (road-warrior-setup), this setup should work without any issues: I can randomly start a tunnel from various different clients, as long as there is only one active at a given time, right?

Best

qdrop

Hi everyone.

Can anyone tell me, what scripts are triggered upon a failover procedure? We're using a default HA setup (as documented in https://docs.opnsense.org/manual/how-tos/carp.html.

The background is that we're using Wireguard as our VPN solution of choice. It's by far the best VPN available today. By far the best stability, performance and speed. So we're not willing to go back to IPsec or OpenVPN.

I would like that our HA setup also automatically stops / starts the Wireguard tunnel on the two cluster members - depending on their CARP status. We're aware, that this will take couple of seconds until the tunnel recovers.

Other solutions (such as having two active Wireguard tunnels) turn out to be far more complex to implement. We value simplicity.

If we're successful implementing that setup I consider documenting the whole setup. There are a lot of engineers trying to accomplish the same thing ;-).

Any help is highly appreciated.

Best

qdrop

As far as I'm aware, the Wireguard server will always respond to the IP from which the last packet originated, right?

No. Just try it .. add some virutal IPs in different ranges and from an endpoint set the different IPs.
It's stateless, the operating system will choose the highest one (depending of it's networking stack).

So I just tried said setup.

• I have two OPNsenses with their own different public IPs (without virtual ones - not necessary / one via Fiber, one via 4G.) and configured them in a HA-cluster (with just a virtual IP on the LAN side).
• I then configure Wireguard similarly on both OPNsenses. On one it's activated, on the other it's deactivated.

The clients behind the HA-setup are now properly connected to the cloud via the Wireguard VPN using the primary node.
The (currently manual) failover-procedure goes as follows:

• I disable Wireguard on the primary node.
• I enable Wireguard on the secondary node.
• Now I switch the CARP-status from the master node to maintenance mode.

The clients behind the HA-setup are now properly connected via the Wireguard VPN. I can also revert back to the primary node doing the same without any issues. On the Wireguard-Server, I can see how the endpoint changes its IP.

So maybe there was a misunderstanding. But in my opinion, my brainchild should still work: I want to automate the activation / deactivation of Wireguard according to the CARP IP state (master / slave).

I'm aware, that this will mean, that when failing over (or back), we will lose states and the VPN will go down couple of seconds. But considering that I'm mitigating a hardware-failure that ain't a problem for me.

I'm open for a remote session to demonstrate how this is working... ;-)
And as always: Any help is still highly appreciated.

Best

qdrop

WireGuard itself can not select the source IP which sends the packets.
Lets say unit 1 has 192.168.1.50, unit 2 has 192.168.1.51.
2 VIPs, 192.168.1.1 and 192.168.1.254.

WireGuard will always reply with the highest IP, there is no binding to an IP as the connection is stateless.
You have to report this feature request to WireGuard directly (did this couple of times).

Thank you very much for your response!

hmm, it should not be a problem at all if one client dies and another picks up it's configuration. I can do the same manually: Having the identical Wireguard configuration on two clients. The first is activated (wg0 active) and the second is deactivated (wg0 deactivated). I then unplug client 1 and just start Wireguard on client 2 with a command (wg-quick up...) - no problems.

Bear in mind: In our configuration, both clients are effectively Wireguard clients; none of which has Wireguard exposed as a server.

As far as I'm aware, the Wireguard server will always respond to the IP from which the last packet originated, right?

To clarify: There is no need for an active-active redundant configuration.

I'm looking into ways to cope with hardware-faults: If one gateway dies entirely, how can we make sure the second one starts the Wireguard tunnel automatically?

Best

qdrop

Hi everyone.

I would like to install a redundant setup at one of our most critical locations. This location is connected to a Wireguard server in the cloud (so it initiates the connection as client).

What I would like to do: Triggering the Wireguard starting script as soon as an appliance detects a state-change to MASTER as well as triggering the Wireguard stopping script as soon as an appliance detects a state-change to SLAVE.

In this setup, both appliances (MASTER and SLAVE) would use the same Wireguard configuration with just one of both being active.

Can anyone explain (a bit in detail) how this can be accomplished? Due to the stateless nature of Wireguard I think this would be pretty easy to implement - at least much easier then setting up active-active redundant tunnels with routing protocols, multiple paths and stuff.

Any help is very appreciated.

Best

qdrop

Go to /usr/local/etc/rc.syshook.d/start/, copy 50-wireguard to 99-wireguard and make the call
/usr/local/etc/rc.d/wireguard restart

Will this work even if the appliance gets online at a much later point in time? Like after couple of minutes?

I implemented another workaround: A static mapping within the unbound resolver.

Still, how can this rc.late hook be implemented exactly?

Hi there.

We face an issue with the wireguard-go plugin: If the WAN port of the OPNsense-appliance comes up delayed (maybe 10-20secs), Wireguard does not manage to resolve the configured endpoint properly:

Code Select Expand


...
Name does not resolve: 'hostname.domain.tld:51820'
Configuration parsing error
[#] rm -f /var/run/wireguard/wg0.sock


When opening the webinterface and clicking on "Save" within the Wireguard configuration, the tunnel establishes just fine.
This issue is a bit critical as we're using the tunnel itself to manage the deployed gateways.

Any help is highly appreciated.

qdrop