1
High availability / Re: Triggered scripts on failover
« on: April 22, 2022, 03:55:03 pm »
jprenken solved this issue: https://gist.github.com/jprenken/18ca7bf14ddae547ae0fdf6f56d72573
Show Posts
This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.
Pages: [1] 2
2
22.1 Legacy Series / Re: Automatic minimal setup / provisioning
« on: April 14, 2022, 10:04:45 am »
I will try to check what happens if you use the nano edition and inject a config.xml directly into the image. It should actually boot properly.
That would to for us: As long as a user can just insert a USB-stick to get it running, we're fine.
That would to for us: As long as a user can just insert a USB-stick to get it running, we're fine.
3
22.1 Legacy Series / Automatic minimal setup / provisioning
« on: April 13, 2022, 03:54:52 pm »
Hi everyone.
We're currently evaluating OPNsense as our gateway software of choice for a large-scale rollout (potentially up to 100 devices in multple countries).
One usecase we're a bit struggling on currently is an automated setup / provisioning routine: We would like to get a new appliance (hardware) online as easily as possible.
We're aware that we can simply put the nano version on a USB-stick and boot from there. But how can we inject an initial config.xml into that image so that this gets applied on boot?
It's ok if an employee only has to configure the BIOS (remotely via a videocall) and plugin a USB-stick. But we don't want that these employees have to coupe with network configuration tasks.
Are there any howtos or recommended procedures to accomplish that?
Best
Thomas
We're currently evaluating OPNsense as our gateway software of choice for a large-scale rollout (potentially up to 100 devices in multple countries).
One usecase we're a bit struggling on currently is an automated setup / provisioning routine: We would like to get a new appliance (hardware) online as easily as possible.
We're aware that we can simply put the nano version on a USB-stick and boot from there. But how can we inject an initial config.xml into that image so that this gets applied on boot?
It's ok if an employee only has to configure the BIOS (remotely via a videocall) and plugin a USB-stick. But we don't want that these employees have to coupe with network configuration tasks.
Are there any howtos or recommended procedures to accomplish that?
Best
Thomas
4
20.1 Legacy Series / Re: Wireguard with MultiWAN setup
« on: April 07, 2021, 09:53:54 am »
Well due to limitations of Wireguard we were never able to have two appliances with an active tunnel on each of them.
Instead - we have a master and slave configuration which is in-sync. We can then trigger enabling / disabling Wireguard with the CARP-events. Check the CARP-scripts to accomplish this.
But true HA / LB is not possible with WG (yet...). So all connection states will be dropped when having a failover-event.
The described setup was only set up in a lab-environment. We decided that hardware-failures are very rare and that we will fail-over manually when our master gateway crashes due to hardware issues.
HA / LB brings high complexity and cost. If our master gateway runs 5-10 years uninterrupted, it's hard to justify these costs to avoid a 20-30' downtime once or twice in that lifetime.
We'll integrate HA / LB when it's natively supported by WG / OPNsense.
Instead - we have a master and slave configuration which is in-sync. We can then trigger enabling / disabling Wireguard with the CARP-events. Check the CARP-scripts to accomplish this.
But true HA / LB is not possible with WG (yet...). So all connection states will be dropped when having a failover-event.
The described setup was only set up in a lab-environment. We decided that hardware-failures are very rare and that we will fail-over manually when our master gateway crashes due to hardware issues.
HA / LB brings high complexity and cost. If our master gateway runs 5-10 years uninterrupted, it's hard to justify these costs to avoid a 20-30' downtime once or twice in that lifetime.
We'll integrate HA / LB when it's natively supported by WG / OPNsense.
5
20.7 Legacy Series / Re: Deprecated Documentation
« on: October 19, 2020, 03:34:53 pm »
haha, yeah we just invested about 2 hours together to reverse-engineer this syshook mechanic. We failed.
First, we did the following: https://github.com/opnsense/plugins/issues/2049#issuecomment-705397846
Secondly, we tried to simply put a bash script inside /usr/local/etc/rc.syshook.d/config:
/usr/local/etc/rc.syshook.d/config/10-bash-git-backup
Third, we triggered rc.syshook manually and supplied two variables: config /conf/backup/file.xml
This triggered the script - but we never managed to trigger it from the actual event.
Basically: What is necessary so this syshook triggers properly? https://docs.opnsense.org/development/backend/autorun.html
Best
First, we did the following: https://github.com/opnsense/plugins/issues/2049#issuecomment-705397846
Secondly, we tried to simply put a bash script inside /usr/local/etc/rc.syshook.d/config:
/usr/local/etc/rc.syshook.d/config/10-bash-git-backup
Code: [Select]
#!/usr/local/bin/bash
set -ux
git clone git@ssh.dev.com/v3/opnsense-config /conf/backup/git
cd /conf/backup/git/
mkdir -p $HOST
cp $1 $HOST/config.xml
git add $HOST/config.xml
git commit -m "Config-Change of $HOST"
git push
cd .. && rm -rf /conf/backup/gitThird, we triggered rc.syshook manually and supplied two variables: config /conf/backup/file.xml
This triggered the script - but we never managed to trigger it from the actual event.
Basically: What is necessary so this syshook triggers properly? https://docs.opnsense.org/development/backend/autorun.html
Best
6
20.7 Legacy Series / Deprecated Documentation
« on: October 19, 2020, 10:52:07 am »
Hi everyone.
I just tried to implement the following: https://docs.opnsense.org/manual/git-backup.html
-> Unfortunately, there is no such plugin.
Further more, the documentation of a linked page is deprecated too: https://docs.opnsense.org/development/backend/autorun.html
-> Unfortunately, there is no folder "config" to embed scripts which are run upon configuration changes.
Maybe it makes sense to update these HowTos or delete them altogether.
Then only one question remains: How can I trigger a script upon configuration changes? Any help is appreciated.
I just tried to implement the following: https://docs.opnsense.org/manual/git-backup.html
-> Unfortunately, there is no such plugin.
Further more, the documentation of a linked page is deprecated too: https://docs.opnsense.org/development/backend/autorun.html
-> Unfortunately, there is no folder "config" to embed scripts which are run upon configuration changes.
Maybe it makes sense to update these HowTos or delete them altogether.
Then only one question remains: How can I trigger a script upon configuration changes? Any help is appreciated.
7
German - Deutsch / Re: Probleme mit X520-DA2: no carrier - neueren Intel Treiber kompilieren ?
« on: October 13, 2020, 09:24:39 pm »
Hello; me again.
We tried to get a disfunctional SFP working with these commands. Yet we failed to install these ports as the kernel sources were not available:
Yet this seems simple: Just install the kernel sources. I just found information on this for FreeBSD: https://unix.stackexchange.com/a/205026
But I'm pretty much unfamiliar with FreeBSD / OPNsense. How can we properly build this kernel module?
Thank you very much for clarifying.
We tried to get a disfunctional SFP working with these commands. Yet we failed to install these ports as the kernel sources were not available:
Code: [Select]
root@gw-zf-01:/usr/ports/net/intel-ix-kmod # make install
===> intel-ix-kmod-3.3.14_1 requires kernel source files in SRC_BASE=/usr/src.
*** Error code 1
Stop.Yet this seems simple: Just install the kernel sources. I just found information on this for FreeBSD: https://unix.stackexchange.com/a/205026
But I'm pretty much unfamiliar with FreeBSD / OPNsense. How can we properly build this kernel module?
Thank you very much for clarifying.
8
High availability / Re: Triggered scripts on failover
« on: October 05, 2020, 11:02:12 am »
Yeah, thank you very much. I'll have a look at this.
Regarding Wireguard: There seems to be a big misunderstanding. A lot of people try to get HA-setups with Wireguard following the standards (active-active tunnels, Policy-based routing, etc...).
What I try to accomplish is not a proper HA-setup from a networking perspective: There will be downtime and there will be packets getting lost. Yet it will still fulfill our requirements to a failover: Getting the system up and running as fast as possible (basically the time it takes to initiate a tunnel and adjust the routing tables).
You can try it yourself: Create a HA-setup and activate the Wireguard tunnel on the active node. Then, disable wireguard on the active node, pull the plug and simply start the Wireguard tunnel on the secondary node: The system will work as expected and the tunnel will establish just fine.
Then, reactivate the disabled node, disable Wireguard on the failover node and reactivate the Wireguard on the primary node: The system is successfully failed back and everything works as expected.
Did I understand everything properly? What I try to accomplish is simply to automate these processes. As both our nodes are clients to a Wireguard server (road-warrior-setup), this setup should work without any issues: I can randomly start a tunnel from various different clients, as long as there is only one active at a given time, right?
Best
qdrop
Regarding Wireguard: There seems to be a big misunderstanding. A lot of people try to get HA-setups with Wireguard following the standards (active-active tunnels, Policy-based routing, etc...).
What I try to accomplish is not a proper HA-setup from a networking perspective: There will be downtime and there will be packets getting lost. Yet it will still fulfill our requirements to a failover: Getting the system up and running as fast as possible (basically the time it takes to initiate a tunnel and adjust the routing tables).
You can try it yourself: Create a HA-setup and activate the Wireguard tunnel on the active node. Then, disable wireguard on the active node, pull the plug and simply start the Wireguard tunnel on the secondary node: The system will work as expected and the tunnel will establish just fine.
Then, reactivate the disabled node, disable Wireguard on the failover node and reactivate the Wireguard on the primary node: The system is successfully failed back and everything works as expected.
Did I understand everything properly? What I try to accomplish is simply to automate these processes. As both our nodes are clients to a Wireguard server (road-warrior-setup), this setup should work without any issues: I can randomly start a tunnel from various different clients, as long as there is only one active at a given time, right?
Best
qdrop
9
High availability / Triggered scripts on failover
« on: October 02, 2020, 04:04:50 pm »
Hi everyone.
Can anyone tell me, what scripts are triggered upon a failover procedure? We're using a default HA setup (as documented in https://docs.opnsense.org/manual/how-tos/carp.html.
The background is that we're using Wireguard as our VPN solution of choice. It's by far the best VPN available today. By far the best stability, performance and speed. So we're not willing to go back to IPsec or OpenVPN.
I would like that our HA setup also automatically stops / starts the Wireguard tunnel on the two cluster members - depending on their CARP status. We're aware, that this will take couple of seconds until the tunnel recovers.
Other solutions (such as having two active Wireguard tunnels) turn out to be far more complex to implement. We value simplicity.
If we're successful implementing that setup I consider documenting the whole setup. There are a lot of engineers trying to accomplish the same thing ;-).
Any help is highly appreciated.
Best
qdrop
Can anyone tell me, what scripts are triggered upon a failover procedure? We're using a default HA setup (as documented in https://docs.opnsense.org/manual/how-tos/carp.html.
The background is that we're using Wireguard as our VPN solution of choice. It's by far the best VPN available today. By far the best stability, performance and speed. So we're not willing to go back to IPsec or OpenVPN.
I would like that our HA setup also automatically stops / starts the Wireguard tunnel on the two cluster members - depending on their CARP status. We're aware, that this will take couple of seconds until the tunnel recovers.
Other solutions (such as having two active Wireguard tunnels) turn out to be far more complex to implement. We value simplicity.
If we're successful implementing that setup I consider documenting the whole setup. There are a lot of engineers trying to accomplish the same thing ;-).
Any help is highly appreciated.
Best
qdrop
10
20.7 Legacy Series / Re: Wireguard failover with CARP and scripts
« on: September 07, 2020, 04:58:28 pm »
As far as I'm aware, the Wireguard server will always respond to the IP from which the last packet originated, right?
No. Just try it .. add some virutal IPs in different ranges and from an endpoint set the different IPs.
It's stateless, the operating system will choose the highest one (depending of it's networking stack).
So I just tried said setup.
- I have two OPNsenses with their own different public IPs (without virtual ones - not necessary / one via Fiber, one via 4G.) and configured them in a HA-cluster (with just a virtual IP on the LAN side).
- I then configure Wireguard similarly on both OPNsenses. On one it's activated, on the other it's deactivated.
The (currently manual) failover-procedure goes as follows:
- I disable Wireguard on the primary node.
- I enable Wireguard on the secondary node.
- Now I switch the CARP-status from the master node to maintenance mode.
So maybe there was a misunderstanding. But in my opinion, my brainchild should still work: I want to automate the activation / deactivation of Wireguard according to the CARP IP state (master / slave).
I'm aware, that this will mean, that when failing over (or back), we will lose states and the VPN will go down couple of seconds. But considering that I'm mitigating a hardware-failure that ain't a problem for me.
I'm open for a remote session to demonstrate how this is working... ;-)
And as always: Any help is still highly appreciated.
Best
qdrop
11
20.7 Legacy Series / Re: Wireguard failover with CARP and scripts
« on: September 04, 2020, 04:10:34 pm »WireGuard itself can not select the source IP which sends the packets.
Lets say unit 1 has 192.168.1.50, unit 2 has 192.168.1.51.
2 VIPs, 192.168.1.1 and 192.168.1.254.
WireGuard will always reply with the highest IP, there is no binding to an IP as the connection is stateless.
You have to report this feature request to WireGuard directly (did this couple of times).
Thank you very much for your response!
hmm, it should not be a problem at all if one client dies and another picks up it's configuration. I can do the same manually: Having the identical Wireguard configuration on two clients. The first is activated (wg0 active) and the second is deactivated (wg0 deactivated). I then unplug client 1 and just start Wireguard on client 2 with a command (wg-quick up...) - no problems.
Bear in mind: In our configuration, both clients are effectively Wireguard clients; none of which has Wireguard exposed as a server.
As far as I'm aware, the Wireguard server will always respond to the IP from which the last packet originated, right?
To clarify: There is no need for an active-active redundant configuration.
I'm looking into ways to cope with hardware-faults: If one gateway dies entirely, how can we make sure the second one starts the Wireguard tunnel automatically?
Best
qdrop
12
20.7 Legacy Series / Wireguard failover with CARP and scripts
« on: September 04, 2020, 11:26:04 am »
Hi everyone.
I would like to install a redundant setup at one of our most critical locations. This location is connected to a Wireguard server in the cloud (so it initiates the connection as client).
What I would like to do: Triggering the Wireguard starting script as soon as an appliance detects a state-change to MASTER as well as triggering the Wireguard stopping script as soon as an appliance detects a state-change to SLAVE.
In this setup, both appliances (MASTER and SLAVE) would use the same Wireguard configuration with just one of both being active.
Can anyone explain (a bit in detail) how this can be accomplished? Due to the stateless nature of Wireguard I think this would be pretty easy to implement - at least much easier then setting up active-active redundant tunnels with routing protocols, multiple paths and stuff.
Any help is very appreciated.
Best
qdrop
I would like to install a redundant setup at one of our most critical locations. This location is connected to a Wireguard server in the cloud (so it initiates the connection as client).
What I would like to do: Triggering the Wireguard starting script as soon as an appliance detects a state-change to MASTER as well as triggering the Wireguard stopping script as soon as an appliance detects a state-change to SLAVE.
In this setup, both appliances (MASTER and SLAVE) would use the same Wireguard configuration with just one of both being active.
Can anyone explain (a bit in detail) how this can be accomplished? Due to the stateless nature of Wireguard I think this would be pretty easy to implement - at least much easier then setting up active-active redundant tunnels with routing protocols, multiple paths and stuff.
Any help is very appreciated.
Best
qdrop
13
20.7 Legacy Series / Re: Wireguard-Go not able to start tunnel if WAN not available on boot
« on: September 04, 2020, 11:19:55 am »Go to /usr/local/etc/rc.syshook.d/start/, copy 50-wireguard to 99-wireguard and make the call
/usr/local/etc/rc.d/wireguard restart
Will this work even if the appliance gets online at a much later point in time? Like after couple of minutes?
14
20.7 Legacy Series / Re: Wireguard-Go not able to start tunnel if WAN not available on boot
« on: September 03, 2020, 04:32:55 pm »
I implemented another workaround: A static mapping within the unbound resolver.
Still, how can this rc.late hook be implemented exactly?
Still, how can this rc.late hook be implemented exactly?
15
20.7 Legacy Series / Wireguard-Go not able to start tunnel if WAN not available on boot
« on: September 03, 2020, 02:19:18 pm »
Hi there.
We face an issue with the wireguard-go plugin: If the WAN port of the OPNsense-appliance comes up delayed (maybe 10-20secs), Wireguard does not manage to resolve the configured endpoint properly:
When opening the webinterface and clicking on "Save" within the Wireguard configuration, the tunnel establishes just fine.
This issue is a bit critical as we're using the tunnel itself to manage the deployed gateways.
Any help is highly appreciated.
qdrop
We face an issue with the wireguard-go plugin: If the WAN port of the OPNsense-appliance comes up delayed (maybe 10-20secs), Wireguard does not manage to resolve the configured endpoint properly:
Code: [Select]
...
Name does not resolve: 'hostname.domain.tld:51820'
Configuration parsing error
[#] rm -f /var/run/wireguard/wg0.sock
When opening the webinterface and clicking on "Save" within the Wireguard configuration, the tunnel establishes just fine.
This issue is a bit critical as we're using the tunnel itself to manage the deployed gateways.
Any help is highly appreciated.
qdrop
Pages: [1] 2