Messages

In /usr/local/etc/wireguard do you have a config file? I've learned you not only need to save configurations for both ends, and make sure that the Local specifies the Peer after the Peer is defined, but you need to then "Save" the overview page that then has the already-saved definitions, or else the appropriate config file isn't placed there. In other locations in the OPNsense UI, you get an "Apply Changes" button on the top right whenever there's an additional step required to apply something you've configured. But the WireGuard module isn't consistent with that design. It's a solid piece of work, just rough around the edges.

I see in /usr/local/etc/rc.syshook.d/carp the file 20-openvpn with references to MASTER and BACKUP. My guess is this is part of a mechanism which can be extended to start and stop WireGuard according to whether the firewall is in MASTER or BACKUP status.

Is OPNsense using CARP in a standard BSD way here, or are the files controlling it somewhat different in arrangement? That is, can I depend on the BSD docs for guidance on this?

Thanks,
Whit

There is something glitchy in that though. It initiated the third tunnel with those "extra" Save options, but it dropped one of the existing tunnels. Going through the pages again for the now-dropped tunnel, pressing Save for the individual as well as group pages, got us back to having three tunnels running.

Again, that was an established tunnel. Nothing was changed on it. And nothing was changed on it since then either. There might be some timing dependency as it starts multiple WireGuard tunnels that can cause it to fail on one.

Whit

Ah! I see the problem. It is necessary not just to hit Save in both Local and Endpoint individual setup pages, but also the Save again on at least the Local page listing each of the Local endpoint -- even though they've already been saved individually. So it's not the Save button for the individual Local and Endpoint setup pages which builds the config file, but the Save button for the page which lists all the Locals.

Perhaps that would be clearer if it were labelled "Apply" rather than "Save"?

Best,
Whit

This is both in Firefox and Chrome. There should be a button there for the export to function, shouldn't there?

Thanks,
Whit

You're not understanding what I'm saying. It's not about IP. It's about integrating a line to start or stop WireGuard into the scripts which run when CARP orchestrates an IP takeover. I don't care about the IPs. I care about whether the system thinks it's the master or the standby. When CARP tells the standby to take over as master, the trick is to have it also start WireGuard, with /usr/local/etc/rc.d/wireguard start. And when it surrenders that, to stop WireGuard similarly.

That there's also IP takeover in this case is coincident, but the state of the virtual IPs is logically independent of this, beyond the coincident use of the same mechanism to trigger both those and WireGuard.

Whit

Thanks for the suggestion. It reveals nothing:

root@OPNsenseFL1:~ # /usr/local/etc/rc.d/wireguard restart

• rm -f /var/run/wireguard/wg0.sock
  
• rm -f /var/run/wireguard/wg1.sock
  
• wireguard-go wg0
  INFO: (wg0) 2019/09/19 15:37:00 Starting wireguard-go version 0.0.20190805
  
• wg setconf wg0 /tmp/tmp.6RZWn0s6/sh-np.V0lHXz
  
• ifconfig wg0 inet 10.222.0.15/32 10.222.0.15 alias
  
• ifconfig wg0 mtu 1420
  
• ifconfig wg0 up
  
• route -q -n add -inet 10.222.0.1/32 -interface wg0
  
• route -q -n add -inet 192.168.1.0/24 -interface wg0
  
• route -q -n add -inet 172.25.0.0/16 -interface wg0
  
• route -q -n add -inet 172.17.0.0/16 -interface wg0
  
• Backgrounding route monitor
  
• wireguard-go wg1
  INFO: (wg1) 2019/09/19 15:37:01 Starting wireguard-go version 0.0.20190805
  
• wg setconf wg1 /tmp/tmp.AMDHjvxD/sh-np.B7mD5q
  
• ifconfig wg1 inet 10.0.222.15/32 10.0.222.15 alias
  
• ifconfig wg1 mtu 1420
  
• ifconfig wg1 up
  
• route -q -n add -inet 10.0.222.15/32 -interface wg1
  
• route -q -n add -inet 10.0.222.110/32 -interface wg1
  
• route -q -n add -inet 172.25.10.0/24 -interface wg1
  
• route -q -n add -inet 172.16.0.0/16 -interface wg1
  
• Backgrounding route monitor
  
  And it stops there, no error, and nothing about the wg2 interface that should be there.
  
  Looking in /usr/local/etc/wireguard, there is only wg0.conf and wg1.conf. Yet the VPN: WireGuard screens on OPNsense show both the Local and Endpoint setups that should produce wg2.conf.
  
  Whit
  
  

After success with two tunnels, trying to add a third, but while ifconfig shows wg0 and wg1, there's no wg2 as there should be now. Is this a limitation in the current implementation here, or have I missed something?

What I'm asking then is a way to script for CARP to turn WireGuard off and on in coordination with IP takeover when the master system is down. I know this is doable. (It's simple enough using UCARP on Linux.) I just don't know my way around how OPNsense uses CARP here.

Yes, this goes beyond the pretty menus. But it's where I need to be.

Is there a way to tie WireGuard to CARP takeover? Obviously I can't just have it already running on both systems, with the same remote connections. On the DR system, once running it should connect to the remote systems at their IPs even if it's not coming from the expected IP on the OPNsense end; that's a feature of WireGuard. I'm guessing this will take scripting tie it to CARP, so that it only starts when CARP triggers it. If so, where should that be tied in?

Apologies if this is documented somewhere I haven't found yet. Thanks for any pointers.

Whit

Any hints at all on how to add virtual IPs without having routing through the static IP already on the interface interrupted? Or is the presumption here that an interface will only have either a static IP or else a set of one or more virtuals?

With Linux, I'm used to mixing them, often using UCARP for the virtuals. Obviously this is a different world. What docs should I be reading to understand this aspect of it?

Finally got in, deleted the added IPs, and the original IP is back in business. But what's the problem here? I added them as /24s, since these are on a /24 block which the gateway is .1 of. I've routinely added IPs to Linux boxes for years, and in that context either a /24 or a /32 would have worked, without screwing up anything for the original IP on an interface. I take it there's something more to know about how either OPNsense or BSD adjusts routing when a new IP goes on the WAN interface. But I've no clue what that is, or how to avoid it breaking things as it partially did here.

All hints are welcomed.

Finally got in from an RDP session to a desktop on the system on the other end of the WireGuard tunnel.

I'd suggest it's a bad idea to have the HTTP_REFERER protection on by default. The odds of blocking a bad actor with that are far lower than the odds that a good actor will need flexibility of access.

It's not the port that it needs to match. Switching to 443 still gives me:

The HTTP_REFERER "https://172.17.10.1/" does not match the predefined settings. You can disable this check if needed under System: Settings: Administration.

I'm at this point because I'm blocked from accessing via the public IP because adding two other IPs to the WAN interface cut that off from being connected to by HTTPS or ping. And my way in now is via a WireGuard tunnel, which in turn I'm trying to access through an xinetd port forwarder set up on the other end of that tunnel. The system on the other end is a remote Linux server without a GUI desktop.

From that system directly I can do w3m and get the login screen on the OPNsense box. But from my desktop remote to that, when I go to the xinetd port forwarding set up there, I get:

The HTTP_REFERER "https://172.17.10.1:444/" does not match the predefined settings. You can disable this check if needed under System: Settings: Administration.

What? I never predefined any setting for this. I've not enabled ssh access. How do I get control back here?