OPNsense Forum

Archive => 22.1 Legacy Series => Topic started by: Layer8 on February 10, 2022, 07:20:40 pm

Title: Bug in wireguard ? Cant start more than 3 wireguard-servers.
Post by: Layer8 on February 10, 2022, 07:20:40 pm
Hey,

i upgraded my homelab-sense to v22.1 last week. Today i added the fourth wireguard server (wg3) for a additional site2site connection.

I tried to add wg3 as interface but i failed at first, because the interface-assignement-page reported:

Quote
The following input errors were detected:

    Port vmx0 was assigned to 2 interfaces: OPT8 OPT11

vmx0 is the interface which is attached to my DSL-modem. I am using it to reach the webinterface of my DSL-modem and also as pppoe parent interface.

But now, vmx0 is also assigned to an interface which is actually assigned to wg0. If i open this wg0_WGServer1... called interface, wg0 is still assigned, but not in the assignment-overview.

Please take a look at the screenshot to see what i mean.

It was possible to workaround the above error-msg, by klicking on the Add-New-Interface button and then directly clicking on the new assigned interface to edit it instead of klicking the save button.
Title: Re: Assignement-Bug with wireguard-interface ?
Post by: mimugmail on February 10, 2022, 07:39:38 pm
So, wg0 is badly assigned? What happens when you change this?
Title: Re: Assignement-Bug with wireguard-interface ?
Post by: Layer8 on February 10, 2022, 07:57:05 pm
Right now, wg0 is still assigned to the wg0_WGServer1-interface "in the background" (as you can see on the screenshot) and its connected with a other site right now.

I am afraid to assigne a other adapter (a VLAN-Adapter for example) to wg0_WGServer1..., because wg0 is notlonger listed in the dropdown menu.
Title: Re: Assignement-Bug with wireguard-interface ?
Post by: Layer8 on February 10, 2022, 08:07:33 pm
OK, thats not true what i said. The site which is connected to wg0 was no longer reachable.


I removed the fourth wireguard-instance from the wireguard local section and wg0 started to work immediately after that. wg0 is also available again in the dropdown menu and wg0 is also assigned to wg0_WGServer1...-Interface in the interface-overview again.

So, looks like there is a bug or limitation in wireguard or opnsense. Its not possible to start four or more wireguard servers.

Can one confirm that?
Title: Re: Bug in wireguard ? Cant start more than 3 wireguard-servers.
Post by: mimugmail on February 10, 2022, 09:13:45 pm
Cant confirm since we raised it from 19 to unlimited due to report that 19 is not enough :)
Maybe you interface names are too long? Something else in there? Hard to say from remote
Title: Re: Bug in wireguard ? Cant start more than 3 wireguard-servers.
Post by: Layer8 on February 10, 2022, 10:43:25 pm
Is there a limitation in length of interface names?

wg0 is 37 chars long
wg1 is 38 chars long
wg2 is 36 chars long

wg3 was also 38chars long.

Cant tests at the moment because some people will be mad if cut off services right now. Maybe tomorrow or at the weekend.
Title: Re: Bug in wireguard ? Cant start more than 3 wireguard-servers.
Post by: dmmincrjr on February 11, 2022, 01:09:15 am
I'm not trying to hijack a thread but I posted something similar in the VPN forum.

https://forum.opnsense.org/index.php?topic=26797.0 (https://forum.opnsense.org/index.php?topic=26797.0)

I only received 1 response from someone else who had the same issue. I've continued to try and troubleshoot without any luck. I resolved the package misconfigured in plugins manager hoping that was the problem but no luck. It was after upgrading from 21.7.8 to 22.1 I encountered the problem and reinstalled 21.7.8 with the same configuration and it now works but would like to upgrade.

I'm just posting this in case these are somehow related and will continue to watch this thread.
Title: Re: Bug in wireguard ? Cant start more than 3 wireguard-servers.
Post by: mimugmail on February 11, 2022, 08:05:14 am
I have no idea about the limitation, but when I'm in the UI I can't imagine where such a long name is correctly displayed. I'd say best to keep it under 32
Title: Re: Bug in wireguard ? Cant start more than 3 wireguard-servers.
Post by: Layer8 on February 11, 2022, 09:19:00 am
I cant see any problems with long interface-names. All long interface names are displayed correctly in the menu bar, in drop down menus if they are wide enough. And everything is working fine with long interface names.

Remember: wg0, wg1 and wg2 are working fine with interface names longer than 32chars.


Only if i add wg3, things start to get strange:

- wg0 stopped working
- suddenly, vmx0 was assigned for the interface, which was wg0 before i added wg3. what's that got to do with it?
- oh and i forgot to say: neither wg0, nor wg3 was displayed under "List Configuration" or "Handshake" section in wireguard interface.







Title: Re: Bug in wireguard ? Cant start more than 3 wireguard-servers.
Post by: mimugmail on February 11, 2022, 01:56:25 pm
Go to UI and do a:

cat /usr/local/etc/wireguard/wg0.conf

And

/usr/local/etc/rc.d/wireguard restart

When you added the 4th service. Maybe wg0 is not available anymore because it doesn't start cause of a config error or overlapping in 4th instance. Just guessing ..

Also, what is the exact reason why you assign those wg interfaces?
Title: Re: Bug in wireguard ? Cant start more than 3 wireguard-servers.
Post by: Layer8 on February 11, 2022, 03:45:54 pm
I will test this at the weekend.

I think its easier to keep the overview of firewall rulesets if i can set it per site/wg-interface. Its messy if dozens of rules of different sites are defined on one wg-interface.

Also, its easier to cut off a site temporarily by just disabling a wg-interface.
Title: Re: Bug in wireguard ? Cant start more than 3 wireguard-servers.
Post by: zitlo on February 11, 2022, 07:37:30 pm
Hello @all

I just updated two boxes to OPNsense 22.1-amd64 and I cant start my wireguard services anymore.

I reinstalled wireguard packages and I rebooted OPNsense twice, no success.
Its difficult right now to get logs, because I do not have SSH access right now...
Title: Re: Bug in wireguard ? Cant start more than 3 wireguard-servers.
Post by: Layer8 on February 13, 2022, 05:19:56 pm
I think i found the reason for this problem and basically it was my fault so its not absolutely a bug, but maybe a not working or net yet implemented interception of misconfiguration.

This error happens, when a wg-instance is configured with an subnetaddress or subnetrange, which overlaps with a subnetaddress or subnetrange of a endpoint - but only if this endpoint is peered with a wg-intance.

I came up with this problem because i did cat /usr/local/etc/wireguard/wg*.conf for all configured wg-interfaces and compared the configuration. I found a overlap between tunnel address of wg0 and a endpoint, which i peered with wg3. This led into a crash of wg0  (which was replaced with vmx0 in the interface-assignement overview) and into a not starting wg3 i think.


So, two suggestions:

1. It would be nice to see a "running config" tab in wireguard, which lists all wg#.conf-igurations in one view. This would give a better overview for debugging.

2. It would be nice if you could add interception of those kind of misconfigurations.


Title: Re: Bug in wireguard ? Cant start more than 3 wireguard-servers.
Post by: zitlo on February 13, 2022, 11:05:02 pm
Hey Layer8,

thank you for this solution, but it didnt solved my problem. I cant start wireguard service.

I moved all configurations /usr/local/etc/wireguard/wg*.conf and wireguard is still not starting anymore.

How I can debug this? Where to look for logs?
Title: Re: Bug in wireguard ? Cant start more than 3 wireguard-servers.
Post by: mimugmail on February 14, 2022, 06:22:56 am
/usr/local/etc/rc.d/wireguard restart
Title: Re: Bug in wireguard ? Cant start more than 3 wireguard-servers.
Post by: zitlo on February 14, 2022, 09:59:07 am
/usr/local/etc/rc.d/wireguard restart

Thank you!

Wireguard restarts just fine, /usr/local/etc/rc.d/wireguard status is showing also an correct output... My wireguard service status inside the gui is red...

Edit:
- I deleted all configurations inside /usr/local/etc/rc.d/wireguard and inside the gui
- I uninstalled the wireguard plugin
- Then I restarted opnsense.
- I innstalled the plugin again
- I created a new configuration

Same problem, gui status showing red and service is starting on cli
Title: Re: Bug in wireguard ? Cant start more than 3 wireguard-servers.
Post by: Patrick M. Hausen on February 14, 2022, 10:44:06 am
Are you using the kmod? If yes, the dashboard will always show a wrong state of the service. You can remove the WireGuard service from the services widget and add the WireGuard widget instead, which gives you an overview of all your peers.
Title: Re: Bug in wireguard ? Cant start more than 3 wireguard-servers.
Post by: zitlo on February 14, 2022, 10:59:03 am
Hello pmhausen,

I dont know what kmod is, but now I discovered (after I uninstalled the wireguard plugin) that I have an unassigned interface (wg0). How can I delete it? from the gui it is not possible
Title: Re: Bug in wireguard ? Cant start more than 3 wireguard-servers.
Post by: Patrick M. Hausen on February 14, 2022, 11:42:45 am
You have (and need) a wgN interface for each WireGuard server that is up and running. If you just uninstalled WG, the interfaces might be left - a reboot will fix that.

You do not need to assign the interfaces.

A kmod is a kernel module. If you did a `pkg install wireguard-kmod` on the command line, you will be running that instead of wireguard-go. Everything will supposedly work just the same but the service status on the dashboard will be broken.
Title: Re: Bug in wireguard ? Cant start more than 3 wireguard-servers.
Post by: zitlo on February 14, 2022, 01:40:21 pm
Thank you for your help!
Title: Re: Bug in wireguard ? Cant start more than 3 wireguard-servers.
Post by: dmmincrjr on February 14, 2022, 04:00:41 pm
I'm having an issue where I can't get one interface to start after the upgrade and I can't figure it out as the config file works in 21.7.8 but not after the upgrade. I have 3 sites that I'm using Wireguard to connect and two of those sites I have upgraded to 22.1 and Wireguard worked without issue.

Edit:

I did finally figure out the issue. I had a misconfiguration of the Allowed IP's in the tunnel. The strange thing was the tunnel worked with the misconfiguration in versions up to 21.7.8 for a couple years which is why I had assumed my configuration was correct and something else was the problem.