OPNsense Forum

Archive => 18.1 Legacy Series => Topic started by: elektroinside on January 12, 2018, 07:50:34 pm

Title: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 12, 2018, 07:50:34 pm
Franco, switched to devel, upgraded to rc1  ;D
Just couldn't wait  ::)

But now.. OpenVPN server will not start:

Code: [Select]
Jan 12 20:46:02 openvpn[56490]: Use --help for more information.
Jan 12 20:46:02 openvpn[56490]: Options error: --verify-client-cert none|optional must be used with --management-client-auth, an --auth-user-pass-verify script, or plugin
Jan 12 20:46:02 openvpn[56490]: DEPRECATED OPTION: --client-cert-not-required, use --verify-client-cert instead
Title: Re: OpenVPN will not start on 18.1.r1
Post by: franco on January 12, 2018, 08:19:06 pm
Hi there,

Whoops, try this then: https://github.com/opnsense/core/commit/0ec330d7

Apply via console...

# opnsense-patch 0ec330d7


Cheers,
Franco
Title: Re: OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 12, 2018, 08:25:19 pm
Yep, this did it, fixed.

P.S. The support here is incredible!  :)

Thank you!
Title: Re: OpenVPN will not start on 18.1.r1
Post by: mimugmail on January 13, 2018, 06:47:16 am
Also fixed for me, thanks! :)
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: franco on January 13, 2018, 10:46:54 am
Thanks guys!

@elektroinside

Can you try this patch on top? https://github.com/opnsense/core/commit/d215ab49

# opnsense-patch d215ab49

(rerun again to remove if not working)

@mimugmail

Same error or different one? It's important.
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 13, 2018, 11:49:48 am
Hi Franco,

I did, but cannot tell if it's working because:


I was investigating this until i saw your post, i'll remove the alias rule to test the VPN patch and get back with the results.

In the meantime, do you have any idea why pf is behaving like this? The alias issue also could be a problem from pf?

Thanks.
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 13, 2018, 11:58:20 am
Confirming that d215ab49 vpn patch works fine:

Issues remaining on my side: the alias resolution and the strange need to restart pf after OPNsense reboot...
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 13, 2018, 12:16:25 pm
Update:

https://github.com/opnsense/core/commit/60e4e8080 seems to have fixed the alias problem.

I still need to restart pf in order to get the internet working...
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: franco on January 13, 2018, 12:37:48 pm
What kind of WAN link do you use? Does this affect IPv4 and IPv6 or just one of them? Can you ping the Internet from the OPNsense box before restarting pf?


Cheers,
Franco
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 13, 2018, 01:07:29 pm
What kind of WAN link do you use? Does this affect IPv4 and IPv6 or just one of them? Can you ping the Internet from the OPNsense box before restarting pf?


Cheers,
Franco
It's a PPPoE link. Disabling IPv6 on the WAN didn't help, so IPv4 for sure is affected. I can reproduce every time.

I can ping from the OPNsense box, i can't from the LAN clients, not until i restart pf. This was not an issue with 17.7.11 (latest stable from the 17 branch, i guess this is it).
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: franco on January 13, 2018, 01:12:59 pm
Can you try flipping this patch to see if it helps?

https://github.com/opnsense/core/commit/50e53ab4


Cheers,
Franco
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 13, 2018, 01:30:07 pm
So..

Code: [Select]
Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|From 50e53ab4a0698f08c21f1b8efefb10622224483a Mon Sep 17 00:00:00 2001
|From: Franco Fichtner <franco@opnsense.org>
|Date: Sat, 16 Sep 2017 17:57:46 +0200
|Subject: [PATCH] interfaces: reload filter before reloading plugins for
| connectivity
|
|PR: https://forum.opnsense.org/index.php?topic=4727.0
|PR: https://github.com/opnsense/core/issues/1403
|---
| src/etc/rc.newwanip   | 7 ++++---
| src/etc/rc.newwanipv6 | 7 ++++---
| 2 files changed, 8 insertions(+), 6 deletions(-)
|
|diff --git a/src/etc/rc.newwanip b/src/etc/rc.newwanip
|index 8271b8476..486d3e2a5 100755
|--- a/src/etc/rc.newwanip
|+++ b/src/etc/rc.newwanip
--------------------------
Patching file etc/rc.newwanip using Plan A...
Reversed (or previously applied) patch detected!  Assuming -R.Hunk #1 succeeded                                                                                                                                                              at 162.
Hmm...  The next patch looks like a unified diff to me...
The text leading up to this was:
--------------------------
|diff --git a/src/etc/rc.newwanipv6 b/src/etc/rc.newwanipv6
|index 6d1259713..1438c4f51 100755
|--- a/src/etc/rc.newwanipv6
|+++ b/src/etc/rc.newwanipv6
--------------------------
Patching file etc/rc.newwanipv6 using Plan A...
Reversed (or previously applied) patch detected!  Assuming -R.Hunk #1 succeeded                                                                                                                                                              at 143.
done
All patches have been applied successfully.  Have a nice day.


After applying the patch, i logged in the GUI. After ~30secs got logged out automatically (something has restarted/reloaded stuff which logged me out) from the GUI, but on the other hand, on the LAN side things started to work again without pf restart.

And so i reapplied the patch, and back to the issue, reproduced again.
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 13, 2018, 01:37:23 pm
And so i removed the patch once again (basically trying again what you previously asked me, removing the patch).
It didn't work this time. I did not got logged out from the GUI and i still needed to restart pf...
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: franco on January 13, 2018, 01:53:21 pm
Sounds like a timing issue, your PPPoE could be slow to receive an IP initially. I don't think the GUI logout is related.

Are you using OpenVPN, IPsec or Dynamic DNS?

If you don't mind, from just having fixed the non-working state, I would like to inspect the diff of the generated rules:

# diff -u /tmp/rules.debug{.old,}


Cheers,
Franco
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: franco on January 13, 2018, 01:54:01 pm
PS: Meh, this thread started with talk about OpenVPN... Are you using OpenVPN to push LAN traffic elsewhere?
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 13, 2018, 02:01:19 pm
Only OpenVPN, which i use to connect with clients from my workplace and phone. I'm pushing all the traffic through the tunnel on the clients, but on the server it's a standard setup with "redirect gateway" enabled...

Not using ddns as my ISP provides this service for me.

Do you want me to open another thread for the pf restart issue and post the diff there, not to mix the thread with other stuff? The OpenVPN issue was fixed anyway...
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: franco on January 13, 2018, 02:19:01 pm
It's ok to continue in here. It's related to OpenVPN boot ordering somehow. :)
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 13, 2018, 02:20:40 pm
I sent you the diff in a PM, it has some data i would not like to be made public...
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: franco on January 13, 2018, 02:50:46 pm
Thanks, it does not look like it's problematic in the rules, but in any case try this patch, I was looking in the wrong place as bootup is different in behaviour...

https://github.com/opnsense/core/commit/27fe55f


Thank you for your help :)

Franco
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 13, 2018, 03:14:44 pm
Rebooted the box twice after applying the patch (1-console, 2-GUI)
First time it worked, the second it didn't...

Sure Franco, any time.
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: franco on January 13, 2018, 03:27:53 pm
Annoying, can you send over the first page of the system logs from after reboot when this works and when it doesn't?

I don't think the rules are different now at all, which either indicates a route issue or something that happens / does not happen after boot.

There are more changes on the development branch in the git repo, but patching them manually on top is getting tricky.


Cheers,
Franco
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 13, 2018, 04:24:53 pm
I have sent you a wetransfer download link in your PM with both.

Thanks Franco!
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 13, 2018, 04:26:08 pm
Can i delete this exact reply? Accidentally quoted myself :) Anyway, is the verbosity level ok?
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: franco on January 14, 2018, 10:01:31 am
Ok, so I think there is a race happening with PPPoE startup, because... you are running a lot of plugins / additional services.

I will have to ask you to update your configuration to master to make sure to try the full rework:

# opnsense-code core
# cd /usr/core
# make upgrade CORE_ABI=18.1 CORE_NAME=opnsense

The changes against 18.1.r1 are minimal at the moment.

I've moved the service config generation out of the actual boot sequence and added messages for when the PPPoE startup is ignored because we are still booting. That should:

(a) make it work more reliably, but still not perfect, but
(b) will let us hopefully confirm that the ignored PPPoE reconfigure is the issue here.


Cheers,
Franco
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 14, 2018, 10:44:32 am
Hi Franco,

In the meantime i've switched back to devel and updated to 18.1.r15.
Same issue though... I also needed to apply the OpenVPN patch as well.

I just tried to update to master, but:

Code: [Select]
root@gateway:~ # opnsense-code core
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
The following 3 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        git: 2.15.1
        p5-Error: 0.17025
        cvsps: 2.1_2

Number of packages to be installed: 3

The process will require 27 MiB more space.
4 MiB to be downloaded.
[1/3] Fetching git-2.15.1.txz: 100%    4 MiB   4.5MB/s    00:01
[2/3] Fetching p5-Error-0.17025.txz: 100%   19 KiB  19.3kB/s    00:01
[3/3] Fetching cvsps-2.1_2.txz: 100%   41 KiB  41.6kB/s    00:01
Checking integrity... done (0 conflicting)
[1/3] Installing p5-Error-0.17025...
[1/3] Extracting p5-Error-0.17025: 100%
[2/3] Installing cvsps-2.1_2...
[2/3] Extracting cvsps-2.1_2: 100%
[3/3] Installing git-2.15.1...
===> Creating groups.
Creating group 'git_daemon' with gid '964'.
===> Creating users
Creating user 'git_daemon' with uid '964'.
Extracting git-2.15.1: 100%
Message from cvsps-2.1_2:

===>   NOTICE:

The cvsps port currently does not have a maintainer. As a result, it is
more likely to have unresolved issues, not be up-to-date, or even be removed in
the future. To volunteer to maintain this port, please create an issue at:

https://bugs.freebsd.org/bugzilla

More information about port maintainership is available at:

https://www.freebsd.org/doc/en/articles/contributing/ports-contributing.html#maintain-port
Message from git-2.15.1:

------------------------------------------------------------------------
*************************** GITWEB *************************************
If you installed the GITWEB option please follow these instructions:

In the directory /usr/local/share/examples/git/gitweb you can find all files to
make gitweb work as a public repository on the web.

All you have to do to make gitweb work is:
1) Copy the files /usr/local/share/examples/git/gitweb/* to a directory on
   your web server (e.g. Apache2) in which you are able to execute
   CGI-scripts.
2) In gitweb.cgi, adjust the variable $projectroot to point to
   your git repository (that is where you have your *.git project
   directories).
*************************** GITWEB *************************************

*************************** CONTRIB ************************************
If you installed the CONTRIB option please note that the scripts are
installed in /usr/local/share/git-core/contrib. Some of them require
other ports to be installed (perl, python, etc), which you may need to
install manually.
*************************** CONTRIB ************************************
------------------------------------------------------------------------
Cloning into '/usr/core'...
remote: Counting objects: 93291, done.
remote: Compressing objects: 100% (36/36), done.
remote: Total 93291 (delta 15), reused 28 (delta 12), pack-reused 93242
Receiving objects: 100% (93291/93291), 56.53 MiB | 4.96 MiB/s, done.
Resolving deltas: 100% (66667/66667), done.
root@gateway:~ # cd /usr/core
root@gateway:/usr/core # make upgrade CORE_ABI=18.1 CORE_NAME=opnsense
pkg: No package(s) matching opnsense
>>> Cannot find package.  Please run 'opnsense-update -t opnsense'
*** Error code 1

Stop.
make: stopped in /usr/core


Think i did something stupid.. i'm not a newbie as a user of freebsd firewalls, i'm a total newbie in freebsd though, i never did go deeper than the GUI up until now :-) Sorry for the headaches.
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 14, 2018, 10:50:11 am
Should i run 'opnsense-update -t opnsense' and then 'make upgrade CORE_ABI=18.1 CORE_NAME=opnsense' ?
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 14, 2018, 11:12:16 am
Ok, the switch to master worked. Let me try to reproduce the issue.
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 14, 2018, 11:25:55 am
I think it worked Franco! I am unable to reproduce the issue!

Awesome job! :)

Can i switch back to LibreSSL? I think i can't just yet :)

Personally, i have one more remaining issue, maybe you could also shed some light there:
https://forum.opnsense.org/index.php?topic=6855.0


Many thanks!
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: franco on January 14, 2018, 01:29:12 pm
Ok, it's not perfect yet but definitely better. If a reboot should ever not work please let me know coupled with the system log as there is a telling debug message now saying "IP renewal ignored during boot".

I assumed you were on 18.1.r1, that's why CORE_NAME had to be modified for master.

Installing the opnsense-devel package will always take you back to 18.1.r_15 as that is the fixed version on the mirror.

To wrap up, all this will make it into 18.1.r2 (RC2). :)

Let me follow up on the other issue later. It's probably not our code....


Cheers,
Franco
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: franco on January 14, 2018, 01:29:47 pm
And LibreSSL will be back when 18.1 is out which comes after 18.1.r2.
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 14, 2018, 02:13:51 pm
Sure thing Franco, whenever you feel like it.
Well, so far so good, 5+ reboots, no failures as of yet. I'll stick with this version until r2, unless you want me to try out something else.
I'll build a vm as well later, but that will not help as much with the pppoe issue, as access to my rack is a bit difficult (to change cables and stuff).

Thanks again for everything!
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 15, 2018, 08:35:02 am
Hi Franco,

Unfortunately the pf restart issue happened again. I'm on my way to a client so logs later, also i got a feeling that i accidentally updated to r1. I'll report back once i'm done with the client.

Thanks
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 15, 2018, 10:12:35 am
Can't collect the logs, I forgot to apply the aliases patch, can't connect to my box (remotely).
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: franco on January 15, 2018, 12:36:33 pm
18.1.r2 will be out tonight... that should make the patching unnecessary.


Cheers,
Franco
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 15, 2018, 05:38:56 pm
Just upgraded to 18.1.r2  ;D

Didn't need to restart pf, OpenVPN working, but aliases still don't work, unfortunately.. There's no /var/db/aliastables until manually refreshed from the console..

Once manually refreshed, even though i had a cron job to refresh them every minute, they still didn't, i had to manually refresh again.
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: franco on January 15, 2018, 06:19:53 pm
Ok, progress is progress. :)

Can you provide the output for:

# df -h

Aliases are still on my list.


Thanks,
Franco
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 15, 2018, 06:39:29 pm
Here you go:

Filesystem         Size    Used   Avail Capacity  Mounted on
/dev/gpt/rootfs     38G    7.2G     28G    21%    /
devfs              1.0K    1.0K      0B   100%    /dev
tmpfs               22G    194M     22G     1%    /var
tmpfs               22G    360K     22G     0%    /tmp
devfs              1.0K    1.0K      0B   100%    /var/dhcpd/dev
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: franco on January 15, 2018, 06:46:58 pm
Aha, /var MFS clears /var/db/aliastables for sure.

Ok, I don't know why Ad changed the behaviour but let's move the alias database to the persistent location...

https://github.com/opnsense/core/commit/6536510

When you apply this patch the next reboot won't work again and you can fix it manually, after that the reboots should not interrupt operation anymore.


Cheers,
Franco
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 15, 2018, 07:14:30 pm
Franco, i think it worked! Very well done!!  :)

Thank you!
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: franco on January 15, 2018, 07:24:22 pm
Okay, thank you for confirming. :)

The initial migration could still be tricky coming from 17.7... This will probably be subject to change once we move into 18.1.x or it must be documented in the 18.1 migration notes.


Cheers,
Franco
Title: Re: [SOLVED] OpenVPN will not start on 18.1.r1
Post by: elektroinside on January 15, 2018, 07:35:25 pm
Of course, any time.

Yes, well documentation is important, of course, but testing these particular changes i guess will require just a few vms and some time. I'll build a few vms myself very soon, so i will be able to test the upgrade/update, and i'll report if i find something.