OPNsense Forum

Archive => 18.1 Legacy Series => Topic started by: qinohe on June 24, 2018, 04:58:16 pm

Title: [solved] Nexcloud: communication failure
Post by: qinohe on June 24, 2018, 04:58:16 pm
Hello all,

Trying the Nextcloud backup, but I seem to bump into something and keep getting
Code: [Select]
The following input errors were detected:

    communication failure
   
after hitting 'Setup/TestNextcloud'
   
My Nextcloud server is Debian specs are:
4.9.0-6-amd64
nextcloud 13.0.4
a separate user for Opnsense backups, TOTP enabled(which shouldn't matter since app id is used)
a app password for Opnsense
Nextcloud machine firewall set to allow Opnsense

The same method is used for an android phone a mediaplayer and ArchLinux on a different account, they seem to work fine.

Logs contain no info about this or I'm looking for the wrong ones.
Live view shows all connections are allowed.
Tried with 2fa and without (on Nextcloud) and logout to activate the change, the failure is the same.

Thanks mark

edit: one thing I forgot to mention may be important or not !?
I'm NOT running my Nexcloud server on:
nextcloud.server.domain
Instead I have it on:
server.domain/nextcloud
Title: Re: Nexcloud: communication failure
Post by: frank_p on June 24, 2018, 05:28:35 pm
I have the same result.
Running nextcloud on a virtual private server in my domain.
I am sure my credentials are correct ;)
Title: Re: Nexcloud: communication failure
Post by: fabian on June 24, 2018, 05:43:07 pm
Do you use TLS with an untrusted certificate?
What do the logs say (OPNsense logs)?
Is

Trying the Nextcloud backup, but I seem to bump into something and keep getting
Code: [Select]
The following input errors were detected:

    communication failure
   
after hitting 'Setup/TestNextcloud'
The error means that the backup is not successful. The real information is in the logs.

My Nextcloud server is Debian specs are:
4.9.0-6-amd64
nextcloud 13.0.4
I developed the code with a nextcloud 13.0.2 server running a GNU/Linux distribution - so this will probably work.
a separate user for Opnsense backups, TOTP enabled(which shouldn't matter since app id is used)
a app password for Opnsense
Nextcloud machine firewall set to allow Opnsense

The same method is used for an android phone a mediaplayer and ArchLinux on a different account, they seem to work fine.
An app password should not be affected by TOTP etc. since they are more like tokens.

Logs contain no info about this or I'm looking for the wrong ones.
Live view shows all connections are allowed.
Tried with 2fa and without (on Nextcloud) and logout to activate the change, the failure is the same.

Thanks mark

App passwords should always work. The log you should check is the syslog.

edit: one thing I forgot to mention may be important or not !?
I'm NOT running my Nexcloud server on:
nextcloud.server.domain
Instead I have it on:
server.domain/nextcloud

forgot http(s):// in front? /remote.php… is appended automatically.
Title: Re: Nexcloud: communication failure
Post by: qinohe on June 24, 2018, 06:11:58 pm
Hey fabian, thanks for the clear answer.

Your first Q. :yes using self signed cert. for my server, all is a localdomain.

Next: what say the logs:
Code: [Select]
config[23141]: {"url":"https:\/\/cloud.localdomain\/nextcloud\/remote.php\/dav\/files\/backer\/","content_type":null,"http_code":0,"header_size":0,"request_size":0,"filetime":-1,"ssl_verify_result":18,"redirect_count":0,"total_time":0.164199,"namelookup_time":0.004971,"connect_time":0.005542,"pretransfer_time":0,"size_upload":0,"size_download":0,"speed_download":0,"speed_upload":0,"download_content_length":-1,"upload_content_length":-1,"starttransfer_time":0,"redirect_time":0,"redirect_url":"","primary_ip":"10.10.100.6","certinfo":[],"primary_port":443,"local_ip":"10.10.100.1","local_port":42812}
Than: no hehe I did not forget the 's'  :P  I click on it from another webpage I don't know what I was thinking here, I don't do that, just the address

I allraedy 'knew' app password should be okay with 2fa but still tested I, wanted to be sure that was not an issue when I post here, thanks.
Title: Re: Nexcloud: communication failure
Post by: fabian on June 24, 2018, 06:40:18 pm
ssl_verify_result":18 means TLS certificate verify issue -> OPNsense does not trust your certificate and rejects the connection.
Here is a user with a simmilar issue (using the CACert CA: https://github.com/opnsense/core/pull/2289#issuecomment-399716802) (https://github.com/opnsense/core/pull/2289#issuecomment-399716802)
Title: Re: Nexcloud: communication failure
Post by: qinohe on June 24, 2018, 07:25:59 pm
Hey fabian, yes that works thanks for the answer.
Now the Nextcloud server is reached trough a webpage served by that same server and thus in the HTTP_REFERER checks,
really I would not have found for a long time,  problem solved for now  8)

Thanks mark

edit: thanks for this app fabian it works well!
Now I still have a question:
How or when is a backup triggered? is it as soon when changes are made, hmm. just tried that seems to not be the case.
NVM. it's in the wiki   :-[
Title: Re: Nexcloud: communication failure
Post by: akron on June 26, 2018, 07:44:34 pm
Hey fabian, thanks for the clear answer.

Your first Q. :yes using self signed cert. for my server, all is a localdomain.

Next: what say the logs:
Code: [Select]
config[23141]: {"url":"https:\/\/cloud.localdomain\/nextcloud\/remote.php\/dav\/files\/backer\/","content_type":null,"http_code":0,"header_size":0,"request_size":0,"filetime":-1,"ssl_verify_result":18,"redirect_count":0,"total_time":0.164199,"namelookup_time":0.004971,"connect_time":0.005542,"pretransfer_time":0,"size_upload":0,"size_download":0,"speed_download":0,"speed_upload":0,"download_content_length":-1,"upload_content_length":-1,"starttransfer_time":0,"redirect_time":0,"redirect_url":"","primary_ip":"10.10.100.6","certinfo":[],"primary_port":443,"local_ip":"10.10.100.1","local_port":42812}
Than: no hehe I did not forget the 's'  :P  I click on it from another webpage I don't know what I was thinking here, I don't do that, just the address

I allraedy 'knew' app password should be okay with 2fa but still tested I, wanted to be sure that was not an issue when I post here, thanks.

Hello,

I have a similar issue, with number 20

is there any fix ?

config[80861]: {"url":"https:\/\/cloud.domain.com\/\/remote.php\/dav\/files\/opnsense\/","content_type":null,"http_code":0,"header_size":0,"request_size":0,"filetime":-1,"ssl_verify_result":20,"redirect_count":0,"total_time":0.033315,"namelookup_time":4.9e-5,"connect_time":0.007027,"pretransfer_time":0,"size_upload":0,"size_download":0,"speed_download":0,"speed_upload":0,"download_content_length":-1,"upload_content_length":-1,"starttransfer_time":0,"redirect_time":0,"redirect_url":"","primary_ip":"192.168.1.5","certinfo":[],"primary_port":443,"local_ip":"192.168.1.1","local_port":32217}


Thank you
Title: Re: [solved] Nexcloud: communication failure
Post by: fabian on June 26, 2018, 10:40:09 pm
Very likely another certificate validation error. I would check host name and time range of the certificate but I don't know this code.
Title: Re: [solved] Nexcloud: communication failure
Post by: akron on June 27, 2018, 12:05:27 pm
Very likely another certificate validation error. I would check host name and time range of the certificate but I don't know this code.

I have put the CA certificate on the path mentioned and now I get another error:

config[29464]: {"url":"https:\/\/cloud.domain.com\/remote.php\/dav\/files\/opnsense\/","content_type":null,"http_code":0,"header_size":0,"request_size":0,"filetime":-1,"ssl_verify_result":1,"redirect_count":0,"total_time":0.25138,"namelookup_time":0.078396,"connect_time":0.086301,"pretransfer_time":0,"size_upload":0,"size_download":0,"speed_download":0,"speed_upload":0,"download_content_length":-1,"upload_content_length":-1,"starttransfer_time":0,"redirect_time":0,"redirect_url":"","primary_ip":"192.168.1.5","certinfo":[],"primary_port":443,"local_ip":"192.168.1.1","local_port":42651}

would be easier to implement on the webui ignore SSL certificate validation ? that would be perfect as we could use any self signed SSL

thank you
Title: Re: [solved] Nexcloud: communication failure
Post by: qinohe on June 27, 2018, 01:33:37 pm
Hey akron, what @fabian says, and a question: did you check the certificate with another connection, did that work?
My guess, there's something wrong with the crt.
Btw. I simply pushed the crt. to the store and that was it, no CA (need to set that up in spare time, heck, I may use Opnsense for that  :P)

Greetings mark
Title: Re: [solved] Nexcloud: communication failure
Post by: akron on June 27, 2018, 02:52:54 pm
Hey akron, what @fabian says, and a question: did you check the certificate with another connection, did that work?
My guess, there's something wrong with the crt.
Btw. I simply pushed the crt. to the store and that was it, no CA (need to set that up in spare time, heck, I may use Opnsense for that  :P)

Greetings mark

I am confuse, I am getting another error now ssl_verify_result":20

what I did was to put the SSL crt /usr/local/share/certs the crt contains the certificate data followed by private key, is this correct? also I tried to put in pem format no change
Title: Re: [solved] Nexcloud: communication failure
Post by: qinohe on June 27, 2018, 03:23:22 pm
Hey akron, what @fabian says, and a question: did you check the certificate with another connection, did that work?
My guess, there's something wrong with the crt.
Btw. I simply pushed the crt. to the store and that was it, no CA (need to set that up in spare time, heck, I may use Opnsense for that  :P)

Greetings mark

I am confuse, I am getting another error now ssl_verify_result":20

Not an authority on this matter but a quote from https://www.openssl.org/docs/man1.0.2/apps/verify.html (https://www.openssl.org/docs/man1.0.2/apps/verify.html) :
Quote
20 X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY: unable to get local issuer certificate

    the issuer certificate could not be found: this occurs if the issuer certificate of an untrusted certificate cannot be found.
^^ probably a wrong value, check your CRT  ;).
Quote
what I did was to put the SSL crt /usr/local/share/certs the crt contains the certificate data followed by private key, is this correct? also I tried to put in pem format no change

I only put theCRT. in the store no keys no nothing,just the CRT.
I guess U understand that this is completely outside the scope of Opnsense  ;D
Title: Re: [solved] Nexcloud: communication failure
Post by: qinohe on July 02, 2018, 04:58:44 pm
I don't know if it would be redundant but If there is interest for it I could make a little guide based  on the use of self signed CRT's with the help of the OPNsense and put it in Tutorials, there is no entry in the wiki, yet.

Greetings mark
Title: Re: [solved] Nexcloud: communication failure
Post by: fabian on July 02, 2018, 07:22:24 pm
Documentation is always good - The docs repository is here: https://github.com/opnsense/docs

Don't forget to add a warning about the issue, that an update of the system cert bundle may undo the change.
Title: Re: [solved] Nexcloud: communication failure
Post by: qinohe on July 02, 2018, 07:59:12 pm
Nice to know, I'll start with  a text based guide I put in https://forum.opnsense.org/index.php?board=24.0 (https://forum.opnsense.org/index.php?board=24.0)
Users can give their experience/findings and I create a wiki page. I already have a guide running on my Mediawiki server, but it's not ready for distribution.

Yea, if the store is updated your input is gone, it just happened a few hours ago, but I was prepared  8), I will add the warning..
Title: Re: [solved] Nexcloud: communication failure
Post by: Alwaysin on October 07, 2018, 09:36:59 pm
Hello,

I also have trouble creating backups on Nextcloud. Maybe it's due to 14.x version?

Code: [Select]
×Close
The following input errors were detected:

communication failure

Relevant log files:

Code: [Select]
Oct 7 21:26:45 config[4588]: {"url":"https:\/\/blabla.blabla.com\/remote.php\/dav\/files\/blabla\/","content_type":"application\/xml; charset=utf-8","http_code":404,"header_size":1229,"request_size":192,"filetime":-1,"ssl_verify_result":0,"redirect_count":0,"total_time":0.638179,"namelookup_time":3.8e-5,"connect_time":0.002203,"pretransfer_time":0.019544,"size_upload":0,"size_download":228,"speed_download":357,"speed_upload":0,"download_content_length":-1,"upload_content_length":-1,"starttransfer_time":0.638121,"redirect_time":0,"redirect_url":"","primary_ip":"<removed IP>","certinfo":[],"primary_port":443,"local_ip":"<removed ip>","local_port":45690}
Oct 7 21:26:45 config[4588]: Error while fetching filelist from Nextcloud
Title: Re: [solved] Nexcloud: communication failure
Post by: fabian on October 07, 2018, 10:09:40 pm
No, I've just upgraded mine to v14 and it still works. From your log I think your path is wrong (maybe a subdirectory instead of root?)
Title: Re: [solved] Nexcloud: communication failure
Post by: Alwaysin on October 09, 2018, 05:35:37 pm
No, I've just upgraded mine to v14 and it still works. From your log I think your path is wrong (maybe a subdirectory instead of root?)

Yes I have to specify a subdirectory, which I've created on my user space.

Title: Re: [solved] Nexcloud: communication failure
Post by: qinohe on October 09, 2018, 07:43:29 pm
Hi Alwaysin, what do mean with 'have to specify a sub-directory'?

In my guide you don't, just the name and path to server.

Greetings mark
Title: Re: [solved] Nexcloud: communication failure
Post by: Alwaysin on October 09, 2018, 10:19:53 pm
Hi Qinohe, sorry for the wrong terminology, I meant what is called the "Backup Directory", which is a subdirectory of the main directory that is the name of the user.

I've followed exactly the steps as described here https://github.com/opnsense/docs/blob/master/source/manual/how-tos/cloud_backup.rst#setup-nextcloud-api-usage but I'm still facing this communication failure :(

Does it matter if the user already exists and is used not only for OPNsense backup? Or it the user is an LDAP account?
Title: Re: [solved] Nexcloud: communication failure
Post by: fabian on October 09, 2018, 10:47:12 pm
Does it matter if the user already exists and is used not only for OPNsense backup? Or it the user is an LDAP account?

The authentication mechanism is probably not relevant but the user must exist and can be used for other things as well (you may generate an application token for any app you are using, but I recommend a separate backup user with the directory read only shared for users who need it)
Title: Re: [solved] Nexcloud: communication failure
Post by: qinohe on October 09, 2018, 11:09:09 pm
Well, the backup directory is only a name in the form and points to nowhere, at least for you and me, this is done by OPNsense, only set a name like opnsense-backup

Than suppose the nextcloud server is at https://some.domain/nextcloud, than that's what you use as URL address

the form would be:
Code: [Select]
URL                      https://some.domain/nextcloud
User Name           the user created on nextcloud to do the backup
Password              app password created in users backup account
Directory Name    opnsense-backup
Hope that helps, it should work at least here it does and @fabian already said his was working  ;)

But the same username itself can be used on various machines on the network, however, I would choose a single used user for that purpose btw. like @fabian says or did you mean something different?

Greetings, mark
Title: Re: [solved] Nexcloud: communication failure
Post by: Alwaysin on October 10, 2018, 09:32:07 am
Thank you all for for putting so much efforts into helping me, much appreciated :)

So I retried everything to make sure:

- created a dedicated local user "opnsense-backup" on my nextcloud
- logged in as said user, created an app password

- went to OPNsense, checked "enable", user name as previously created user, password as given to me by nextcloud, directory name opnsense-backup

And it works!

But I retried again with my LDAP account and it does not work. I think it is because Nextcloud gives the user a random set of character when you bind it to LDAP such as "A1F54823-801A-4R3D-A2C3-B93657CE5310", which appears in the username column but still you have to login with the login of LDAP. And the folder that has to be accessed with webdav is the random-characters string, not the LDAP fancy username.
Title: Re: [solved] Nexcloud: communication failure
Post by: qinohe on October 10, 2018, 02:59:31 pm
Hi Alwaysin, glad it works, now how to deal with LDAP I don't know, I understand it's working but I never used it.

https://wiki.opnsense.org/manual/how-tos/user-ldap.html

You probably already went trough that page, but this is as far as my help can stretch  :P

Greetings, mark
Title: Re: [solved] Nexcloud: communication failure
Post by: Alwaysin on October 10, 2018, 03:39:36 pm
I meant LDAP on Nextcloud side !
The backup feature of OPNsense to Nextcloud doesn't know how to deal with LDAP accounts on Nextcloud, but this is not a problem, I've created a local user on Nextcloud.
Title: Re: [solved] Nexcloud: communication failurehttps://forum.opnsense.org/index.php
Post by: qinohe on October 10, 2018, 04:15:12 pm
There are no other processes involved I think it's the best and maybe even the most secure way to set it up like you did now.

Btw. in post #20 @fabian says the mechanism is probably not relevant, so I guess there should be a way to get the LDAP working!?  :D
Maybe someone with more knowledge about this should answer that.

Greetings, mark