Let's Encrypt: Automation: SCP fails

[abulafia]

My automation for copying the Let's Encrypt certificate to my local ESXI server fails:

1. automation task set up and "test connection" claims everything is fine: Connection and upload test succeeded.

Naming "cert.pem" --> rui.crt
Naming "key.pem" --> rui.key
Naming "ca.pem" --> ca.pem
Naming "fullchain.pem" --> castore.pem

2. Manually SCP'ing the files to ESXi works. ESXi finds and uses the copied certificate:

Code Select Expand

  # scp ./cert.pem root@esxi.XXX.de:/etc/vmware/ssl/rui.crt
  # scp ...

(for the avoidance of doubt, no password needs to be entered when doing scp, i.e. key authentication has been set up and is working)

3. Running automation from the "certificate" submenu fails to copy the certificate and yields the following errors in the logfile:

Code Select Expand

opnsense[82577] /usr/local/opnsense/scripts/OPNsense/AcmeClient/upload_sftp.php: Command execution failed, exit code 2. Last input was: {"host":"esxi.XXX.de","host-key":"","port":"22","identity-type":"rsa","user":"root","remote-path":"/etc/vmware/ssl","chgrp":"","chmod":"","chmod-key":"","cert-name":"rui.crt","key-name":"rui.key","ca-name":"ca.pem","fullchain-name":"castore.pem","certificates":"xxx","automation-id":"xxx"}
opnsense[82577] /usr/local/opnsense/scripts/OPNsense/AcmeClient/upload_sftp.php: Failed on {"source":"/tmp/sftp-upload-3UmGMx","target":"ca.pem","mode":"0440","group":false,"delete_source":true}
opnsense[82577] /usr/local/opnsense/scripts/OPNsense/AcmeClient/upload_sftp.php: Failed uploading file '/tmp/sftp-upload-3UmGMx' to 'ca.pem' ; Cause: {"file_not_found":true,"error":"Couldn't fsetstat: No such file or directory"}
opnsense[69190] AcmeClient: running automation: uploadESXI
opnsense[69190] AcmeClient: running automation: restartGUI
opnsense[69190] AcmeClient: running automations for certificate: *.xxx.de


Strangely, ca.pem is the only file that is actually copied over to ESXi when running this ...!

Is this an error in my setup, or is upload_sftp.php broken?

[kristerrenaud]

Keep in mind that when using ssh/scp for the first time you need to accept the host key. This often leads to problems with automations run with non-login users since there is no person that can accept the host key.

If this is your issue, there are several solutions, here are two:

1. Run "yes | scp ...." in the script file once, then remove "yes |".
2. Copy the entry from your user's ~/.ssh/known_hosts to the automation user's known_hosts file.

[abulafia]

Unfortunately (1) that is not my issue and (2) the error persists (on 21.7.2).

The key pair and scp/sftp work.

I see "ca.pem" being copied over, and the upload test manages to put the test file on the sftp server:

[root@esxi:/etc/vmware/ssl] ls -la
[...]
-rw-------    1 root     root            11 Sep 11 20:40 sftp-upload-4Q85Il
-rw-------    1 root     root            11 Sep 11 20:40 sftp-upload-HC3pkO
-rw-------    1 root     root            11 Sep 11 20:39 sftp-upload-Q5ZeG2

[root@esxi:/etc/vmware/ssl] cat sftp-upload-Q5ZeG2
upload-test

The issue seems to be that the upload script fails to execute the proper command on the sftp server (ESXi) / that ESXi does not support fsetstat:

2021-09-11T22:23:26   opnsense[84570]   /usr/local/opnsense/scripts/OPNsense/AcmeClient/upload_sftp.php: Failed uploading file '/tmp/sftp-upload-YzGRyN' to 'ca.pem' ; Cause: {"file_not_found":true,"error":"Couldn't fsetstat: No such file or directory"}

How can I get the ACME client to execute a a short script to scp the files over? i.e. how do I change the "Run Command" of the ACME client from the limited pre-defined actions to executing a local script?

[yoomoo]

Hi,
did you find any solution for this problem?

[abulafia]

Unfortunately, no - would be great to get this to work somehow!