Messages

I run my home connection off a Topton n100 router box. Opnsense is virtualised in proxmox with 2 of the 4 nics pci-passed-through. It works really well and is certainly able to handle my 1gbps symmetric fibre connection with ease. Peaks at about 15% CPU usage when it's saturating the line and hovers the 4% mark during idle / regular home usage. I do not use any kind of IDS/IPS though.

Aha, got to the bottom of it. Proposed a fix here https://github.com/opnsense/plugins/pull/3745

So the commands are actually succeeding. But the issue goes deeper. I'm trying to get acme validation for gcloud to work, and gcloud commands run from within acme run from opnsense are also returning code 120, whereas on a regular shell they're returning 0.

I'm not sure what's going on at this point, this is potentially a weird interaction between the PHP code running the exec command

I have one of the newer topton n100 boxes. So far I couldn't be happier. I have it virtualised within proxmox with PCI passthrough to 2 of the 4 2.5gb intel nics and it has been really solid and able to handle my gigabit connection without issue. The CPU runs at 44C idling at ~6% CPU usage and the chassis is hot to the touch. This is by design as it's passively cooled.

So I've managed to find the php code that ran the gcloud commands, which I've tweaked like so:

Code Select Expand


LeUtils::run_shell_command("/usr/local/bin/gcloud --verbosity=debug config configurations list 2>&1 >> /tmp/deleteme.log", $proc_env);
        LeUtils::run_shell_command("/usr/local/bin/gcloud --verbosity=debug config configurations create ${gcloud_config} 2>&1 >> /tmp/deleteme.log", $proc_env);
        LeUtils::run_shell_command("/usr/local/bin/gcloud --verbosity=debug config configurations activate ${gcloud_config} 2>&1 >> /tmp/deleteme.log", $proc_env);
        LeUtils::run_shell_command("/usr/local/bin/gcloud --verbosity=debug auth activate-service-account --key-file=${gcloud_key_file} 2>&1 >> /tmp/deleteme.log", $proc_env);
        LeUtils::run_shell_command("/usr/local/bin/gcloud --verbosity=debug config set account ${gcloud_account} 2>&1 >> /tmp/deleteme.log", $proc_env);
        LeUtils::run_shell_command("/usr/local/bin/gcloud --verbosity=debug config set project ${gcloud_project} 2>&1 >> /tmp/deleteme.log", $proc_env);


The only output I get is that of `gcloud --verbosity=debug config configurations list`, which exits without error. All the other commands exit with code 120. I believe the issue is coming from python itself.

From https://bugs.python.org/issue29130:

David is right. The 120 code was added in Issue 5319, as a way of indicating a problem in the final stages of the interpreter exiting. The two conditions that trigger this are calling the flush() method on sys.stdout and sys.stderr. If you add a dummy flush() implementation, it no longer exits with 120:

If I run the commands directly on the shell they all succeed, so perhaps there's an issue with the way the PHP code is shell_exec'ing the gcloud commands

I'm setting up Let's Encrypt certs via the ACME plugin google cloud DNS. google cloud sdk and acme plugins both are installed from the plugins page.

When I manually try to renew the cert, it fails with the following on the logs (anonymised)

Code Select Expand


2023-10-06T10:21:21 Notice opnsense AcmeClient: certificate must be issued/renewed: router.my.cert
2023-10-06T10:21:21 Notice opnsense AcmeClient: issue certificate: router.my.cert
2023-10-06T10:21:21 Notice opnsense AcmeClient: using CA: letsencrypt
2023-10-06T10:21:21 Notice opnsense AcmeClient: account is registered: Let's Encrypt
2023-10-06T10:21:21 Notice opnsense AcmeClient: Google Cloud DNS project name: my-project-infrastructure
2023-10-06T10:21:22 Error opnsense /usr/local/opnsense/scripts/OPNsense/AcmeClient/lecert.php: AcmeClient: The shell command '/usr/local/bin/gcloud --quiet config configurations create acme-651fd0c54d1f40-20765247' returned exit code '120'
2023-10-06T10:21:23 Error opnsense /usr/local/opnsense/scripts/OPNsense/AcmeClient/lecert.php: AcmeClient: The shell command '/usr/local/bin/gcloud --quiet config configurations activate acme-651fd0c54d1f40-20765247' returned exit code '120'
2023-10-06T10:21:24 Error opnsense /usr/local/opnsense/scripts/OPNsense/AcmeClient/lecert.php: AcmeClient: The shell command '/usr/local/bin/gcloud --quiet auth activate-service-account --key-file=/tmp/acme_dns_gcloud_651fd0c54d1f40-20765247.json' returned exit code '120'
2023-10-06T10:21:24 Error opnsense /usr/local/opnsense/scripts/OPNsense/AcmeClient/lecert.php: AcmeClient: The shell command '/usr/local/bin/gcloud --quiet config set account terraform@my-project-infrastructure.iam.gserviceaccount.com' returned exit code '120'
2023-10-06T10:21:26 Error opnsense /usr/local/opnsense/scripts/OPNsense/AcmeClient/lecert.php: AcmeClient: The shell command '/usr/local/bin/gcloud --quiet config set project my-project-infrastructure' returned exit code '120'
2023-10-06T10:21:26 Notice opnsense AcmeClient: using challenge type: Google Cloud DNS - my.cert
2023-10-06T10:21:32 Error opnsense AcmeClient: domain validation failed (dns01)
2023-10-06T10:21:32 Error opnsense AcmeClient: validation for certificate failed: router.my.cert


Now since it's using --quiet I can't see what the problem is. If I go on to a root shell I can see though that the cloud configuration not only exists, but it also works (eg I can list my gcloud storage buckets).

If I delete the gcloud configuration and try again to renew, same thing happens, and the configuration also gets re-created.

Is there any way to at least remove the `--quiet` from those commands to see what's up?