Messages

oddly, today the file is a different size (smaller) without doing a delete.  (the unbound logging was off / has been off)

root@xyz:/var/cache/unbound.duckdb # ls -lh
total 2107724
-rw-r--r--  1 unbound unbound  1.4K May  8 08:44 client.csv
-rw-r--r--  1 unbound unbound  214B May  8 08:45 load.sql
-rw-r--r--  1 unbound unbound  2.0G May  8 08:45 query.csv
-rw-r--r--  1 unbound unbound  1.5K May  8 08:45 schema.sql

not sure what is going on here

I did that, but the /var/cache/unbound.duckdb directory is still there.  ok to delete it?

This directory seems to be taking up a lot of disk space.  13G for the query.csv file.

1.  Can I delete it?
2.  How can I stop it from being rebuilt & growing?

root@xyz:/var/cache/unbound.duckdb # ls -lh
total 13917196
-rw-r--r--  1 unbound unbound  1.9K May  6 09:31 client.csv
-rw-r--r--  1 unbound unbound  214B May  6 09:32 load.sql
-rw-r--r--  1 unbound unbound   13G May  6 09:32 query.csv
-rw-r--r--  1 unbound unbound  1.5K May  6 09:32 schema.sql

Thanks, yes, that's what I needed.  There was an old, expired certificate in the list for some reason.  I deleted it and now the list in haproxy looks fine.

I hadn't realized that was where the certificate list lived!

Thank you!

I was experiencing this issue, but the hotfix seems to have addressed it, but I now see another problem related to it.

I'm now seeing a duplicate certificate for one domain in the HAProxy Public Service Certificates, even though there is only 1 certificate for that domain in the ACME plugin list. 

I've re-run the automation from the ACME service several times, but I'm still seeing two certificates in HAProxy when there should be only one.  Since the name is the same for both, I don't know which is correct.

ok, yes, that worked.

Adding "none" to that config, converted the ALERT into a WARNING and the config check passes on startup, so the haproxy service will startup automatically on reboots.

I did also have to change the log host to it's IP address because that 'none' setting is only relevant for backend servers, not log hosts.  But, that's fine.

Really appreciate all your help!  And, I hope this thread helps someone else with the same issue.

Final thought -- it'd be nice if there was a way to control the ordering of rules in the haproxy configs to eliminate the

Code Select Expand

"[WARNING]  (55248) : parsing [/usr/local/etc/haproxy.conf:112] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before." warnings completely. 

Feature request?

Ok, that now shows the problem, and not resolving a hostname is causing an alert, which then I guess, fails the startup. 

Here's the output of the logfile:

Code Select Expand


[WARNING]  (23218) : parsing [/usr/local/etc/haproxy.conf:114] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.
[WARNING]  (23218) : parsing [/usr/local/etc/haproxy.conf:116] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.
[NOTICE]   (23218) : haproxy version is 2.4.19-ec55434
[NOTICE]   (23218) : path to executable is /usr/local/sbin/haproxy
[ALERT]    (23218) : parsing [/usr/local/etc/haproxy.conf:183] : 'server plex' : could not resolve address 'host1.localdomain'.
[ALERT]    (23218) : Failed to initialize server(s) addr.


I tried rebooting a few times and either 1 or 2 different hosts would show up as alerts, so I assume if I correct this one by changing to an IP address, it will just trigger the next host, and so on.

I'm using unbound for DNS, and I'm wondering if there's some issue with waiting for unbound to start and begin resolving addresses at the time of haproxy startup?  Or, DHCP possibly, since these hosts are in DHCPv4 settings and maybe are not getting registered as hosts until too late in the startup process.

Again, odd that this only started in 22.7.7 -- other than changing all hosts to IP addresses, is there any other solution?

OK, I tried changing the log host, host1.localdomain to its IP address, but same error message appears on the reboot and haproxy does not start manually.

BTW, when I do a "Test Syntax" in haproxy, I do get:

Code Select Expand


[WARNING] (59447) : parsing [/usr/local/etc/haproxy.conf.staging:107] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.
[WARNING] (59447) : parsing [/usr/local/etc/haproxy.conf.staging:109] : a 'http-request' rule placed after a 'use_backend' rule will still be processed before.
Warnings were found.
Configuration file is valid


But, it has been like that for a long time and has worked without issue.

Could it be that there was some update that if there's a config file "warning" that startup is prevented (on purpose)?

Hi @Fright,

I applied the patch, and then did a reboot, but the problem is still there.  haproxy does not start after the reboot.  And, when I start haproxy manually from the Dashboard, it works, everything is ok.

The prior error is gone, and the only related log message I could find now in the "General" log is this:

Code Select Expand


/usr/local/etc/rc.d/haproxy: WARNING: failed precmd routine for haproxy


I'm attaching the sanitized haproxy.conf export, as mentioned.  Thank you!

In the shell, I get this:

Code Select Expand


root@kbl:~ # configctl template reload OPNsense/HAProxy 2 > /dev/null ; /usr/local/opnsense/scripts/OPNsense/HAProxy/syncCerts.py sync --output json
{"modified": [], "deleted": [], "add_count": 0, "remove_count": 0, "update_count": 0, "del_count": 0}


And, as you expected, in the backend log I see this:

Code Select Expand


[8c855732-8a22-4ffd-9648-d93ea83bf186] Inline action failed with not all arguments converted during string formatting at Traceback (most recent call last): File "/usr/local/opnsense/service/modules/processhandler.py", line 506, in execute inline_act_parameters = self.parameters % tuple(parameters) TypeError: not all arguments converted during string formatting


Any clues from that?

Hi @Taomyn,

I do have hostnames in my config, but it has been working fine for a couple years until now with that setup.   I can't imagine that the upgrade from 27.7.6 to 27.7.7 would uncover that problem, but who knows.

@fraenki, my router is headless.  Is there any other way of finding the failed on startup message logs?

Hi @fraenki,

Nothing interesting from the command line start process.

Code Select Expand


root@kbl:~ # service haproxy start
haproxy already running?  (pid=88652).
root@kbl:~ # service haproxy stop
Waiting for PIDS: 88652.
root@kbl:~ # service haproxy start
Starting haproxy.


The issue I see is only with the startup of haproxy at reboot.  I don't see the errors or problem when I start it manually.

Is there another logfile from the startup process?

In the upgrade logs (22.7.6 -> 22.7.7) I see these error/warning messages related to certs and haproxy.  Could they be related to the problem at startup?

[certs error]
https://forum.opnsense.org/index.php?topic=29557.0

[haproxy deprecation warning]
https://forum.opnsense.org/index.php?topic=29538.0

And, also this one:

Code Select Expand


Message from py39-urllib3-1.26.10,1:

--
Since version 1.25 HTTPS connections are now verified by default which is done
via "cert_reqs = 'CERT_REQUIRED'". While certificate verification can be
disabled via "cert_reqs = 'CERT_NONE'", it's highly recommended to leave it on.

Various consumers of net/py-urllib3 already have implemented routines that
either explicitly enable or disable HTTPS certificate verification (e.g. via
configuration settings, CLI arguments, etc.).

Yet it may happen that there are still some consumers which don't explicitly
enable/disable certificate verification for HTTPS connections which could then
lead to errors (as is often the case with self-signed certificates).

In case of an error one should try first to temporarily disable certificate
verification of the problematic urllib3 consumer to see if that approach will
remedy the issue.
You may need to manually remove /usr/local/etc/ssl/cert.pem if it is no longer needed.
You may need to manually remove /usr/local/openssl/cert.pem if it is no longer needed.


Hi,

I just upgraded to 27.7.7 and now haproxy fails to start after a reboot.  I see in the release notes that the haproxy plug-in was updated in this release to v3.12.

A related message in the log looks like this:

Code Select Expand


[1ff53fdb-8812-4a5b-bd04-04cddac2fa89] Script action failed with Command 'configctl template reload OPNsense/HAProxy 2 > /dev/null; /usr/local/opnsense/scripts/OPNsense/HAProxy/syncCerts.py sync --output json ' returned non-zero exit status 1. at Traceback (most recent call last): File "/usr/local/opnsense/service/modules/processhandler.py", line 482, in execute subprocess.check_call(script_command, env=self.config_environment, shell=True, File "/usr/local/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'configctl template reload OPNsense/HAProxy 2 > /dev/null; /usr/local/opnsense/scripts/OPNsense/HAProxy/syncCerts.py sync --output json ' returned non-zero exit status 1.


I am able to start haproxy manually from the Dashboard.

Additionally, rolling back to os-haproxy v3.11 by doing a ---

Code Select Expand

opnsense-revert -r 22.7.6 os-haproxy

-- did not correct the problem.  haproxy now no longer starts on reboot in either version.

Please advise --- thanks!

Hi all,

After upgrading to 22.1, none of my LAN devices could access the internet.  From ssh-ing into the router console, I could ping google.com but not from my laptop on the LAN.  I could also access the Opnsense web interface.  All services were running.

I was seconds from downloading a new 21.7 image and rebuilding everything, when I noticed an error message related to a floating rule I had for ssdp --

"01-28-22 19:45:34 [ There were error(s) loading the rules: no IP address found for ssdp ]"

It didn't seem like that could be blocking all traffic, but after disabling the rule everything worked.  All outbound traffic resumed, correctly.

See attached for the rule that was disabled.  The ssdp alias points to the ssdp multicast address 239.255.255.250

Just to be safe, I also disabled all UDP Broadcast Relay configs.  I'm not sure if this was related or not, but I'll test that also later, when I don't need to be online.  I was/am using the "os-udpbroadcastrelay" plug-in version 1.0_2

I'm wondering if there's either an issue with having a failing rule or the plug-in.