Messages

Hi,

I'm using PPPoE with a tagged VLAN (201) on my WAN interface, to connect to CenturyLink fiber. Everything seems to be working okay, but I notice this error on bootup:

Code Select Expand

2022-09-02T14:42:27-07:00 Error php /usr/local/etc/rc.bootup: ROUTING: entering configure using defaults
2022-09-02T14:42:25-07:00 Error php /usr/local/etc/rc.bootup: The command '/usr/sbin/ngctl msg 'igb0_vlan201': setautosrc 1' returned exit code '71', the output was 'ngctl: send msg: No such file or directory'

Searching the forums, I see several references to this error, but mostly only from longer log dumps that aren't discussing this error in particular. I can find no clear explanation for the error. Anyone have any thoughts?

I had the same issue after upgrade: high cpu usage by a python process. Was gone after a reboot.

Code Select Expand

root    48330 99.4  0.3   33936   22620  -  R    22:51      2:07.71 /usr/local/bin/python3 /usr/local/opnsense/service/configd_ctl.

I have upgraded but not rebooted my other machine which show the same issue, can I do anything helpful in investigating this?

Edit: I see lots of these in the log:

Code Select Expand

2021-09-22T20:28:12 configctl[48330] event @ 1632335220.47 msg:
2021-09-22T20:28:12 configctl[48330] event @ 1632335220.47 msg:
2021-09-22T20:28:12 configctl[48330] event @ 1632335220.47 msg:
2021-09-22T20:28:12 configctl[48330] event @ 1632335220.47 msg:
2021-09-22T20:28:12 configctl[48330] event @ 1632335220.47 msg:
2021-09-22T20:28:12 configctl[48330] event @ 1632335220.47 msg:
2021-09-22T20:28:12 configctl[48330] event @ 1632335220.47 msg:
2021-09-22T20:28:12 configctl[48330] event @ 1632335220.47 msg:
2021-09-22T20:28:12 configctl[48330] event @ 1632335220.47 msg:
2021-09-22T20:28:12 configctl[48330] event @ 1632335220.47 msg:

syslog-ng has high cpu load now as well.

If you don't reboot, those will fill up your logs pretty quickly, to the tune of over 40 GB per day of logs in my case.

https://forum.opnsense.org/index.php?topic=24148.0

Franco commented there (and also on Reddit this morning) that "Something in the combination of Python and Syslog-ng update of 21.1.9" started causing this on occasion and that they're looking into it.

P.S. I realized after writing this that the timing on July 27th suggests this occurred around the upgrade to 21.1.9 from 21.1.8.

I'm on OPNsense 21.1.9_1-amd64. I noticed a sudden increase in disk space usage on the dashboard. I found an anomoly in the system.log from two days ago, which seemed to resolve itself without a reboot before I noticed (uptime is 20 days currently).

Code Select Expand


root@opn:/var/log/system # ls -lF
total 52649044
-rw-------  1 root  wheel         1368 Jul 23 21:36 system_20210723.log
-rw-------  1 root  wheel         1368 Jul 24 21:37 system_20210724.log
-rw-------  1 root  wheel         1364 Jul 25 21:38 system_20210725.log
-rw-------  1 root  wheel         1362 Jul 26 21:39 system_20210726.log
-rw-------  1 root  wheel  46888471248 Jul 27 23:28 system_20210727.log
-rw-------  1 root  wheel   7010838657 Jul 28 21:41 system_20210728.log
-rw-------  1 root  wheel         1536 Jul 29 13:56 system_20210729.log


Whoa, yeah... that's quite an increase! Let's take a closer look:

Code Select Expand


# tail system_20210727.log
Jul 27 23:28:27 opn.localdomain configctl[40506]: event @ 1627449996.77 msg:
Jul 27 23:28:27 opn.localdomain configctl[40506]: event @ 1627449996.77 msg:
Jul 27 23:28:27 opn.localdomain configctl[40506]: event @ 1627449996.77 msg:
Jul 27 23:28:27 opn.localdomain configctl[40506]: event @ 1627449996.77 msg:
Jul 27 23:28:27 opn.localdomain configctl[40506]: event @ 1627449996.77 msg:
Jul 27 23:28:27 opn.localdomain configctl[40506]: event @ 1627449996.77 msg:
Jul 27 23:28:27 opn.localdomain configctl[40506]: event @ 1627449996.77 msg:
Jul 27 23:28:27 opn.localdomain configctl[40506]: event @ 1627449996.77 msg:
Jul 27 23:28:27 opn.localdomain configctl[40506]: event @ 1627449996.77 msg:


... but how many of those lines are there?

Code Select Expand


# grep configctl system_20210727.log | wc -l
601134222


Wow, over 600 million. And I have no idea what they're about, particularly with the "msg" field being empty.

Any thoughts? How would I go about debugging this further?

This method works for me on OPNsense 21.1.4-amd64:

Code Select Expand

pkg add "https://install.speedtest.net/app/cli/ookla-speedtest-1.0.0-freebsd.pkg"