Messages

1

21.7 Legacy Series / Syslogd segmentation fault after upgrade to 21.7.1

« on: August 27, 2021, 05:28:55 pm »

I have 3 OPNsense installations. 2 of 3 updated from 21.1 to 21.7 without any problem. The last one I stumbled upon a problem with syslogd not starting. syslog-ng DOES start without any problems.

Logging does not work (no new logs in /var/log).
The daemon won't start using the GUI (no response).
I tried starting the daemon manually:

dmesg shows:

-> pid: 60070 ppid: 88710 p_pax: 0xa50<SEGVGUARD,ASLR,NOSHLIBRANDOM,NODISALLOWMAP32BIT>
pid 60070 (syslogd), jid 0, uid 0: exited on signal 11 (core dumped)

When starting syslogd manually:

# /usr/local/sbin/syslogd -s -c -c -P /var/run/syslog.pid -p /var/run/legacy_log -S /var/run/legacy_logpriv -k -s -s -f /var/etc/syslog.conf   
syslogd: child pid 33106 exited on signal 11 (core dumped)

When starting syslogd in debug mode:

/usr/local/sbin/syslogd -d -s -c -c -P /var/run/syslog.pid -p /var/run/legacy_log -S /var/run/legacy_logpriv -k -s -s -f /var/etc/syslog.conf

The following error is shown at the end:

# /usr/local/sbin/syslogd -d -s -c -c -P /var/run/syslog.pid -p /var/run/legacy_log -S /var/run/legacy_logpriv -k -s -s -f /var/etc/syslog.conf
Trying peer: /var/run/legacy_log
new socket fd is 6
listening on socket
sending on socket
Trying peer: /var/run/legacy_logpriv
new socket fd is 7
listening on socket
sending on socket
off & running....
init
loading timezone data via tzset()
cfline("*.*                %/var/log/audit.log", f, "audit", "*")
cfline("*.*                %/var/log/configd.log", f, "configd.py", "*")
cfline("*.*                %/var/log/dhcpd.log", f, "dhcpd,dhcrelay", "*")
cfline("*.*                %/var/log/filter.log", f, "filterlog", "*")
cfline("*.*                %/var/log/gateways.log", f, "dpinger", "*")
cfline("*.*                %/var/log/lighttpd.log", f, "lighttpd", "*")
cfline("*.*                %/var/log/pkg.log", f, "pkg,pkg-static", "*")
cfline("*.*                %/var/log/portalauth.log", f, "captiveportal", "*")
cfline("*.*                %/var/log/ppps.log", f, "ppp", "*")
cfline("*.*                %/var/log/resolver.log", f, "unbound", "*")
cfline("*.*                %/var/log/routing.log", f, "radvd,routed,rtsold,olsrd,zebra,ospfd,bgpd,miniupnpd", "*")
cfline("*.*                %/var/log/wireless.log", f, "hostapd", "*")
cfline("*.*                %/var/log/dnsmasq.log", f, "dnsmasq", "*")
cfline("*.*                %/var/log/ipsec.log", f, "charon", "*")
cfline("*.*                %/var/log/ntpd.log", f, "ntp,ntpd,ntpdate", "*")
cfline("*.*                %/var/log/openvpn.log", f, "openvpn", "*")
cfline("*.*                %/var/log/squid.log", f, "(squid-1)", "*")
cfline("*.*                %/var/log/suricata.log", f, "suricata", "*")
cfline("local3.*                                                        %/var/log/vpn.log", f, "-(squid-1),audit,bgpd,captiveportal,charon,configd.py,dhcpd,dhcrelay,dnsmasq,dpinger,filterlog,hostapd,lighttpd,miniupnpd,ntp,ntpd,ntpdate,olsrd,openvpn,ospfd,pkg,pkg-static,ppp,radvd,routed,rtsold,suricata,unbound,zebra", "*")
cfline("local4.*                                                        %/var/log/portalauth.log", f, "-(squid-1),audit,bgpd,captiveportal,charon,configd.py,dhcpd,dhcrelay,dnsmasq,dpinger,filterlog,hostapd,lighttpd,miniupnpd,ntp,ntpd,ntpdate,olsrd,openvpn,ospfd,pkg,pkg-static,ppp,radvd,routed,rtsold,suricata,unbound,zebra", "*")
cfline("local7.*                                                        %/var/log/dhcpd.log", f, "-(squid-1),audit,bgpd,captiveportal,charon,configd.py,dhcpd,dhcrelay,dnsmasq,dpinger,filterlog,hostapd,lighttpd,miniupnpd,ntp,ntpd,ntpdate,olsrd,openvpn,ospfd,pkg,pkg-static,ppp,radvd,routed,rtsold,suricata,unbound,zebra", "*")
cfline("*.notice;kern.debug;lpr.info;mail.crit;daemon.none              %/var/log/system.log", f, "-(squid-1),audit,bgpd,captiveportal,charon,configd.py,dhcpd,dhcrelay,dnsmasq,dpinger,filterlog,hostapd,lighttpd,miniupnpd,ntp,ntpd,ntpdate,olsrd,openvpn,ospfd,pkg,pkg-static,ppp,radvd,routed,rtsold,suricata,unbound,zebra", "*")
Segmentation fault (core dumped)

The core is dumped to /syslogd.core
I've tried to analyse this with gdb:

(gdb) core syslogd.core
[New LWP 100190]
Core was generated by `/usr/local/sbin/syslogd -s -c -c -P /var/run/syslog.pid -p /var/run/legacy_log -'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000000209f0f in ?? ()
(gdb) bt
#0  0x0000000000209f0f in ?? ()
#1  0x0000011893eab020 in ?? ()
#2  0x0000011893eab039 in ?? ()
#3  0x0000011893eab000 in ?? ()
#4  0x0000000000000000 in ?? ()

How to analyse this further?
Thanks.

2

17.1 Legacy Series / Re: SNMP service is crashing regularly

« on: April 12, 2017, 09:32:26 am »

Ok, that could be a problem to find the issue then. My first thought was a proxy blacklist or an IDS signature update using the full memory is responsible for taking a large amount of memory.

We don't use any blacklists or IDS functionality.

3

17.1 Legacy Series / Re: SNMP service is crashing regularly

« on: April 10, 2017, 08:58:05 am »

Has this to do with an action like a cron job?

We have no custom cron jobs configured. bsnmp is killed at different times (00:10, 07:00...).

4

17.1 Legacy Series / Re: SNMP service is crashing regularly

« on: April 07, 2017, 08:37:55 am »

We experience the same problem on an OPNsense HA pair. SNMP is crashing every couple of days on both nodes. Graphs show memory spike, then bsnmp daemon gets killed. Log shows:

Code: [Select]

pid 18770 (bsnmpd), uid 0, was killed: out of swap space
We already upped memory from 1GB to 2GB, but this does not help. There is no swap space avaialable:

Code: [Select]

root@opnsense:~ # swapinfo
Device          1K-blocks     Used    Avail Capacity Any advice?

5

16.7 Legacy Series / Re: Update HA pair

« on: January 23, 2017, 09:44:15 am »

Yes, but how to update if the LAN is unplugged? If I replug the LAN it will switch back to being the active node.

6

16.7 Legacy Series / Re: Update HA pair

« on: January 20, 2017, 04:10:22 pm »

Hi Bart, thanks for the reply. Do you know how to failover to the other device other than unplugging or shutting down the active node?

Thanks.

7

16.7 Legacy Series / Re: Update HA pair

« on: January 18, 2017, 02:15:18 pm »

Does anyone have experience with this? Thanks.

8

16.7 Legacy Series / Update HA pair

« on: November 15, 2016, 04:44:53 pm »

Hi,

I currently run 2 OPNsense 16.7.0 instances in active/standby using CARP and pfSync. There are updates available. Is there any best practices method available for updating a HA setup of OPNsense?