Remote syslog logging | Kernel messages from OPN > syslog server

Started by Seimus, June 14, 2025, 02:16:07 AM

Previous topic - Next topic
Hey all,

Is anyone using, remote logging to a central syslog server?

I am currently setting one, and I can see all messages from OPNsense on it, only not the kernel messages.

Did somebody see similar behavior or its just me?

Regards,
S.
Networking is love. You may hate it, but in the end, you always come back to it.

OPNSense HW
APU2D2 - deceased
N5105 - i226-V | Patriot 2x8G 3200 DDR4 | L 790 512G - VM HA(SOON)
N100   - i226-V | Crucial 16G  4800 DDR5 | S 980 500G - PROD

Did you check the configuration actually generated on the system. Maybe the code for some reason exempts the kernel messages ...
Deciso DEC750
People who think they know everything are a great annoyance to those of us who do. (Isaac Asimov)

Well I see it enabled in the xml config.

<Syslog version="1.0.2">
<general>
<enabled>1</enabled>
<loglocal>1</loglocal>
<maxpreserve>31</maxpreserve>
<maxfilesize/>
</general>
<destinations>
<destination uuid="aed08ccb-bd15-4097-8a98-3019be22d6d7">
<enabled>1</enabled>
<transport>udp4</transport>
<program>
audit,configd.py,dhcpd,dhcrelay,dnsmasq,filterlog,firewall,dpinger,charon,kea-ctrl-agent,kea-dhcp4,kea-dhcp6,lighttpd,monit,ntp,ntpd,ntpdate,openvpn,pkg,pkg-static,captiveportal,ppp,unbound,bgpd,miniupnpd,olsrd,ospfd,routed,zebra,suricata,wireguard,hostapd,zenoverlay
</program>
<level>debug,info,notice,warn,err,crit,alert,emerg</level>
<facility>
kern,user,mail,daemon,auth,syslog,lpr,news,uucp,cron,authpriv,ftp,ntp,security,console,local0,local1,local2,local3,local4,local5,local6,local7
</facility>
<hostname>-omitted-</hostname>
<certificate/>
<port>514</port>
<rfc5424>1</rfc5424>
<description>01-monit-stack_syslog-ng</description>
</destination>
</destinations>
</Syslog>

And I see kern logs in OPNsense itself. I would like to try to capture it pcap, but generating kern messages its a bit hard. In theory I can kill something on the FW that will generate kern message and pcap it....

Regards,
S.
Networking is love. You may hate it, but in the end, you always come back to it.

OPNSense HW
APU2D2 - deceased
N5105 - i226-V | Patriot 2x8G 3200 DDR4 | L 790 512G - VM HA(SOON)
N100   - i226-V | Crucial 16G  4800 DDR5 | S 980 500G - PROD

I had a look as well what is configured in syslog-ng in OPNsense

# cat /usr/local/etc/syslog-ng.conf
@version:4.8
@include "scl.conf"

#
# options
#
options {
    keep_hostname(yes);
    use_fqdn(yes);
    chain_hostnames(off);
    flush_lines(0);
    threaded(yes);
    create_dirs(yes);
};

source s_all {
    internal();
    file("/dev/klog" follow-freq(0) flags(no-parse) program-override("kernel"));
    unix-dgram("/var/run/log" flags(syslog-protocol));
    unix-dgram("/var/run/logpriv" perm(0600) flags(syslog-protocol));
    unix-dgram("/var/dhcpd/var/run/log" dir_perm(0755) flags(syslog-protocol));
    unix-dgram("/var/unbound/var/run/log" dir_perm(0755) flags(syslog-protocol));

};

Quotefile("/dev/klog" follow-freq(0) flags(no-parse) program-override("kernel"));

This specifies the kernel log, so its properly configured. But I thing this configuration is actually for the local logging on OPNsense.

When I try to look inside klog I get a busy message. Not sure if its even possible to cat it out

Quote# cat /dev/klog
cat: /dev/klog: Device busy



Looking into "remote" configuration

cat syslog-ng-destinations.conf

### log target aed08ccb-bd15-4097-8a98-3019be22d6d7 : 01-monit-stack_syslog-ng ####
filter f_aed08ccbbd1540978a983019be22d6d7_program {
    program("audit") or program("configd.py") or program("dhcpd") or program("dhcrelay") or program("dnsmasq") or program("filterlog") or program("firewall") or program("dpinger") or program("charon") or program("kea-ctrl-agent") or program("kea-dhcp4") or program("kea-dhcp6") or program("lighttpd") or program("monit") or program("ntp") or program("ntpd") or program("ntpdate") or program("openvpn") or program("pkg") or program("pkg-static") or program("captiveportal") or program("ppp") or program("unbound") or program("bgpd") or program("miniupnpd") or program("olsrd") or program("ospfd") or program("routed") or program("zebra") or program("suricata") or program("wireguard") or program("hostapd") or program("zenoverlay")
};
filter f_aed08ccbbd1540978a983019be22d6d7_level {
    level(debug,info,notice,warn,err,crit,alert,emerg)
};
filter f_aed08ccbbd1540978a983019be22d6d7_facility {
    facility(kern,user,mail,daemon,auth,syslog,lpr,news,uucp,cron,authpriv,ftp,ntp,security,console,local0,local1,local2,local3,local4,local5,local6,local7)
};

destination d_aed08ccbbd1540978a983019be22d6d7 {
    network(
        "-omitted-"
        transport("udp")
        port(514)
        ip-protocol(4)
        flags(syslog-protocol)
        persist-name("aed08ccbbd1540978a983019be22d6d7")
    );
};

log {
    source(s_all);
    filter(f_aed08ccbbd1540978a983019be22d6d7_program);
    filter(f_aed08ccbbd1540978a983019be22d6d7_level);
    filter(f_aed08ccbbd1540978a983019be22d6d7_facility);
    destination(d_aed08ccbbd1540978a983019be22d6d7);
};

This is how is the remote configured. The kernel is as kern under facility.

Quotefilter f_aed08ccbbd1540978a983019be22d6d7_facility {
    facility(kern,user,mail,daemon,auth,syslog,lpr,news,uucp,cron,authpriv,ftp,ntp,security,console,local0,local1,local2,local3,local4,local5,local6,local7)
};

NOW the question is will this pick up the logs from /dev/klog?

Regards,
S.
Networking is love. You may hate it, but in the end, you always come back to it.

OPNSense HW
APU2D2 - deceased
N5105 - i226-V | Patriot 2x8G 3200 DDR4 | L 790 512G - VM HA(SOON)
N100   - i226-V | Crucial 16G  4800 DDR5 | S 980 500G - PROD

Anyway I opened a ticket on git directly cause I find it weird

https://github.com/opnsense/core/issues/8800

Regards,
S.
Networking is love. You may hate it, but in the end, you always come back to it.

OPNSense HW
APU2D2 - deceased
N5105 - i226-V | Patriot 2x8G 3200 DDR4 | L 790 512G - VM HA(SOON)
N100   - i226-V | Crucial 16G  4800 DDR5 | S 980 500G - PROD