Text only | Text with Images

OPNsense Forum

English Forums => General Discussion => Topic started by: r0ckky on November 20, 2020, 07:18:03 PM

Title: log standard.. or how to decipher logs
Post by: r0ckky on November 20, 2020, 07:18:03 PM
hello all,

Friday question here :)

i am exporting the logs into logstash and i need some help deciphering the log structure

for example:

Code Select
<134>Nov 20 15:35:55 OPNsense.localdomain filterlog: 14,,,0,igb0,match,block,in,4,0x0,,119,17591,0,DF,6,tcp,52,<redacted IP >,<redacted IP >,63652,7680,0,S,3174183196,,64240,,mss;nop;wscale;nop;nop;sackOK

im ok right up to this point

Code Select
,0,S,3174183196,,64240,,mss;nop;wscale;nop;nop;sackOK

is there some sort of guide or technical opensense doc that details what each of these fields represents ?
what the numbers represent what mss, nop etc mean in regards to the firewall log output?


i tried the logstash-filter-opnsensefilter (https://github.com/fabianfrz/opnsense-logstash-config) but it dosnt install cleanly on the latest logstash version, and whilst i dont have any errors on the logstash conf files, it refuses to utilise the logstash plugin which might be that its now out of date.

i built a logstash grok for a specific log event
Code Select
%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} %{DATA:syslog_program} %{NUMBER:rulenr},,,%{WORD:rid},%{WORD:interface},%{WORD:reason},%{WORD:action},%{WORD:dir},%{WORD:version},%{WORD:tos},,%{NUMBER:ttl},%{NUMBER:id},%{NUMBER:offset},%{WORD:ipflags},%{NUMBER:protonumber},%{WORD:protocol},%{NUMBER:length},%{IP:src_ip},%{IP:dst_ip},%{WORD}=%{NUMBER:datalen}

this works against : 
Code Select
<134>Nov 20 18:11:24 OPNsense.localdomain filterlog: 82,,,0,igb2,match,pass,out,4,0x0,,63,6038,0,DF,1,icmp,36,<redacted IP >,<redacted IP >,datalength=16

which is a simple icmp event.. seperates into nice seperate fields, no parse failures... wonderful... but i have all sorts of data type flowing through the firewall so im wondering if anyone else is having or had similar issues and how you got around it.

Many thanks

R
Title: Re: log standard.. or how to decipher logs
Post by: franco on November 21, 2020, 09:12:15 AM
Hi there,

Take a look at: https://github.com/opnsense/ports/blob/master/opnsense/filterlog/files/description.txt


Cheers,
Franco
Title: Re: log standard.. or how to decipher logs
Post by: r0ckky on November 21, 2020, 11:14:42 AM
you sir are a fricken legend!!!!!!!!! ;D
Title: Re: log standard.. or how to decipher logs
Post by: fabian on November 21, 2020, 07:49:03 PM
It is easy to parse using my plugin:

https://github.com/fabianfrz/logstash-filter-opnsensefilter
Title: Re: log standard.. or how to decipher logs
Post by: r0ckky on November 22, 2020, 04:11:13 PM
i really hoped it would be, but it has problems finding the correct gem on the rubygems site..

ill pull the ruby stuff from your git hub and try pushing it in manually

ive been creating my own conf for the opensense but id like to gives yours a go as it looks much cleaner...

do you know if your filter is going to be ok with opensense 20.7 ?
Text only | Text with Images