Best log viewer?

[spidysense]

What is some of the best ways to view the Firewall & Suricata logs?
Any good plugins for this? Which one do you use? What are the benefits of your choice?

Thanks.

[bigops]

I also have the same question.  If we forward the logs to a Syslog server the view is not very pretty.   Is there documentation on the log format somewhere or is there a specific log analyzer that supports OPNsense logs?

[fabian]

you can use an ELK stack with this little config as a base: https://github.com/fabianfrz/opnsense-logstash-config

[spidysense]

Can you describe how to set that up? Not as simple as installing a package 

[fabian]

That is a bit of work but it does its job: You need logstash, kibana and elastic search. (all of them can be downloaded from the vendor if your distribution does not provide them from here: https://www.elastic.co/

First of all, you will need an elastic search database. Should be easy to configure (usually works out of the box).
In the next step, you need Kibana as a viewer. It should also work out of the box but you may want to adjust the config file. Kibana is a frontend for elastic search and it helps you to visualize data. This will be your firewall logs later.
In the next step, you need logstash, which is a software to transform logs. It needs to listen on the syslog port as a syslog server and stores the data in the elastic search database. For that, you need a configuration file (for example the one I have mentioned). As far as I can remember, the config file can be used using the following command:

Code: [Select]

logstash -f opnsense.confIn the last step, you need to configure this logstash server as an external syslog server.

Please note: My configuration file needs a custom parser plugin written by me for the PF logs to work. It can be installed using the logstash-plugin command.

[guest15389]

I was going to test that out.

Where is the filter you have installed? I was searching around and can't seem to find it/figure out how to install it.

[fabian]

That one? https://rubygems.org/gems/logstash-filter-opnsensefilter

install:

Code: [Select]

logstash-plugin install logstash-filter-opnsensefilter


[AveryFreeman]

That one? https://rubygems.org/gems/logstash-filter-opnsensefilter

install:

Code: [Select]

logstash-plugin install logstash-filter-opnsensefilter


What are you all running ELK stack on, a separate Linux system?

How do you get OPNSense to transmit logs to SysLog server?

[fabian]

1) Yes

2) You can configure syslog under system, settings

[spetrillo]

Does the filter get implemented on the OPNsense firewall?

[beyondnoyeb]

That one? https://rubygems.org/gems/logstash-filter-opnsensefilter

install:

Code: [Select]

logstash-plugin install logstash-filter-opnsensefilter


Hate brining up an older thread; but don't see the need in making a new one since the topic is the same.

Hoping you can help get this running.. I've got your plugin installed, i'm running 7.6 version of the elk stack.  I've edited my conf file that I was already collecting syslog's on port 5514 to include the main body of your conf file. 

For some reason when i'm looking in kibana it's still showing the crappy layout of unparsed info instead of using what you've built. 

Does your stuff work with the latest elk stack versions?  And anything else i'm missing?

[fabian]

Last version I have tested was v6 but I only had to update the config once and that was because OPNsense changed the syslog implementation with 20.1.

In worst case you need to debug your logstash configuration by running it in foreground and sending stuff to the stdout.

[beyondnoyeb]

@fabian,

Curious, do you use your elk stack for anything else or is it just devoted to opnsense output? 

[fabian]

My config is a standalone for OPNsense but you can use it for other services as well but then you need to tag the events or filter by origin ip.