Please bring back logging

[bigops]

The logging changes in 18.1 is really frustrating.  Recently had to spend more than 2 days troubleshooting a block issue.  The earlier answer to my question on logging was answered by Franco as all the information is available in live view.  But that is simply not true.  My issue was as follows.  One of the clients was complaining of connection issues to a VMWare service which was used infrequently by some  critical users.  Whenever the team troubleshoots the issue using live view as the client was not connected there is no information on what was being blocked.  The overview tab also is not of any help as there is no display where it links the client IP address to what is being blocked, only a total count of what was blocked with no drill down.  In addition the information is limited to 5000 entries which is very small and gets filled up fairly quickly.  Finally had to manually download the plain view, export it to Excel and figure of what was happening.  With limited information on the type of log generated this was another long process. 

Do not understand why OPNsense had to tweak a working logging and introduce something with less features 

[mimugmail]

If you use the firewall in business (which I guess since you talk about clients) you should consider troubleshoot via Console. It's quicker than the UI .. just do a "clog -f /var/log/filter.log" to see what's going on. You can pick interesting parts via "| grep" and see packets with tcpdump (e.g. entering but not going out).

I never use the UI but shouldn't "Plain View" do the trick for you?

[franco]

If someone replies to what you write, it's easier to reply to them again. I understand venting frustration everywhere possible, yet if you're not following up on your own topics you are warming others to the possibility that you won't follow up at all so they'll consider to stop replying in the future.

https://forum.opnsense.org/index.php?topic=7397.0

[bigops]

Franco

 The intention was not to post everywhere, but I considered the earlier post as as closed since you had provided an answer. 

To the answer by mimugmail , thanks for the response.   Filter log will be useful, but the engineers do not have console access due to security considerations.  (Mandatory two factor).  The plain view is not designed for humans  .  See the screenshot

[franco]

Okay, so, with an ideal page in mind: what do you expect? What is currently missing?

Needles to say, keeping old unmaintained code is no work, but improving it is overly difficult so we opt for going the long way of asking how much is needed and rebuild from there.


Thanks,
Franco

[bigops]

Thanks Franco

 From my perspective it could be one of the suggestions below other than to bring back the old code which you said is not maintainable.  From the Overview section of logging we have a very good view, and the issue is that there is no drill down capability which will solve the issue, otherwise either in one of these views (see attachment) there could be way to find link to either the IPs (attachment 1) of the ports (attachment 2 ) which resulted in these blocks.  Also it would be helpful to have a tab to increase the number of lines parsed from 5000 to something  like 20K

[franco]

You are looking for something like Netflow Insight reporting for the firewall logs?

[bigops]

Something like Netflow insight would be awesome, but I don't think I am looking for anything that fancy as it can be achieved using the ELK, for larger installations.  Just a easier way to identify (other than from the live view which is real time) what is getting blocked for an IP.  All the information is there in the  plain view.  What is missing is something to filter this to a more readable format

[franco]

There is a ticket to enable each data type found in the raw log as a column in the new live view:

https://github.com/opnsense/core/issues/2195

It would then be able to filter with the search box.

Last tweak we are actively discussing a tag-based filter, where you can mix your search input. For now it only searches a single string. Separating by whitespace makes the search fail, because it's looking for that whitespace...


Cheers,
Franco

[bigops]

Thanks.  I saw that this is listed for the 18.7 release.  In the interim is there any chance to bring back the normal view even though it is deprecated ?  I read somewhere that the rule creation is an issue in the new code, so only the view will be required.