"live view" results do not persist on screen: they disappear from view

[PerpetualNewbie]

Is there a way to have "live view" (Firewall -> Log files -> Live view) setup with a filter (example:  "interface_name=YOUR_NAME" and "action!=pass" and "src!=255.255.255.255" and "dst!=255.255.255.255")

and then show the results and have them persist on the page?

The behaviour I am seeing right now is not like earlier releases of OPNSense...

Now, a collection of matched to the filter appear in the browser, then in 0.2 seconds to 5 seconds disappear. Later other matches appear, then disappear. I've had to resort to "control-a , control-c" to copy the results when I see them, so I can paste them into a text editor, so I can read them.

At one time (maybe 1 year ago), matches to filter would appear on page an not disappear. Each match would appear in order, one row after the next until max allowed rows were found, then stop, and not disappear unless we selected the "refresh" option/button.

Thanks!

[atom]

The problem has already been discussed here. 

 https://forum.opnsense.org/index.php?topic=31360.0

There is a hot fix from @Fright.
But there are too few reports about this problem, so it will not be fixed for the time being.

[Fright]

IMHO in this case, the understanding of what the choice of the maximum number of records should mean may differ: show records that match filters among the latest max_rows records in pf log OR show latest max_rows records that match the filters. now it has started to work according to the first scenario.
so to make records appear longer, you can try to reduce the logging of rules that definitely don't fit your filters (besides, it is good for resources) and increase the limit (unfortunately, i expect a performance impact if the limit is increased above 2k).
I'd love to continue the discussion with @AdSchellevis when he has time (i think he's pretty busy with 23.1 right now) and if the number of reports increases.
imho there is still a room for client-side performance improvement.

[franco]

The key issue is the auto-refresh now works like manual refresh like it was described here clearing the records when they indeed disappear from the window of search results (it depends on the number of logs generated per time period) and that is why results now disappear into the haystack sooner or later.

Unfortunately the previous implementation of the auto-refresh did lazy-clearing of the results presented and that seems to be a desired result....

My best guess here is that a single-shot mode is required which turns itself off after it finds a result so it can be inspected.

It's easy to want to have a live-view that provides historical browsing of data, but it's a monitoring tool for current events as they are generated and disappear from immediate interest. A live-log inspired mode for the "plain log" data is probably what some other people are looking for here which also allows browsing backwards in time, but it has to be taken as a larger project than just doing a lazy-clearing of results.


Cheers,
Franco

[atom]

Hi Franco,

I always have use cases for both scenarios you describe.
Both - stopping at the first hit and a long term view -  are very useful features of a firewall live log.

Regards,
atom

[Fright]

@franco, thanks for reply!
imho the live log lost a little in functionality with the loss of some retrospectiveness ..
To understand what to do next, could you explain the reasons for changing the logic: do you insist that this is how it should work or the reason, for example, is in the matter of performance or something else?
Since in the first case we need to think about some kind of tool that repeats the previous functionality, and in the second - may be we can still try to do something 
let me explain: the previous functionality helped (without restricting the logging of other rules) to evaluate the frequency of triggering a certain rule or track the results of some actions (for example, https://forum.opnsense.org/index.php?topic=31525.0 it helps to quickly see how many requests and from which  subnets comes when trying to obtain a cert. or with https://forum.opnsense.org/index.php?topic=31475.0 it helps to quick check if roots were poking. Etc).
the use of a plain log for the same purposes causes me doubts, primarily because we cannot trust rulenr-label conversion over a long period of time.
(therefore, only the almost complete repetition of the live log page comes to my mind, only without auto-refresh: grab a number of recent records and apply filters)