Looking for testers Q-Feeds plugin

[Lurick]

I realized I forgot to reply to the earlier quote but thank you for addressing those concerns I had so quickly!
One thing I just realized might be good to have on the roadmap is whitelisting. Either inbound or outbound integrated into Q-Feeds. Say I want a host to to not be restricted by q-feeds but still protected in other ways if that makes sense, it would be good to be able to easily whitelist source/destinations (public or private IPs) without the need for additional floating rules.

[Monviech (Cedrik)]

You do not need additional floating rules.

In the current one, set an Alias as Source (invert it in the rule) in which you add all hosts that should be excluded.

This means, all hosts that are not the ones in the alias will be inspected.

Same can be done with a inverted destination alias.

[Lurick]

You do not need additional floating rules.

In the current one, set an Alias as Source (invert it in the rule) in which you add all hosts that should be excluded.

This means, all hosts that are not the ones in the alias will be inspected.

Same can be done with a inverted destination alias.

Awesome! That works perfectly :)

[caplam]

Hello,
I didn't see that topic.
I'd like to try it.
I dropped suricata on Lan and vlan interfaces as it was causing issues when i have a spike in traffic.
I use crowdsec on wan but i had to desactivate it for nextcloud (too many false positives). Still active for other apps.
For now the most effective is geoip blocking inbound connections. It's ok for a homelab not so much for a company.
For the test do you need crowdsec disabled ?

[n0ahg]

Hi,

I'll give it a go if you still need more testers.

Thanks
Ray

[Q-Feeds]

Hello,
I didn't see that topic.
I'd like to try it.
I dropped suricata on Lan and vlan interfaces as it was causing issues when i have a spike in traffic.
I use crowdsec on wan but i had to desactivate it for nextcloud (too many false positives). Still active for other apps.
For now the most effective is geoip blocking inbound connections. It's ok for a homelab not so much for a company.
For the test do you need crowdsec disabled ?

Great thanks in advance, I'll send you the instructions shortly. No there's no need to disable crowdsec.

[Q-Feeds]

Hi,

I'll give it a go if you still need more testers.

Thanks
Ray

Great! Thank you in advance Ray, I'll send you the instructions shortly.

[tessus]

Thanks for your answers.

- Alerting / Notifications

I'll bring it up with the team. Although as you mentioned as well, I think this should be broader then just the Q-Feeds plugin.

Yea, this is something that might have to be developed with the OPNsense core team together. That is, if they want to have a "notification service" that is available to all of OPNsense (and its plugins). If not, alerting/notifications would still be important for your plugin.

As for a general design for notification/alerts, people should be able to create multiple "providers" (email, webhook, snmp, push service, ...) for sending alerts/notification. And then the user just chooses one or more providers for sending out alerts/notifications.

- Feedback on the feedback on feedback :D
We actually had quite a long brain storm today about the auto-deploy rules feature. For now, we've decided to put it on hold, mainly because there's really no "one rule fits all" approach. We're also cautious that users might assume, "If it's auto-created, it must be correct."
What's your take on this? How would you imagine a perfect auto-deploy function that works for everyone (or at least most users)?

I can give you a better answer after I have tested your plugin in more detail.

But I can imagine the following workflow and settings:

1. create a new rule automatically
2. leave the rule inactive or activate it (create a new setting, so people can choose)
3. send a notification/alert (create a new setting, so people can choose)

as for 2, one could even go further and create several "classes" or "groups" of rules, which allows users to create activate/keep-inactive setting per group. (but that might overdo it, just thinking out loud...)

[Q-Feeds]

Thanks for your answers.

- Alerting / Notifications

I'll bring it up with the team. Although as you mentioned as well, I think this should be broader then just the Q-Feeds plugin.

......

Thank you for your input! It would definitely be interesting to do so, especially if you're able to filter on it as well. So for example only notifications if machine X got a hit. We've added it to the list but it won't be soon.

That way of rule creation could work indeed, forcing a user to review them. We'll bring that to the table as wel! Thank you so much!

[Q-Feeds]

!! Update !!

I have some great news!! We've finally been able to tackle the rate-limit issue. And we've made some major improvements to the plugin.

We've changed the logic the old rate limit works. Now we're introducing a data delay. The community license now has a 7-day data delay, the Plus license a 4-hour delay and the Premium license is still the latest set. That said you can (try to) pull the data as many times (within boundaries) as you want on a day, you will receive the dataset of 7 days / 4 hours ago / 20 minutes respectively. The update mechanism in the plugin automatically handles the right update time.

We also added an event page to the plugin to see the actual activity. This will only work if you've applied logging on the rules where the Alias is bound to.

We've improved the widget with some more data.

And the plugin now moved from 'Services' to 'Security'

The new update scheme is already active. If you want to test the new Plugin functionality you can run the following command:

Code Select Expand

pkg add -f <same URL but with "-0.1_1.pkg" as extension>If you can't get it to work please send us a PM.

Please do not share the URL yet on the forum since we want to keep the testing group under control for now :)

Known issue: the widget on the TIP dashboard only shows the Premium count currently for all users. We will change this in the upcoming (work)days. We might spend some weekend hours on it :)

Once more we want to thank you all for you feedback! And obviously we keep on working on the rest of the list.

Kind regards,

Stefan

[Lurick]

!! Update !!

I have some great news!! We've finally been able to tackle the rate-limit issue. And we've made some major improvements to the plugin.

We've changed the logic the old rate limit works. Now we're introducing a data delay. The community license now has a 7-day data delay, the Plus license a 4-hour delay and the Premium license is still the latest set. That said you can (try to) pull the data as many times (within boundaries) as you want on a day, you will receive the dataset of 7 days / 4 hours ago / 20 minutes respectively. The update mechanism in the plugin automatically handles the right update time.

We also added an event page to the plugin to see the actual activity. This will only work if you've applied logging on the rules where the Alias is bound to.

We've improved the widget with some more data.

And the plugin now moved from 'Services' to 'Security'

The new update scheme is already active. If you want to test the new Plugin functionality you can run the following command:

Code Select Expand

pkg add -f <same URL but with "-0.1_1.pkg" as extension>If you can't get it to work please send us a PM.

Please do not share the URL yet on the forum since we want to keep the testing group under control for now :)

Known issue: the widget on the TIP dashboard only shows the Premium count currently for all users. We will change this in the upcoming (work)days. We might spend some weekend hours on it :)

Once more we want to thank you all for you feedback! And obviously we keep on working on the rest of the list.

Kind regards,

Stefan

Events page is empty and doesn't seem to load even though logging is enabled on the alias rules. I also noticed Events is on the menu but not in the tabs, overall though I look forward to testing this version going forward for the changes.

[Q-Feeds]

!! Update !!

I have some great news!! We've finally been able to tackle the rate-limit issue. And we've made some major improvements to the plugin.

We've changed the logic the old rate limit works. Now we're introducing a data delay. The community license now has a 7-day data delay, the Plus license a 4-hour delay and the Premium license is still the latest set. That said you can (try to) pull the data as many times (within boundaries) as you want on a day, you will receive the dataset of 7 days / 4 hours ago / 20 minutes respectively. The update mechanism in the plugin automatically handles the right update time.

.....

Events page is empty and doesn't seem to load even though logging is enabled on the alias rules. I also noticed Events is on the menu but not in the tabs, overall though I look forward to testing this version going forward for the changes.

Wel it could take up to 30 seconds to load the actual events. The missing tab is interesting, can't seem to reproduce that. Anyone else experiencing that?

[Seimus]

Updated,

I like the new changes.

1. The update was seamless, I had not issues at all. After update the Section moved from Services to a new main Section called Security.
2. The new Events TAB, is working for me, it loaded over 50K records e.g 4days. Is this hard-coded?
3. The events show as well Interfaces (NICE!) but only LAN based ones. The ingress block on WAN doesn't show the WAN interfaces in Events.
4. The widget is now much more better.

Regards,
S.

NOTE: The Events TAB takes time to load, as its parsing the log (Collected events from the firewall log for QFeed aliases), so the larger it is the longer it will take relative to the CPU performance.

[tessus]

I am not able to change my email address on the TIP web site.

I have noticed that the notifications are sent to the email I registered with. However, I usually like to add a modifier so that it is easier to filter. e.g. user+qfeeds@example.com

[tessus]

I am getting a warning:

QFeeds requires additional memory to be reserved for aliases. Please increase `Firewall Maximum Table Entries` in `Firewall: Settings: Advanced` to at least 2 million items.

However, when I check my settings, it is blank and my default is 10,000,000. (Note: Leave this blank for the default. On your system the default size is: 10000000)

I'd rather not set a value there, but go with my default which is already 5 times the value the QFeeds plugin requires.