NtopNG plugin and issues under OPNSense since 23.7 series

[rfox]

As referenced in a previous thread from Legacy - the same issues persists with 24.1.6 and latest ntopng plugin -

Network discovery appears to not work properly - can only see 8.0.0.0 on all interfaces - and when a scan is performed, nothing shows up - see one error in logs: 24/Apr/2024 10:15:14 [NetworkDiscovery.cpp:968] Error while sending DHCP discovery

NTopng support channels don't help because they say it's a plugin and I should seek help from OPNSense - which in turn says, it's a plugin - check with NTopng, we are not responsible.

Here's the reference to the previous thread in Legacy:
https://forum.opnsense.org/index.php?topic=36041.0

[franco]

To be frank if they do not support their product you should stop bothering with it.

They certainly don't care about the plugin in the community version the've copied to make their enterprise version.

I think they just want you to pay to get their help.


Cheers,
Franco

[rfox]

To be frank if they do not support their product you should stop bothering with it.

They certainly don't care about the plugin in the community version the've copied to make their enterprise version.

I think they just want you to pay to get their help.


Cheers,
Franco

I suspect you are right Franco!  Interestingly enough, the plugin author is the same for Mailtrail as well as Net-SNMP, vnstat, and more!  Very grateful for such contributors, makes OPNSense that much more attractive as a platform - but if nobody feels responsible for the quality of the plugins - we may need a rethink ?!?

I see there is a "tier" rating for the community plugins - yet maybe we should have a rating type of system which reflects level of support - like ones which are "use at your own risk" vs "highly recommended" from OPNSense / DeCiso 

How is this handled on the "Official" paid version of OPNSense (or with original hardware?)

Thx again for the prompt response!

[rfox]

Would also be helpful to know if I'm the only one affected or is this issue specific to my configuration . . .

[franco]

The problem is the scope here which is unclear. Is this a plugin issue or an issue with the ntop software? Can this be fixed in the plugin? If yes a PR to the plugin is trivial even for Michael, but figuring out what the problem is could be hours of work if it's even solvable and not related to the FreeBSD package that ntop is releasing.

Not supporting OPNsense is ok... but not supporting open source users is a bit strange considering it's open source.


Cheers,
Franco

[rfox]

The problem is the scope here which is unclear. Is this a plugin issue or an issue with the ntop software? Can this be fixed in the plugin? If yes a PR to the plugin is trivial even for Michael, but figuring out what the problem is could be hours of work if it's even solvable and not related to the FreeBSD package that ntop is releasing.

Not supporting OPNsense is ok... but not supporting open source users is a bit strange considering it's open source.


Cheers,
Franco

Well said Franco and understood.  I will crawl back into my cave 

Just wanted to point out that the issue still exists with latest versions and see if someone else was affected . . . It makes sense to have a tool like NTopNG running directly on the main firewall (middle point of all traffic)

I don't want to mess up my firewall by trying to install ntopng directly on the BSD underneath OPNSense.

Alternative is to create a VM dedicated to this task and give it access to all VLANs and see if said problem still exists ?!?

Thx again - and keep up the great work!

[franco]

Maybe a netflow export can be of assistance? Most of the time running reporting on the main box isn't the ideal solution (although from a capture standpoint that is practical). Maybe zenarmor can do what you need it to do too.


Cheers,
Franco

[rfox]

Using Zenarmor already - find it excellent!  Wanted to try ntopng because I used to use the original ntop way back when and really found it useful . . . not sure one can replace the other, they are more complementary to eachother IMHO

[franco]

It depends on the specific use cases, of course.

Just for fun I installed the community ntopng and it shows data and performed a network discovery finding 3 devices which looks about right. No errors (un)fortunately. It's probably not the plugin, worst case maybe a firewall setup issue.


Cheers,
Franco

[rfox]

It depends on the specific use cases, of course.

Just for fun I installed the community ntopng and it shows data and performed a network discovery finding 3 devices which looks about right. No errors (un)fortunately. It's probably not the plugin, worst case maybe a firewall setup issue.


Cheers,
Franco

Thanks Franco - I take it you tested on a bare metal device  I actually have mine running under Proxmox - so it might be a VM issue    Otherwise, all has been running for almost 8 months now (on an R86s G2 device) - I will investigate further and see how far I get . . .

Cheers,
Robert

[franco]

Hi Robert,

I don't know the inner workings, but is the Proxmox maybe unwilling to let the interfaces go into promiscuous mode?


Cheers,
Franco

[rfox]

Update from my side - trying to troubleshoot and reduce the possibilities - I installed NTopNG on a freshly install Debian VM instance under Proxmox to see how that behaved - and the discovery actually works as expected ?!? 
So that eliminates the Proxmox networking issue (I read somewhere that physical interfaces are normally set to promiscuous by default)

Next experiment will be to install a test OPNSense VM and try the ntopng plugin first - and see if discovery works - if it does, then it's something with my production instance - if it doesn't, then it could be the plugin itself - In that case, I will deinstall the plugin and try to manually install on the BSD beneath OPNSense and see if it's any better !  Worse case, it may be something with the way BSD handles virtual devices under Proxmox and either I'll have to forget discovery or try another approach with span ports and port mirroring on a Proxmox bridge instead of NTopNG running under OPNsense ?!? (as described here:  https://monach.us/operations/sending-promisc-traffic-within-proxmox/

I will report back with the result when I can!! Wish me luck . . .

[rfox]

Update again - I completely removed ntopng & redis plugins - manually deleted all related databases from the filesystem - then reinstalled "fresh" on the production system - Unfortunately, didn't change - I still can't perform a network discovery (btw - the MAC address shown after discovery is the OPNsense interface 192.168.30.1 - not sure where 9.0.0.0 comes from ?!?

Will try a fresh OPNsense install next . . . Stay tuned to this channel!

[rfox]

A new update!  I installed NTopNG on a freshly installed OPNSense with and without the plugin - same issue continues with network discovery.  I then tried with the "brctl setageing vmbr1 0" command whcih gave me more info on the VLAN but didn't allow discovery to work . . .

My suspicion is that it's something to do with the core BSD interfaces when they run under Proxmox VirtIO

Guess i will give up for now . . . but may try again on the weekend 

[franco]

Can you use an emulated E100 adapter just for testing?


Cheers,
Franco