Problem installing the latest ntopng

[mihak]

I followed the instructions on how to install ntopng 4.3:
https://packages.ntop.org/FreeBSD/

The problem is that even though I have a new repo added and visible to pkg manager, it still doesn't want to pull ntopng from ntop repo - it rather pulls it out from default OPNSense repo.

The new package is there:
> pkg search ntopng
ntopng-4.0.d20200917,1         Network monitoring tool with command line and web interfaces
os-ntopng-1.2                  Traffic Analysis and Flow Collection
os-ntopng-devel-1.2            Traffic Analysis and Flow Collection
ntopng-4.3.210107              High speed network traffic monitor

But pkg manager prefers to pull the old one instead of the new one:
> pkg info
...
ntopng-4.0.d20200917,1         Network monitoring tool with command line and web interfaces
...

It doesn't matter how high the priority I set in ntop.conf - pkg update will still pull from OPNSense repo, not ntop repo...

[mihak]

ok, I figured it out how to force OPNSense to choose the latest ntopng repo after it is added to the repo list:

sudo pkg install -r ntop -f ntopng nprobe n2disk
Updating ntop repository catalogue...
ntop repository is up to date.
All repositories are up to date.
The following 3 package(s) will be affected (of 0 checked):

Installed packages to be REINSTALLED:
        n2disk-3.7.210107 [ntop]
        nprobe-9.3.210107 [ntop]
        ntopng-4.3.210107 [ntop]

Number of packages to be reinstalled: 3

833 KiB to be downloaded.

Proceed with this action? [y/N]: y
[1/2] Fetching nprobe-9.3.210107.txz: 100%  667 KiB 170.8kB/s    00:04   
[2/2] Fetching n2disk-3.7.210107.txz: 100%  166 KiB 170.1kB/s    00:01   
Checking integrity... done (0 conflicting)
[1/3] Reinstalling ntopng-4.3.210107...
[1/3] Extracting ntopng-4.3.210107: 100%
[2/3] Reinstalling nprobe-9.3.210107...
[2/3] Extracting nprobe-9.3.210107: 100%
[3/3] Reinstalling n2disk-3.7.210107...
[3/3] Extracting n2disk-3.7.210107: 100%

[mimugmail]

I think the priority has to be below 11 since OPN uses 11.

[mihak]

Nope, I just uninstalled and tried it fresh - ntopng 4.3 (from ntop repo) will not be prioritized over ntpong 4.0 (from OPNSense repo) even when priority is set below 11:
ntop: {
       url: "https://packages.ntop.org/FreeBSD/${ABI}/latest",
       priority : 5
       enabled: true
}

[mimugmail]

Yep, franco is currently working on it

[mihak]

I don't know if this was just my problem, but my ntopng 4.3 service on OPNSense was failing periodically because it didn't have write permissions on /var/db/ntopng directory structure. A (dirty) fix with chmod -R 777 * did the trick and made ntopng 4.3 stable on OPNSense.

[TheLinuxGuy]

I noticed that ntopng from opnsense somehow got downgraded to 3.4? It used to be 4.0 I believe.

Thanks for the note in this thread I did the upgrade via:
# pkg install -r ntop -f ntopng nprobe n2disk

Also did the chmod trick, do we know if this issue is resolved? The latest 21.1 opnsense upgrade seems to have downgraded my ntopng from 4.0 to 3.4 and since i noticed it I tried to get latest via here.

[franco]

I don't know what you are talking about and honestly it's difficult to help with external ntop repository usage. We talked to them and noted that their build settings are not 100% compatible so some bouncing back and forth is the least problematic issue going forward...

But anyway, these are our versions as per FreeBSD ports:

https://pkg.opnsense.org/FreeBSD:12:amd64/21.1/MINT/21.1/LibreSSL/All/

ntopng-4.2.d20201228,1.txz   2021-01-26 15:38   7.2M   
ndpi-3.4.d20201222,1.txz   2021-01-26 15:38   524K   

Assuming "downgraded my ntopng from 4.0 to 3.4" actually meaning ndpi would make a bit more sense.

For major updates - and this was requested by many - we cleanly reinstall all our packages from our mirror... Meaning I cannot see a bug here other than ntop shipping duplicate packages and not adhering to our build parameters.


Cheers,
Franco

[RadOD]

I don't know what you are talking about and honestly it's difficult to help with external ntop repository usage. We talked to them and noted that their build settings are not 100% compatible so some bouncing back and forth is the least problematic issue going forward...

But anyway, these are our versions as per FreeBSD ports:

https://pkg.opnsense.org/FreeBSD:12:amd64/21.1/MINT/21.1/LibreSSL/All/

ntopng-4.2.d20201228,1.txz   2021-01-26 15:38   7.2M   
ndpi-3.4.d20201222,1.txz   2021-01-26 15:38   524K   

Assuming "downgraded my ntopng from 4.0 to 3.4" actually meaning ndpi would make a bit more sense.

For major updates - and this was requested by many - we cleanly reinstall all our packages from our mirror... Meaning I cannot see a bug here other than ntop shipping duplicate packages and not adhering to our build parameters.


Cheers,
Franco

I'm guessing he is seeing this

Code: [Select]

ntopng Community v.3.4.0
on the ntopng page after you log in.  This was confusing me too. I know this can't be 3.4.0 -- but it also keeps bugging me to upgrade to 4.2.0 even though pkg info says I'm on 4.2.0 already.

Code: [Select]

ntopng-4.2.d20210122,1


[RobLatour]

I am seeing the same thing, an older version (ntopng Community v.3.4.0 | © 1998-20 - ntop.org) being reported on the bottom of the page.

Also, constant nag pop-ups that say "A new ntopng stable version (v.4.2.0) is available for download: please upgrade.". 

When I check OPNSense - System - Firmware - Status - check for updates it says there are none.

When this is resolved, the Firmware Status check pick up the changes?

[franco]

I am seeing the same thing, an older version (ntopng Community v.3.4.0 | © 1998-20 - ntop.org) being reported on the bottom of the page.

Also, constant nag pop-ups that say "A new ntopng stable version (v.4.2.0) is available for download: please upgrade.".

Ntop GUI reports go to Ntop. OPNsense GUI reports go to OPNsense. I'm guessing these two fall into the first category.

When I check OPNSense - System - Firmware - Status - check for updates it says there are none.

When this is resolved, the Firmware Status check pick up the changes?

Again, I don't know what to resolve: Ntop in FreeBSD not being latest or Ntop reporting things that it doesn't deliver on in terms of updates or something else entirely?

If you install ntop from OPNsense repo you will stay on OPNsense repo at least that is what the package manager will do. If their plugin depends on an "ntopng" package that exists twice it can always pull the wrong one. We ask third party repositories not to ship duplicated package names that will confuse the package manager. That's just a guess, but since we've never seen such issues before that is a probable mistake.

If Ntop wants to resolve these things they know how to contact us and until then you will have to keep bringing these things up with them because we can't do more from our end.


Cheers,
Franco

[RobLatour]

Thanks, I could not get onto the NTOP Community forum - their verification server was not working for me.  However, I did send them a message via a web contact form on their site.  Hopefully, they will see that and respond.

[RobLatour]

I was able to reach NTOP support via Discord.

They were very helpful and noted I had the community edition of NTOP installed and suggested I remove that and install the enterprise edition - as per: https://www.ntop.org/guides/ntopng/third_party_integrations/opnsense.html

So I did that - the enterprise edition was installed without a license key. 

On the OPNSense - Services - ntopoing Enterprise - Settings - License (tab) window I also checked the option 'Community Mode'; all other settings were left to thier defaults.

So the above + a firewall reboot + resetting my ntop admin account's password + reboot the firewall again got it working.

To reset the ntop admin password to its defaults (user id: admin, password: admin) I issued this command:
   sudo redis-cli del ntopng.user.admin.password
(not sure if the sudo was required, but it worked).

Of note on the OPNSense - System - Firewall - Status - Plugins (tab) window, the plugin has a name of 'os-ntopng-enterprise (misconfigured)' - regardless it is working.

To sign on: I am now using the address:
http://192.168.1.1:3000/
192.168.1.1 being the address of my OPNSesne firewall.

I haven't figured out how to access the NTOP GUI via https yet, which was easier with the older version of the OPNSense window that is shown here (as the last screenshot on the page):
https://www.ntop.org/guides/ntopng/third_party_integrations/opnsense.html

The newer version seems to require a certificate, which I have not yet figured out how to generate - although these posts will hopefully help me figure it out:
https://www.ntop.org/ntopng/best-practices-to-secure-ntopng/
https://www.ntop.org/ntopng/securing-ntopng-with-ssl-and-lets-encrypt/

If anyone has more info on that last point, or even better a video, please feel free to add a link

Hope this will be of help to others

[RobLatour]

Also, one additional note, GeoMapping stopped working after the update described above.
To get it to work I copied the MaxMind files
  GeoLite2-ASN.mmdb
  GeoLite2-Country.mmdb
  GeoLite2-City.mmdb
into
/usr/share/GeoIP/
as opposed to
/usr/local/share/ntopng/httpdocs/geoip/
where they had been stored for the prior version of ntop's use.

Plus then after this change, restart the NTOPing service.

[kosta]

@RobLatour
I saw your post, this:

On the OPNSense - Services - ntopoing Enterprise - Settings - License (tab) window I also checked the option 'Community Mode'; all other settings were left to thier defaults.

Where is this? I don't have that TAB, and if I install Enterprise, I always have the countdown.