Cannot Get NTOPNG Geo Mapping To Work

[bungo63]

Hi, I am fairly new to OPNsense and am evaluating with it with a view to migrating from Untangle. The feature that I need is Geo mapping  within NTOPNG but I have not been able to get it to work. I can get NTOPNG working ok. I have followed the instructions in the OPNSense manuals (which seemed easy enough) but still dont get any GEO Map options in NTOPNG.

Yes I have the Maxmind key and its installed.

I have looked at the installation guide on GITHUB but I don't understand how to install packages. I've googled OpenBSD commands but none of these seem to work. Also I cannot find any NTOPNG directories that are mentioned on posts or YouTube.

Also another challenge seems to be the the GUI of NTOPNG seems to have changed (from the one commonly seen on YouTube) . I don't see where you have to install the Google API key, so I assume you get that goodness when the Geo Map function is actually working.

One last thing which may be connected. I'm running my OPNsense as a VM in Virtual Box. When the VM is booted i'm seeing a lot of REDIS error messages in the VM console. I know there is a dependency upon the REDIS service so I always make sure this is running - but it makes no difference.

Any advice would be greatly appreciated.

[hushcoden]

Same situation here, and every time I log in into ntopng I have a message saying the Geolocation has not been enabled and I have to visit this link https://github.com/ntop/ntopng/blob/dev/doc/README.geolocation.md

Frankly, it's quite confusing...

[gpb]

Did you manually copy the geo files to the appropriate folder?  I've had to do this for each upgrade of OPNsense (a couple exceptions maybe...but mostly every time).  I'm not running ntopng since 12.1.5 as it causes excessive CPU usage for me...but this is the procedure I would need to perform to get the geo data feature enabled (along with the google api key for the map).

Copy files to /usr/local/share/ntopng/httpdocs/geoip/ (from MaxMind)

GeoLite2-ASN.mmdb
GeoLite2-Country.mmdb
GeoLite2-City.mmdb

chmod +x /usr/local/opnsense/scripts/OPNsense/Ntopng/generate_certs.php

[hushcoden]

Thanks, I can see the Geo Map has appeared in the 'Hosts' menu.

So, no chance of using GeoIPupdate, updates mus be done manually ?? And yes, my CPU usage is too high...

Also, it's not Google Maps but OpenStreetMap (therefore I can't use any Google API key) and I should be located in the middle of Rome    but I'm actually living about 1,200 miles away...

Something it's not working, it's a shame...

[bungo63]

Dear GPB,
Thank you so much for providing these instructions. I have now got NETOPNG to display the geo mapping data.
I will write up a dummies guide on this for other users as the instructions that are provided I feel are "economic" and do not cater for the newbies.

So from this experience I have some queries that perhaps someone in the know can answer;
1. According to the OPNSense manual you have to cut and paste just one Maxmind  URL (which applies to Countries) into the firewall alias when it seems to me that two others are needed - ASN and cities)
2. According to the OPNSense manual the URL that is pasted into the points to CSV files. From GBP's instructions the mmdb files are the ones that work ( I can vouch for this).
3. Do CSV files work and if not then the wrong files re[ being pointed to?
4. The Netopng instructions are obviously for a vanilla install as the directory structure described does not apply to OPNSense, It would be useful if there was a document hat detailed the file locations perhaps?
5. The idea that creating the Firewall alias is "all these to it" is totally misleading. Dong this seems to achieve  nothing regardless of how many times you stop/start services or reboot the system.

[gpb]

Hey, no problem, I got it from a video on youtube quite some time ago, nothing I discovered myself.  I can't say for sure, but my assumption is with the CSV and database there are two packages, OPNsense and ntopng...the latter using the database format.  I found I needed all the files for ntopng to work.  I haven't looked at the new version, like I said I was getting 100% CPU using it and disabled it for the time being.  It looked like ntopng had a major revision, so maybe they have some bugs they're working through...just guessing.

Video here: https://youtu.be/MywgacMD_GI

[bungo63]

Dear GPB
I spent at least 10 hours trying to get his working. Most of  YouTube videos seemed focused upon the symptoms of the Maxmind licensing changes.
I only copied the EN databases across. I assumed they were all the same content just different languages.
I don't see the google map window in NTOPNG anymore. Have you got this working or maybe its no longer supported.
My CPU has increased but I have very little throughput so its no proving to be a problem.

AT the end of the day, whilst its interesting learning about Linux and stuff, paying $50 per year to Untangle for something that just works (for my basic needs)  may be a compelling argument.

[gpb]

I wouldn't blame OPNsense for this, it's my understanding it's the new version of ntopng.  I haven't looked at it other than when I was trying to understand how it changed for about 30 minutes a few weeks ago (initial reaction - not impressed)...then disabled it due to the high CPU.  Here's my post with some discussion about it.

https://forum.opnsense.org/index.php?topic=16679.0

EDIT: I shouldn't say not impressed, like @hushcoden said, it's openstreetmap now and not google, so no API needed.  It doesn't have the eye candy graphics, but I guess it's as functional.  I read what they did in 4.0, trying to open up for future development creating a more open model for people to contribute.  I should learn how to use it better lol...a tutorial about what it can do would be nice...probably something out there I'll see if I can find.

[hushcoden]

@gpb indeed I'm not impressed, but I would be happy if it would properly work: does it happen to you also it cannot find your location?

Like I said, it thinks I'm in Rome, don't understand why...

[gpb]

Oh, I didn't realize you were still having that problem.  No, it finds my location fine.  I think for that to work you have to allow your browser to report your location, that's how it used to be.  I probably gave it the permission for that in the past and haven't had to reset it.  I'm not 100% certain...but try that.

[RobLatour]

gpb: thank you for your post, your suggestion to

Copy files to /usr/local/share/ntopng/httpdocs/geoip/ (from MaxMind)

GeoLite2-ASN.mmdb
GeoLite2-Country.mmdb
GeoLite2-City.mmdb

Worked for me.   

It seems I did not need to do the following:

chmod +x /usr/local/opnsense/scripts/OPNsense/Ntopng/generate_certs.php

(or at least yet, as just copying the files worked for me).

In any case, thanks so much for posting - I had spent hours trying to figure this out following the documented instructions, but with no success until reading your post.