ntopng: geolocation

[juggie]

Hey All,

geolocation does not seem to be working with ntopng plugin.  I've manually ran the download scripts etc, but to no avail, all ips are still non geolocated.  The status of ntopng does show that maxmind 1.3.2 is compiled in, and the maxmind files are on my opensense system.

I've run both of the following, but no positive impact after a restart of ntopng:
root@OPNsense:/var/tmp/ntopng # /usr/local/bin/geoipupdate.sh
Fetching GeoIP.dat and GeoIPv6.dat...
/usr/local/share/GeoIP/GeoIPupdate.U6v09v/GeoI100% of  694 kB 4043 kBps 00m00s
/usr/local/share/GeoIP/GeoIPupdate.Xwug8W/GeoI100% of 1187 kB 3953 kBps 00m00s
root@OPNsense:/var/tmp/ntopng # /usr/local/bin/ntopng-geoipupdate.sh
Fetching GeoLiteCity.dat...
-                                             100% of   12 MB 8199 kBps 00m02s
Fetching GeoLiteCityv6.dat...
-                                             100% of   14 MB 6580 kBps 00m02s
Fetching GeoIPASNum.dat...
-                                             100% of 2475 kB 5326 kBps 00m00s
Fetching GeoIPASNumv6.dat...
-                                             100% of 3006 kB 5775 kBps 00m01s


Any ideas?

Edit:  They switched geo location libs in 3.6 so I assume this is related.

[juggie]

I figured this out.

The code for ntopng 3.6 now looks for the following 2 files:
root@OPNsense:/usr/local/share/ntopng/httpdocs/geoip # ls -al *.mmdb
-rw-r--r--  1 2000  2000   6297816 Dec 18 10:26 GeoLite2-ASN.mmdb
-rw-r--r--  1 2000  2000  61305552 Dec 18 15:33 GeoLite2-City.mmdb

which come from:
https://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz
https://geolite.maxmind.com/download/geoip/database/GeoLite2-ASN.tar.gz

None of the scripts/tools to auto update the db's deal with these new file formats.

[deekdeeker]

Thank you this fixed for me !

J

[mimugmail]

So you downloaded them manually, put them in the specified folder and then it worked?
What about this ntopng update script? Is there a way to specify this folder?

[deekdeeker]

Correct, im not going to make a script as i believe this will be fixed in a upcoming release...

[juggie]

So you downloaded them manually, put them in the specified folder and then it worked?
What about this ntopng update script? Is there a way to specify this folder?

Correct.  /usr/local/bin/ntopng-geoipupdate.sh will need to be updated to do the right thing.

[mimugmail]

Ok, so we'll wait for 3.8 and if the error still exists can you open an issue in github?

[jkemp]

Thanks, juggle that got it working for me also

[nickjnaude]

Here ya go 

Code: [Select]

#!/bin/sh

set -eu
mkdir -p "/usr/local/share/ntopng/httpdocs/geoip"

# arguments:
# $1 URL
# $2 output file name
_fetch() {
    url="$1"
    out="$2"
    TEMPFILE="$(mktemp "/usr/local/share/ntopng/httpdocs/geoip/GeoIP.dat-XXXXXX")"
    trap 'rc=$? ; set +e ; rm -f "'"$TEMPFILE"'" ; exit $rc' 0
    if fetch -o - "$url" | tar -x --strip-components 1 -f - "*/$out" >> "$TEMPFILE" ; then
        chmod 444 "$TEMPFILE"
        if ! mv -f "$TEMPFILE" "/usr/local/share/ntopng/httpdocs/geoip/$2" ; then
            echo "Unable to replace /usr/local/share/ntopng/httpdocs/geoip/$2"
            return 2
        fi
    else
        echo "$2 download failed"
        return 1
    fi
    rm -f "$TEMPFILE"
    trap - 0
    return 0
}

echo Fetching GeoLite2-City.tar.gz...
_fetch "https://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz" GeoLite2-City.mmdb

echo GeoLite2-ASN.tar.gz...
_fetch "https://geolite.maxmind.com/download/geoip/database/GeoLite2-ASN.tar.gz" GeoLite2-ASN.mmdb


[franck]

Hello,

Could you tell me what md5sum you have for your file GeoLiteCity.dat
I would like to know if I have the last free version (from april 2018)
Mine is a4c35dca1735c4ec151b82f26d0d1115

As you donwloaded in december, I guess you have the last before been removed from download by Maxmind

Thanks