GeoIP with ipinfo stopped working ?

Started by Kayakero, Today at 02:45:36 PM

Previous topic - Next topic
I've been using ipinfo for GeoIP for months with no problems, updating daily, and yesterday it stopped updating

it was like this every day before

<165>1 2025-12-03T17:07:01-03:00 OPNsense-VPN.localhost firewall 6400 - [meta sequenceId="1"] found .gz format, process
<165>1 2025-12-03T17:07:20-03:00 OPNsense-VPN.localhost firewall 6400 - [meta sequenceId="2"] geoip updated (files: 496 lines: 5785121)

yesterday


<165>1 2025-12-04T17:08:03-03:00 OPNsense-VPN.localhost firewall 24737 - [meta sequenceId="1"] found .zip format, process
<163>1 2025-12-04T17:08:03-03:00 OPNsense-VPN.localhost firewall 24737 - [meta sequenceId="2"] geoip update failed : File is not a zip file
<165>1 2025-12-04T17:08:03-03:00 OPNsense-VPN.localhost firewall 24737 - [meta sequenceId="3"] geoip updated (files: 0 lines: 0)

downloading same configured url manually https://ipinfo.io/data/ipinfo_lite.csv.gz?token=xxxx with curl gives a 302 redirection and then goes to a real and good .gz

I don't understand ..



Best ask them directly.


Cheers,
Franco

Today at 02:52:03 PM #2 Last Edit: Today at 02:59:11 PM by Kayakero
No, I'm almost sure it always worked like this. with a redirection, I didn't mean that was the problem, was just explaining.

I tried to follow/debug the script but got lost ...
/usr/local/opnsense/scripts/filter/download_geoip.py

I don't understand why it says a zip instead of a gz like befoe and like it really is.

> geoip update failed : File is not a zip file

but the code didn't change? if they offer a gz file with a zip ending that's not good.


Cheers,
Franco

I'm seeing the same thing. From the timings this is before I updated to 25.7.9 from 25.7.8

Relevant logs from one of my firewalls.

Quote[...]
2025-12-05T11:42:13    Notice    firewall    geoip updated (files: 0 lines: 0)
2025-12-05T11:42:13    Error    firewall    geoip update failed : You have reached your 10 downloads per day limit for ipinfo_lite.csv.gz from [ip.ad.dr.ess] Please reach out to increase your limit via support@ipinfo.io. [http_code: 429]
2025-12-05T11:42:13    Notice    firewall    geoip updated (files: 0 lines: 0)
[...]
2025-12-05T11:42:01    Error    firewall    geoip update failed : File is not a zip file
2025-12-04T11:46:03    Notice    firewall    geoip updated (files: 0 lines: 0)
2025-12-04T11:46:03    Error    firewall    geoip update failed : You have reached your 10 downloads per day limit for ipinfo_lite.csv.gz from [ip.ad.dr.ess]. Please reach out to increase your limit via support@ipinfo.io. [http_code: 429]
[...]
2025-12-04T11:41:04    Error    firewall    geoip update failed : File is not a zip file
2025-12-04T11:41:03    Notice    firewall    geoip updated (files: 0 lines: 0)
2025-12-04T11:41:03    Error    firewall    geoip update failed : File is not a zip file
2025-12-04T11:41:01    Notice    firewall    geoip updated (files: 0 lines: 0)
2025-12-04T11:41:01    Error    firewall    geoip update failed : File is not a zip file
2025-12-03T11:40:08    Notice    firewall    geoip updated (files: 496 lines: 5785121)
2025-12-02T11:39:06    Notice    firewall    geoip updated (files: 496 lines: 4954678)
2025-12-01T11:38:05    Notice    firewall    geoip updated (files: 496 lines: 4951034)
[...]

> geoip update failed : File is not a zip file

but the code didn't change? if they offer a gz file with a zip ending that's not good.


Cheers,
Franco

It works just fine for me and there is a very simple explanation that hides behind this log line:

2025-12-05T11:42:13    Error    firewall    geoip update failed : You have reached your 10 downloads per day limit for ipinfo_lite.csv.gz from [ip.ad.dr.ess] Please reach out to increase your limit via support@ipinfo.io. [http_code: 429]

Ipinfo has a daily download limit that you have exceeded, probably because you use the same token on multiple OpnSense instances.
Intel N100, 4* I226-V, 2* 82559, 16 GByte, 500 GByte NVME, ZTE F6005

1100 down / 800 up, Bufferbloat A+

i am not using this product.  but i did sign up for it.   i stayed with Opnsense Business edition geoblocking

anyways.  my lite account says unlimited requests using the API access.

seems weird they would be blocking all of a sudden?

Today at 03:45:02 PM #8 Last Edit: Today at 03:46:40 PM by MoonbeamFrame
I'm using unique tokens on all firewalls.

I'm deducing that the maximum download exceeded is due to the firewall making multiple attempts to download the file, which matches the logs.

If I use curl to download the file from another location I see ?token=f2cbc8898bc30a appended to the filename.

I'm also seeing:

In order to use GeoIP, you need to configure a source in the GeoIP settings tab

When I go into the Firewall: Aliases





> I'm deducing that the maximum download exceeded is due to the firewall making multiple attempts to download the file

Yes, because it stopped being able to read the file yesterday:

2025-12-04T11:41:01    Error    firewall    geoip update failed : File is not a zip file
2025-12-03T11:40:08    Notice    firewall    geoip updated (files: 496 lines: 5785121)

Whether or not that's because of the update I doubt at this point. It seems circumstantial.


Cheers,
Franco

Quote from: DEC670airp414user on Today at 03:38:19 PMi am not using this product.  but i did sign up for it.  i stayed with Opnsense Business edition geoblocking

anyways.  my lite account says unlimited requests using the API access.

seems weird they would be blocking all of a sudden?

Look again.

Their API handles single IP queries and is unlimited, indeed.
The download of their database is limited as indicated by the error message.

And BTW: It works just fine for me.
Intel N100, 4* I226-V, 2* 82559, 16 GByte, 500 GByte NVME, ZTE F6005

1100 down / 800 up, Bufferbloat A+


Apart from the one firewall mentioned, all the other firewalls have now updated today without issue.

the only thing I can assume is that ipinfo removed the "Content-Disposition" header ( it's hosted in cloudflare it doesn't make sense ).
because in geoip.py it gets the name from there. and that header doesn't exist now, tested with curl verbose. I don't know how it was before.

I've forced the name ending in .gz in there so it goes thru the .gzip code instead of the zip code with
filename = "ipinfo_lite.csv.gz"
and it worked.


        if url is not None and url.lower().startswith('http'):
            # flush data from remote url to temp file and unpack from there
            with tempfile.NamedTemporaryFile() as tmp_stream:
                try:
                    r = requests.get(url)
                except Exception as e:
                    syslog.syslog(syslog.LOG_ERR, 'geoip update failed : %s' % e)
                    return result
                if r.status_code == 200:
                    msg = EmailMessage()
                    msg["Content-Disposition"] = r.headers.get("Content-Disposition", '')
                    filename = msg.get_filename()
                    syslog.syslog(syslog.LOG_NOTICE, 'filename : %s .' % filename)
                    filename = "ipinfo_lite.csv.gz"
                    tmp_stream.write(r.content)
                    tmp_stream.seek(0)
                    if not filename or filename.lower().endswith('.zip'):
                        syslog.syslog(syslog.LOG_NOTICE, 'found .zip format, process')
                        cls.process_zip(tmp_stream, result)
                    elif filename.endswith('.gz'):
                        syslog.syslog(syslog.LOG_NOTICE, 'found .gz format, process')
                        cls.process_gzip(tmp_stream, result)
                    # dump location hash (detect changes in geoIP source selection)
                    open(cls._src_hash_file, 'w').write(cls._source_hash())
                else:
                    syslog.syslog(syslog.LOG_ERR,
                                  'geoip update failed : %s [http_code: %s]' % (r.text.replace('\n', ''), r.status_code)
                    )