OPNsense Forum

English Forums => 25.7, 25.10 Series => Topic started by: Kayakero on December 05, 2025, 02:45:36 PM

Title: GeoIP with ipinfo stopped working ?
Post by: Kayakero on December 05, 2025, 02:45:36 PM
I've been using ipinfo for GeoIP for months with no problems, updating daily, and yesterday it stopped updating

it was like this every day before

<165>1 2025-12-03T17:07:01-03:00 OPNsense-VPN.localhost firewall 6400 - [meta sequenceId="1"] found .gz format, process
<165>1 2025-12-03T17:07:20-03:00 OPNsense-VPN.localhost firewall 6400 - [meta sequenceId="2"] geoip updated (files: 496 lines: 5785121)

yesterday


<165>1 2025-12-04T17:08:03-03:00 OPNsense-VPN.localhost firewall 24737 - [meta sequenceId="1"] found .zip format, process
<163>1 2025-12-04T17:08:03-03:00 OPNsense-VPN.localhost firewall 24737 - [meta sequenceId="2"] geoip update failed : File is not a zip file
<165>1 2025-12-04T17:08:03-03:00 OPNsense-VPN.localhost firewall 24737 - [meta sequenceId="3"] geoip updated (files: 0 lines: 0)

downloading same configured url manually https://ipinfo.io/data/ipinfo_lite.csv.gz?token=xxxx with curl gives a 302 redirection and then goes to a real and good .gz

I don't understand ..


Title: Re: GeoIP with ipinfo stopped working ?
Post by: franco on December 05, 2025, 02:49:16 PM
Best ask them directly.


Cheers,
Franco
Title: Re: GeoIP with ipinfo stopped working ?
Post by: Kayakero on December 05, 2025, 02:52:03 PM
No, I'm almost sure it always worked like this. with a redirection, I didn't mean that was the problem, was just explaining.

I tried to follow/debug the script but got lost ...
/usr/local/opnsense/scripts/filter/download_geoip.py

I don't understand why it says a zip instead of a gz like befoe and like it really is.
Title: Re: GeoIP with ipinfo stopped working ?
Post by: franco on December 05, 2025, 02:57:46 PM
> geoip update failed : File is not a zip file

but the code didn't change? if they offer a gz file with a zip ending that's not good.


Cheers,
Franco
Title: Re: GeoIP with ipinfo stopped working ?
Post by: MoonbeamFrame on December 05, 2025, 03:15:02 PM
I'm seeing the same thing. From the timings this is before I updated to 25.7.9 from 25.7.8

Relevant logs from one of my firewalls.

Quote[...]
2025-12-05T11:42:13    Notice    firewall    geoip updated (files: 0 lines: 0)
2025-12-05T11:42:13    Error    firewall    geoip update failed : You have reached your 10 downloads per day limit for ipinfo_lite.csv.gz from [ip.ad.dr.ess] Please reach out to increase your limit via support@ipinfo.io. [http_code: 429]
2025-12-05T11:42:13    Notice    firewall    geoip updated (files: 0 lines: 0)
[...]
2025-12-05T11:42:01    Error    firewall    geoip update failed : File is not a zip file
2025-12-04T11:46:03    Notice    firewall    geoip updated (files: 0 lines: 0)
2025-12-04T11:46:03    Error    firewall    geoip update failed : You have reached your 10 downloads per day limit for ipinfo_lite.csv.gz from [ip.ad.dr.ess]. Please reach out to increase your limit via support@ipinfo.io. [http_code: 429]
[...]
2025-12-04T11:41:04    Error    firewall    geoip update failed : File is not a zip file
2025-12-04T11:41:03    Notice    firewall    geoip updated (files: 0 lines: 0)
2025-12-04T11:41:03    Error    firewall    geoip update failed : File is not a zip file
2025-12-04T11:41:01    Notice    firewall    geoip updated (files: 0 lines: 0)
2025-12-04T11:41:01    Error    firewall    geoip update failed : File is not a zip file
2025-12-03T11:40:08    Notice    firewall    geoip updated (files: 496 lines: 5785121)
2025-12-02T11:39:06    Notice    firewall    geoip updated (files: 496 lines: 4954678)
2025-12-01T11:38:05    Notice    firewall    geoip updated (files: 496 lines: 4951034)
[...]
Title: Re: GeoIP with ipinfo stopped working ?
Post by: franco on December 05, 2025, 03:16:04 PM
> geoip update failed : File is not a zip file

but the code didn't change? if they offer a gz file with a zip ending that's not good.


Cheers,
Franco
Title: Re: GeoIP with ipinfo stopped working ?
Post by: meyergru on December 05, 2025, 03:30:50 PM
It works just fine for me and there is a very simple explanation that hides behind this log line:

2025-12-05T11:42:13    Error    firewall    geoip update failed : You have reached your 10 downloads per day limit for ipinfo_lite.csv.gz from [ip.ad.dr.ess] Please reach out to increase your limit via support@ipinfo.io. [http_code: 429]

Ipinfo has a daily download limit that you have exceeded, probably because you use the same token on multiple OpnSense instances.
Title: Re: GeoIP with ipinfo stopped working ?
Post by: DEC740airp414user on December 05, 2025, 03:38:19 PM
i am not using this product.  but i did sign up for it.   i stayed with Opnsense Business edition geoblocking

anyways.  my lite account says unlimited requests using the API access.

seems weird they would be blocking all of a sudden?
Title: Re: GeoIP with ipinfo stopped working ?
Post by: MoonbeamFrame on December 05, 2025, 03:45:02 PM
I'm using unique tokens on all firewalls.

I'm deducing that the maximum download exceeded is due to the firewall making multiple attempts to download the file, which matches the logs.

If I use curl to download the file from another location I see ?token=f2cbc8898bc30a appended to the filename.

I'm also seeing:

In order to use GeoIP, you need to configure a source in the GeoIP settings tab

When I go into the Firewall: Aliases




Title: Re: GeoIP with ipinfo stopped working ?
Post by: franco on December 05, 2025, 05:02:07 PM
> I'm deducing that the maximum download exceeded is due to the firewall making multiple attempts to download the file

Yes, because it stopped being able to read the file yesterday:

2025-12-04T11:41:01    Error    firewall    geoip update failed : File is not a zip file
2025-12-03T11:40:08    Notice    firewall    geoip updated (files: 496 lines: 5785121)

Whether or not that's because of the update I doubt at this point. It seems circumstantial.


Cheers,
Franco
Title: Re: GeoIP with ipinfo stopped working ?
Post by: meyergru on December 05, 2025, 05:38:03 PM
Quote from: DEC670airp414user on December 05, 2025, 03:38:19 PMi am not using this product.  but i did sign up for it.  i stayed with Opnsense Business edition geoblocking

anyways.  my lite account says unlimited requests using the API access.

seems weird they would be blocking all of a sudden?

Look again.

Their API handles single IP queries and is unlimited, indeed.
The download of their database is limited as indicated by the error message.

Title: Re: GeoIP with ipinfo stopped working ?
Post by: MoonbeamFrame on December 05, 2025, 05:46:35 PM

Apart from the one firewall mentioned, all the other firewalls have now updated today without issue.
Title: Re: GeoIP with ipinfo stopped working ?
Post by: Kayakero on December 05, 2025, 07:01:26 PM
the only thing I can assume is that ipinfo removed the "Content-Disposition" header ( it's hosted in cloudflare it doesn't make sense ).
because in geoip.py it gets the name from there. and that header doesn't exist now, tested with curl verbose. I don't know how it was before.

I've forced the name ending in .gz in there so it goes thru the .gzip code instead of the zip code with
filename = "ipinfo_lite.csv.gz"
and it worked.


        if url is not None and url.lower().startswith('http'):
            # flush data from remote url to temp file and unpack from there
            with tempfile.NamedTemporaryFile() as tmp_stream:
                try:
                    r = requests.get(url)
                except Exception as e:
                    syslog.syslog(syslog.LOG_ERR, 'geoip update failed : %s' % e)
                    return result
                if r.status_code == 200:
                    msg = EmailMessage()
                    msg["Content-Disposition"] = r.headers.get("Content-Disposition", '')
                    filename = msg.get_filename()
                    syslog.syslog(syslog.LOG_NOTICE, 'filename : %s .' % filename)
                    filename = "ipinfo_lite.csv.gz"
                    tmp_stream.write(r.content)
                    tmp_stream.seek(0)
                    if not filename or filename.lower().endswith('.zip'):
                        syslog.syslog(syslog.LOG_NOTICE, 'found .zip format, process')
                        cls.process_zip(tmp_stream, result)
                    elif filename.endswith('.gz'):
                        syslog.syslog(syslog.LOG_NOTICE, 'found .gz format, process')
                        cls.process_gzip(tmp_stream, result)
                    # dump location hash (detect changes in geoIP source selection)
                    open(cls._src_hash_file, 'w').write(cls._source_hash())
                else:
                    syslog.syslog(syslog.LOG_ERR,
                                  'geoip update failed : %s [http_code: %s]' % (r.text.replace('\n', ''), r.status_code)
                    )

Title: Re: GeoIP with ipinfo stopped working ?
Post by: Netlearn on December 06, 2025, 04:42:34 AM
In five different OPNsense, all of them on 25.7.9:

Screenshot 2025-12-06 at 04-36-55 General Log Files Firewall.png

Screenshot 2025-12-06 at 04-37-25 General Log Files Firewall.png

Screenshot 2025-12-06 at 04-37-40 General Log Files Firewall.png

Screenshot 2025-12-06 at 04-37-59 General Log Files Firewall.png
Title: Re: GeoIP with ipinfo stopped working ?
Post by: Netlearn on December 06, 2025, 04:44:19 AM
Screenshot 2025-12-06 at 04-38-18 General Log Files Firewall.png


Three of them have no problem, two of them don't seem to be happy with the file format.
Title: Re: GeoIP with ipinfo stopped working ?
Post by: IPinfo on December 06, 2025, 09:11:59 AM
Hi,

I am Abdullah, the DevRel of IPinfo. I will try my best to help you here.

Our file format will never change to anything else without a significant amount of communication. Please understand that a file format change without alerting our user base is a catastrophic change that we will never make. So, I suspect this is some error message being interpreted as zip file or something.

I am not clear about the implementation of how the database is downloaded, but we do have a checksums API endpoint that you should use to verify the download.

Reference: https://ipinfo.io/developers/database-download

The download process requires you to go through a redirect path because the data is stored in a cloud storage bucket.

Reference: https://ipinfo.io/developers/ipinfo-lite-database (See the code section)

The API provides unlimited usage, but the data downloads are subject to rate limits. It permits 10 downloads per unique IP address multiplied by unique access token. This means that to reach the rate limit, you probably downloaded it 10 times using the token you are using from the same IP address.

Reference: https://community.ipinfo.io/t/announcement-we-are-adding-rate-limits-to-data-downloads/358

You have shared your API access token: `f2cbc8898bc30a` which according to our database is a not an active or assigned token.

---

Please let me know if this problem persists. We will be happy to take a look. Our community forum is available here: https://community.ipinfo.io/

— Abdullah | DevRel, IPinfo
Title: Re: GeoIP with ipinfo stopped working ?
Post by: IPinfo on December 06, 2025, 09:13:58 AM
Quote from: Kayakero on December 05, 2025, 07:01:26 PMthe only thing I can assume is that ipinfo removed the "Content-Disposition" header ( it's hosted in cloudflare it doesn't make sense ).

Let us investigate this issue. I have escalated this to engineering.

— Abdullah | DevRel, IPinfo
Title: Re: GeoIP with ipinfo stopped working ?
Post by: IPinfo on December 06, 2025, 03:56:23 PM
Hi,

We were slowly migrating to a different cloud storage service over a period of time. We have rolled back the system migration entirely.

We have been doing incremental migration throughout the process, and thanks to the post and the only message we received on the support portal, we discovered reliance on the URI header metadata. This was not a hard transition, which has been rolled back, and further investigation will be conducted in the next few days.

During this transition, thanks to this post, we found out that the Content-Disposition header was no longer included in the final download response. This header was optional, but OpnSense relies on it to detect the filename or file type. We have been doing the slow rollout for several weeks, and this was the first issue we received.

The file itself has not changed; it's still a .csv.gz gzip file, but because the header is missing, some scripts may incorrectly treat it as a ZIP instead of a GZIP file. We have rolled back our entire deployment to our previous storage service.

We will investigate closely in the coming days. Thank you for understanding.

— Abdullah | DevRel, IPinfo
Title: Re: GeoIP with ipinfo stopped working ?
Post by: SteffenDE on December 06, 2025, 08:51:30 PM
Great to hear, thanks for the support.
Title: Re: GeoIP with ipinfo stopped working ?
Post by: Netlearn on December 07, 2025, 05:38:10 AM
Thanks for the fast response.

Updates work on the five units.
Title: Re: GeoIP with ipinfo stopped working ?
Post by: IPinfo on December 07, 2025, 06:14:07 AM
Thank you very much, everyone, for your kindness and patience. To confirm again:

- The engineering team performed the rollback, and we are now aware of the issue.
- @Kayakero's review helped us to pinpoint the issue.
- This was not related to rate limits. Due to multiple retries to the API endpoint we returned 429 rate limit error.
- We will proceed with the migration to the new cloud storage system; however, at this moment, we do not think Opnsense needs to patch anything. We will make the adjustments on our end.

If there is any issue, please reach out to us on our community: https://community.ipinfo.io/

Opnsense is a major supporter of our IPinfo Lite service, so we owe it to the Opnsense community to handle data issues related to us in our community.

— Abdullah | DevRel, IPinfo
Title: Re: GeoIP with ipinfo stopped working ?
Post by: craftnix on December 24, 2025, 09:50:02 AM
I just did a fresh installation and now see the same errors:

geoip update failed : File is not a zip file
geoip update failed : You have reached your 10 downloads per day limit for ipinfo_lite.csv.gz from x.x.x.x. Please reach out to increase your limit via support@ipinfo.io. [http_code: 429]

So seems like the issue has reoccurred or has not been fixed in certain regions.
Title: Re: GeoIP with ipinfo stopped working ?
Post by: IPinfo on December 24, 2025, 02:52:31 PM
Quote from: craftnix on December 24, 2025, 09:50:02 AMI just did a fresh installation and now see the same errors:

geoip update failed : File is not a zip file
geoip update failed : You have reached your 10 downloads per day limit for ipinfo_lite.csv.gz from x.x.x.x. Please reach out to increase your limit via support@ipinfo.io. [http_code: 429]



Hi,

I hope you are doing well. I am Abdullah, the DevRel of IPinfo.

Can you please reach out to our support team? They will investigate and report back. The issue should have been resolved as we engineered the migration several days ago and have not seen any issues.

— Abdullah | DevRel, IPinfo
Title: Re: GeoIP with ipinfo stopped working ?
Post by: craftnix on December 25, 2025, 02:58:49 PM
Hi Abdullah,

Looks like the issue has resolved itself now. The database was downloaded yesterday at 21:00 my time. I'll monitor if the next update succeeds also.

Other people seem to have had the same issue where it might take a day or two for the download to succeed.
Title: Re: GeoIP with ipinfo stopped working ?
Post by: IPinfo on December 28, 2025, 04:48:24 PM
Thank you very much. Keep me informed of any additional issues.

— Abdullah | DevRel, IPinfo