GeoIP updater repeatedly hits MaxMind after failed refresh – OPNsense 26.7.3_8

Started by almapavi, Today at 07:28:46 PM

Previous topic - Next topic
I investigated this further and I believe I have identified the sequence causing the MaxMind daily download limit to be reached.

OPNsense version:
OPNsense 26.7.3_8 (amd64)

The GeoIP database itself was present and appeared healthy:

/usr/local/sh
are/GeoIP/alias.stats
/usr/local/share/GeoIP/alias/NL-IPv4
...other country files...

NL-IPv4:

size=308932 bytes
modified=2026-08-30 13:50:03

The issue started approximately 24 hours later, when the GeoIP database became eligible for refresh.

The first GeoIP error was not a MaxMind quota error. It was:

geoip update failed :
<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>
The request signature we calculated does not match the signature you provided.
Check your secret access key and signing method.
</Message>
...
</Error>
[http_code: 403]

The failed request was against MaxMind's redirected Cloudflare R2/S3-compatible storage URL.

After this initial 403 SignatureDoesNotMatch, OPNsense started making GeoIP download requests extremely rapidly. MaxMind's download history showed close to one request per second.

Shortly afterward, OPNsense started reporting:

geoip update failed :
Daily GeoIP database download limit reached
[http_code: 429]

There were many of these within only a few seconds.

For example:

2026-08-31T13:51:50  HTTP 429
2026-08-31T13:51:50  HTTP 429
2026-08-31T13:51:51  HTTP 429
2026-08-31T13:51:51  HTTP 429
2026-08-31T13:51:52  HTTP 429
2026-08-31T13:51:52  HTTP 429
2026-08-31T13:51:53  HTTP 429
...

This does not appear to be caused by an aggressive GeoIP cron schedule.

The firewall also has normal hostname aliases that resolve periodically, for example:

resolving 2 hostnames for stats_reportcpanel_com
resolving 2 hostnames for ai_reportcpanel_com
resolving 3 hostnames for AllowList

These run approximately every six minutes.

Based on the GeoIP code behavior, the likely sequence appears to be:

GeoIP database becomes older than ~24 hours
        ↓
OPNsense determines GeoIP update is required
        ↓
MaxMind download attempted
        ↓
Download fails with HTTP 403 SignatureDoesNotMatch
        ↓
Existing NL-IPv4 file remains older than 24 hours
        ↓
Next GeoIP lookup still determines that an update is required
        ↓
Another MaxMind download is attempted immediately
        ↓
Failure
        ↓
Another GeoIP lookup
        ↓
Another download
        ↓
...
        ↓
MaxMind download quota exhausted
        ↓
HTTP 429

The important issue appears to be that there is no effective retry delay/backoff after a failed GeoIP download.

A temporary upstream failure therefore causes every subsequent GeoIP lookup/update operation to attempt another complete database download.

That explains why a single failed daily refresh resulted in dozens of MaxMind requests in a very short period.

As a temporary workaround, I updated the modification timestamp of the existing valid GeoIP sentinel file:

touch /usr/local/share/GeoIP/alias/NL-IPv4

This does not change the GeoIP ranges themselves, but it prevents OPNsense from considering the existing database stale and repeatedly attempting another download until the MaxMind quota clears.

It would probably be beneficial for the GeoIP updater to implement something such as:

Failed GeoIP update
        ↓
record last attempt time
        ↓
do not retry for X minutes/hours

or otherwise ensure that only one GeoIP database download attempt can occur during a single alias-update process.

A temporary MaxMind/provider failure should ideally generate one error and continue using the existing GeoIP database rather than repeatedly downloading until the account rate limit is reached.

At this point the evidence suggests:

The existing GeoIP database was valid.
The normal ~24-hour refresh became due.
The initial MaxMind/R2 download failed with 403 SignatureDoesNotMatch.
OPNsense immediately retried many times.
Those retries exhausted the MaxMind GeoLite daily download allowance.
Subsequent requests returned HTTP 429.

Hopefully this helps reproduce the issue.