Work around high miss ratio of maxmind geoip

[franco]

https://github.com/opnsense/core/issues/7779 ;)

[Jyling]

That may address the issue of the accuracy but does not explain why an inaccurate location passed the block filter. According to MaxMind, the last subnet was in Thailand. I block that country. They made the connection. They should have been blocked.

[meyergru]

I have never experienced that a geoblock IP was not blocked when I correctly set it. Are you by any chance trying fancy stuff with aliases like excluding something from the geoip list, like discussed here?

That being said, the Maxmind geoip lite database that is normally used is not quite the same as what is presented here:

https://www.maxmind.com/en/geoip-demo

If you enter 45.201.11.0 as IP into that demo, you will be shown a network of 45.201.10.0/23, which is not actually separate in the downloaded Maxmind set, which gives:

Code Select Expand

45.201.0.0/21,1605651,934292,,0,0,
45.201.8.0/22,1605651,934292,,0,0,
45.201.12.0/24,798544,934292,,0,0,
45.201.13.0/24,2661886,934292,,0,0,
45.201.14.0/24,2623032,934292,,0,0,
...

However, 45.201.11.0 is still in Thailand, so if it is not blocked for you, something else must be wrong (see point 1 below).



On a side note, the linked issue 7779 also has a patch linked, that makes IPinfo.io available as an alternative provider for added accuracy (their database has almost 10 times more entries than Maxmind's). However, for the time being, there are two problems with this:

1. Your firewall alias table size will have to be increased significantly or you will get problem. I had to enlarge it to 50.000.000 entries, instead of the default ~1.000.000. Even with just Maxmind, the default size will probably not suffice.

2. When you do that, the hidden cron job that updates the aliases will take extremely long to run on a normal CPU (for me: ~15 seconds on an N100) . The actual table update that is called within the script seems to be a blocking operation, such that you will experience latency spikes of a range in hundreds of milliseconds (for me, it was 1.5s). This will delay packets, such that real-time traffic like VoIP will drop out every 60 seconds.

Franco said that this feature will therefore not find its entry into the next release 25.7.2.

[BrandyWine]

The network 45.201.11.0/24 is in Berlin, Germany, but MaxMind places it in Thailand. The latter is blocked by my firewall alias, but they are still able to make connections. So, something is very, very wrong with both MaxMind and its GeoIP implementation in open sense. I cannot trust it after all of these breaches.

Where are you getting your GEO IP info from? What's telling you 45.201.11.0/24 is physically in Berlin?
Side example, CONTABO has registered block in Germany but they moved a datacenter to France, so it's more appropriate to GEO it in France, that's where the IP are.

Maxmind appears to be correct for 45.201.11.0/24:

45.201.11.0 - 45.201.11.255 is an IP address range owned by 3xK Tech GmbH and located in Thailand

Registered blocks do not mean that's where they are physically located, etc.

What meyergru said, if you block Thailand and it wasn't blocked, then something else is the cause.

[meyergru]

At least, the IP 45.201.11.0 is in Berlin, unless the laws of physics are out of order...

You cannot view this attachment.

When you look at https://inventivehq.com/network-latency-calculator/, you will find that 0.38ms corresponds to a theoretical maximum distance of ~100km between the probe and the IP location, less is more likely.

Also, Maxmind's geoip lite did not discriminate the 3xK Tech ranges 45.201.10.0/24 and 45.201.11.0/24. because, as I showed, they list only 45.201.8.0/22, and there are ranges in it that do not belong to 3xK Tech. I guess that "geoip lite" does indeed indicate less resolution than their commercial offerings, as I also pointed out.

[Jyling]

The actual IP is in MaxMind. I post the subnet. Speculations aside, it has to be blocked, but it passes.

[Patrick M. Hausen]

Then please show the block rule in detail.

[marjohn56]

https://github.com/opnsense/core/issues/7779 ;)

And it seems to be working nicely...

[meyergru]

Yes, apart from the two mentioned problems...

[BrandyWine]

At least, the IP 45.201.11.0 is in Berlin, unless the laws of physics are out of order...

0.38ms is fast.
Global latency does appear DE is fastest to dst IP
https://tools.bunny.net/latency-test?query=45.201.11.2

9 of 10 geo location services show 45.201.11 in Bangkok.
https://www.iplocation.net/ip-lookup

If it's in Berlin then why are so many geo services getting it wrong?

[marjohn56]

Yes, apart from the two mentioned problems...

They are not issues that affect my system. Lots of memory to spare and a decently quick CPU. I had an issue with Amazon web services being blocked, even though they were in Ireland and that is allowed. I had to put in a separate firewall entry to allow them. Changing to IPInfo has meant I no longer need that extra rule.

[franco]

That may address the issue of the accuracy but does not explain why an inaccurate location passed the block filter. According to MaxMind, the last subnet was in Thailand. I block that country. They made the connection. They should have been blocked.

That's not what the initial text said. If you cannot be sure about accuracy how can you sure about inconsistent blocking?

[Jyling]

If you cannot be sure about accuracy how can you sure about inconsistent blocking?

For instance, like I wrote 2ce already, the country that MaxMind places the IP in is enabled on the alias that is used in the  block rule. I tested it and determined that it works, using my test addresses, but when it comes to actual production it sometimes allows the connection that is supposed to be blocked.
I have no way of knowing whether the IP could have changed its location in MaxMind DB between the moment in time when a hacker tries to break in and when I check MaxMind, which is one plausible excuse, but this happens sometimes.

[franco]

> it sometimes allows the connection that is supposed to be blocked

This hinges on 3 factors:

1. current alias state
2. frequency of updates
3. particular IP in question

It's easy to test your theory if you adhere to these 3 constants, because I doulbt as long an 1. and 3. are known and good that it won't work, which only leaves 2 which means the database changes WRT to a particular address, but you can even test this over time easily.

Taking an IP address for 3 and determining it should have been blocked misses point 1. and 2.

Also, if a connection is allowed and not timed out it would likely also persist when it becomes available in the database (although it's suspicious you'd have a multi-day connection to an IP address, but that's a different issue).


Cheers,
Franco

[Jyling]

It seldom takes us longer than 8 hours between an attack report and the check.
I doubt MaxMind updates in the mean time.