OPNsense Forum

Archive => 18.1 Legacy Series => Topic started by: ekke on February 07, 2018, 12:02:23 am

Title: /usr/local/etc/bogonsv6 too big
Post by: ekke on February 07, 2018, 12:02:23 am
(https://www.dropbox.com/s/7tswaldhlmkdgyg/opnsense.PNG?dl=0)
https://www.dropbox.com/s/7tswaldhlmkdgyg/opnsense.PNG?dl=0

ll /usr/local/etc/bogonsv6
-rw-r--r--  1 root  wheel  1492763 Jan 30 23:09 /usr/local/etc/bogonsv6

I get a error when OPNSense tries to load this list. for now I have commented it out.

Title: Re: /usr/local/etc/bogonsv6 too big
Post by: phoenix on February 07, 2018, 07:24:48 am
I can't imagine why it's that large, mine isn't:

ll /usr/local/etc/bogonsv6
-rw-r--r--  1 root  wheel  860 Feb  1 11:58 /usr/local/etc/bogonsv6

Perhaps your is a hangover from the last release of OPNsense that you had installed or is this a clean install?
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: franco on February 07, 2018, 02:14:44 pm
It's normally over 1 MB once updated. The small file is the bootstrap file we ship with the core package.

If bogons are too big, you will have to disable them. Never heard of it before, half-guessing it is due to your hardware... how much RAM do you have?


Cheers,
Franco
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: marjohn56 on February 07, 2018, 02:50:39 pm
Here's mine for comparision

admin@gateway:~ % ll /usr/local/etc/bogonsv6
-rw-r--r--  1 root  wheel  1508542 Feb  1 03:16 /usr/local/etc/bogonsv6
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: mausy5043 on February 07, 2018, 04:46:04 pm
I can't imagine why it's that large, mine isn't:

ll /usr/local/etc/bogonsv6
-rw-r--r--  1 root  wheel  860 Feb  1 11:58 /usr/local/etc/bogonsv6

There is a *really* well-hidden option in Firewall > Diagnostics > pfTables
The button in the top-right corner : [Update bogons]. When clicked it downloads the latest list of bogons. Which can be fairly large.

Code: [Select]
$ la /usr/local/etc/bogons*
-rw-r--r--  1 root  wheel    62572 Feb  7 16:44 /usr/local/etc/bogons
-rw-r--r--  1 root  wheel      132 Jan 29 13:12 /usr/local/etc/bogons.sample
-rw-r--r--  1 root  wheel  1514021 Feb  7 16:44 /usr/local/etc/bogonsv6
-rw-r--r--  1 root  wheel      860 Jan 29 13:12 /usr/local/etc/bogonsv6.sample

And looking at my logs I see nothing special.

Code: [Select]

Feb  7 16:43:45 gateway configd.py: [dc6fa705-7811-47fd-a2d1-9c4bbc11a04b] request content of pf bogons table
Feb  7 16:44:04 gateway configd.py: [b4dfd709-84b2-41da-bc55-e26c2bab476f] update bogons database
Feb  7 16:44:04 gateway root: rc.update_bogons is starting up
Feb  7 16:44:04 gateway root: rc.update_bogons is beginning the update cycle
Feb  7 16:44:05 gateway root: rc.update_bogons is ending the update cycle
Feb  7 16:44:05 gateway configd.py: [2edfb7f2-a740-488b-a2e4-0aee5e383c64] request content of pf bogons table
Feb  7 16:46:22 gateway configd.py: [3f82124c-4eca-4186-b412-d5e27172e084] request content of pf bogonsv6 table

I tend to agree with @franco that this might be a memory problem.
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: phoenix on February 07, 2018, 07:18:01 pm
It's normally over 1 MB once updated. The small file is the bootstrap file we ship with the core package.
My mistake.  :-[ Doesn't that file get updated automatically? I always assumed that was the case, I've just updated it manually via the helpful hint from mausy5043 and it's now the 1.5MB size.
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: franco on February 07, 2018, 08:49:52 pm
Bill, do you have IPv6 turned off under Firewall: Settings: Advanced?


Cheers,
Franco
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: phoenix on February 07, 2018, 08:54:49 pm
Hi Franco

Yes, IPv6 is enabled and has been since I went to fibre in August last year.
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: franco on February 07, 2018, 08:58:47 pm
Hmm, and do you set "block bogons" anywhere in your interfaces?

Also in your system log do you see "Not saving IPv6 bogons table (IPv6 Allow is off and table-entries limit is potentially too low" ?


Cheers,
Franco
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: ekke on February 07, 2018, 09:13:05 pm
It's normally over 1 MB once updated. The small file is the bootstrap file we ship with the core package.

If bogons are too big, you will have to disable them. Never heard of it before, half-guessing it is due to your hardware... how much RAM do you have?


Cheers,
Franco

xeon 1230v3 8GB RAM
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: phoenix on February 07, 2018, 09:17:27 pm
Hmm, and do you set "block bogons" anywhere in your interfaces?
Yes, it's set on the WAN interface only and nowhere else.

Also in your system log do you see "Not saving IPv6 bogons table (IPv6 Allow is off and table-entries limit is potentially too low" ?
Nothing except the fact I started the update and the file did get updated.
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: franco on February 07, 2018, 09:31:41 pm
Weird, but out of ideas. :(
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: phoenix on February 07, 2018, 09:44:08 pm
Weird, but out of ideas. :(
No worries, I didn't miss it when it wasn't running. :)
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: Ren on February 07, 2018, 10:05:54 pm
i had the same issue after updating last night. I ended up bumping the Firewall Maximum Table Entries to 500,000 and rebooting. I can spare 500MB of RAM for this.

(https://image.ibb.co/feZNGH/Firewall_Maximum_Table_Entries.png)
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: marjohn56 on February 07, 2018, 10:39:32 pm
Perhaps why I have never had the issue, my FW Max Entries was already set to 1 Million. Plenty of RAM to play with.
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: phoenix on February 08, 2018, 08:11:39 am
Hmm, and do you set "block bogons" anywhere in your interfaces?

Also in your system log do you see "Not saving IPv6 bogons table (IPv6 Allow is off and table-entries limit is potentially too low" ?
Franco

Just an update on this. I'd set-up a test server on a separate VM. It's a clean install of 18.1and "Block bogon networks" is enabled on the WAN link but the file hasn't been updated:

Code: [Select]
ll /usr/local/etc/bogonsv6
860 Feb  1 11:58 /usr/local/etc/bogonsv6
There's nothing in the logs until I do the update and the file updates correctly.
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: franco on February 08, 2018, 08:36:48 am
Hmmm, okay. Let's leave it as is then. :)


Cheers,
Franco
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: cardins2u on April 06, 2018, 05:50:33 am
I'm getting this same errors .

anyone found a fix yet?

@franco
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: franco on April 07, 2018, 05:46:07 pm
It doesn't appear to be a real bug. The bogons size seems to fluctuate via http://www.team-cymru.org/Services/Bogons and that's causing this long known issue in the state table where IPv6 is said to be too large, but up until now we never observed it in OPNsense.

Here's a recent pfSense ticket observing / addressing the same:

https://redmine.pfsense.org/issues/8417

I'm not sure what the best approach is for us yet.


Cheers,
Franco
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: marjohn56 on April 07, 2018, 06:35:22 pm
Is it not just a case of increasing the maximum table entries?

I have never seen this issue, but one of the first things I needed to do when setting up Opnsense was to increase the max table entries.
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: Reiter der OPNsense on April 07, 2018, 08:39:08 pm
I agree with marjohn56. In my opinion this is not a bug, but you shouldn't just ignore it. The bogonsv6 seems to be simply too big and will probably not be processed correctly. I got this error message during my IPv6 experiments on two boxes. The obvious thing would be to increase the corresponding standard value from 200,000 to a reasonable size. In my case, the value of 500,000 makes the error message disappear.
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: NOYB on April 07, 2018, 10:11:20 pm
Not setting the maximum table entries to an appropriate size when bogons v6 is enabled is a bug in my opinion.  Whether or not some other product does is irrelevant.

Been running maximum table entries at 1,000,000 "forever" precisely for this reason.  But it should be set to an appropriate size automatically when/if bogons v6 is enabled.
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: marjohn56 on April 07, 2018, 11:15:13 pm
I go with NOYB, 200K for V4 only and bounce it to 500K if bogons v6 is enabled.

I also run 1,000,000 as it happens but then my proc can cope with it. I have a feeling I was running 500K when I used an APU2.

@Franco - would you be happy with that?

I'm happy to look at doing a PR for it if no-one else wants to.
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: elektroinside on April 08, 2018, 12:17:47 am
It's not a bug, it's a feature :D It's just missing, so the actual bug would be that it is missing this feature :D
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: franco on April 08, 2018, 11:55:18 am
Bug or feature, all contributions are welcome ;)

...or try this then...

https://github.com/opnsense/core/commit/fc0c66e8
https://github.com/opnsense/core/commit/5dd172e


Cheers,
Franco
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: marjohn56 on April 08, 2018, 12:55:31 pm
Nice one...
Title: Re: /usr/local/etc/bogonsv6 too big
Post by: elektroinside on April 09, 2018, 09:11:19 am
There you go, this is what i call support :)
Thank you Franco!