Messages

> IIUC, someone created this thread when he found that he can not
  enable or disable firewall rules after an upgrade anymore, and was
  advised to install a plug-in to fix that.

Which is correct, but that's in the release notes. You don't need the
plugin to use and/or migrate the rules, but you need the plugin to
administrate them.

It's kinda in the release notes: 'Kinda' because it's very well
hidden.  I think it would be something that should be pointed out in
the 'Migration notes, known issues and limitations' section.

If someone doesn't read that section it's his fault.  I wouldn't blame
anyone for not reading the long list of items most or all of which
don't tell users anything.

> but I made the experience that the firewall rules didn't work after
  the migration and had all to be redone

Which is entirely different.  It may be because of FreeBSD 15.1 or
something else.  It looks like we don't know?

Different from what?

It's not because of FreeBSD.  It's because things work differently
with the new rules so that the old ones can not easily be converted
into new ones.  I guess that there may be configurations for which the
migration works: That's only when no conversion is needed.  That
doesn't mean it would work for everyone, and it means that a big fat
warning is needed.

I'm basically using a configuration that goes as described in this post:
https://forum.opnsense.org/index.php?topic=28447.msg138309#msg138309

If you want to use IPv6 and don't have a static prefix, what other
solution is there?  It's working great, and it doesn't migrate.

> Don't continue to pretend that it is an easy and harmless migration
  that wouldn't even require a big fat warning.  It is not.

I never pretended it was easy and harmless for you.  This is the core
of the issue.  It is your experience.  Not everyone else's.

I'm sure I'm not the only one to find out that the migration wasn't
easy and harmless.  Even if I'm the only one to experience that it
wasn't, it does warrant a big fat warning because there can always be
other users for which it won't be harmless and easy.

The core issue is that the migration was made to appear as being
harmless and easy and that the GUI suggested that we better do it
soon, and that we were not informed that we might have to redo our
firewalls and that we don't need to do the conversion anytime soon.

You right now still deny that it is not harmless and easy by
claiming that I'd be the only one for whom it wasn't.  Even if I am
the only one, it proves that your assumption that it is harmless and
easy is wrong.

> I wonder why that is.

Because you don't know what the problem is, but feel entitled to raise
your concern in advance?  If I know your exact problem I can probably
help or point to the right bits of documentation.

I'm not raising my concern in advance.

I did the migration because it was made to appear easy and harmless a
while after there was the entry for the new rules in the GUI,
suggesting that I better migrate my rules as long as it is
possible. Living in the past seldwhen is a good idea, meaning that
things like software keep moving on, and when you don't keep up you
can get so far behind that you eventually find that updating isn't
possible anymore. I've had that happen and it's a situation I really
don't want to be in again.

I'm raising my concern only after I know what the problem is.  The
problem is that the migration can easily go wrong, and my concern is
that there needs to be a big fat warning about it instead of making it
appear easy and harmless.

I don't know exactly in detail what causes the problem, just see
above.  I don't expect you/the developers to fix the cause(s) so that
the problem doesn't exist anymore.  The big fat warning would suffice,
so everyone who planning to do the migration can make backups and set
aside time to try it out and do whatever they deem necessary
beforehand.

The user's approach needs to be 'the migration may go wrong and is
not necessary' instead of 'the migration is harmless and easy and
should be done as soon as possible'.

> However, if you use the Community Edition, I believe you also bear
  some responsibility for regularly reading forum posts; otherwise,
  you might find yourself facing a problem eight months down the line
  that has already been thoroughly discussed and addressed. This
  applies equally to experienced forum members and OPNsense newcomers.

I did that until you drove me away.

Besides, users would have to not only read every post just in case
they might have a problem some months later but also remember it all.
I think that is demanded way too much.

It's like saying that every user of whatever software needs to read
everything about it in case he encounters a problem.

It doesn't work that way.  When I'm encountering a problem, I try to
solve it myself.  Asking questions anywhere is a last resort when
everything else fails.  The internet has mutated from a rather
friendly and helpful domain into the most hostile environment I
know. I don't want to ask questions because it almost always only
stirrs up useless and stupid discussions in which people try to insult
me, call me a troll and censor me.

Even if I wanted to read everything just to keep up to date, every day
would have to be at least a week long.

What if someone uses the business edition?

> Sometimes perhaps, but I'm surprised at the trivial questions that
  are being raised seeing the code that was touched and people not
  realising how many technical issues they would never notice because
  a) they don't have the setup or b) changes are handled in a way that
  few regressions arise to begin with.

And yet you expect these people to read every forum post to be
informed and to remember all the technical issues, regressions and
setups they never encounter or have.

Maybe if the 'Migration notes, known issues and limitations' section were
more exensive and there were only a link to the (rather irrelevant)
long list of items, then people wouldn't have so many trivial
questions.

> A great example is the Intel microcode plugin that if I had read the
  forum properly would have known I should have removed it before
  upgrading to 26.7.

> It's a very complex issue involving the microcode updates and the
  operating system that has been going on for years now -- and both
  are not under our immediate influence.  And here, also, it was
  clearly in the release notes:

https://github.com/opnsense/changelog/blob/b0be678d6235cb8da05446df7ea233c38cdcd0a7/community/26.7/26.7#L101

I guess you mean this:


"The CPU microcode early loading has been known to be flaky on some
setups.  A fix is in the FreeBSD 15.1 boot loader code, but can only
be reached by reinstall or manually updating the boot code of your
system after the upgrade succeeded.  If you want to be on the safe
side during the upgrade itself please remove the plugin before
proceeding."


I still don't know which plug-in this is referring to and how I would
update the boot code.

I can only guess that 'boot code' means the boot manager.  What
exactly am I supposed to do after reading this when I'm about to
update?  Why isn't the boot manager being updated automatically during
the update?

What I might do is figure out how one updates the boot manager of
FreeBSD.  But then, it would be pretty pointless because OPNSense
might be different from FreeBSD, so that might not work.  I wouldn't
take that chance.

I'm finding this also unclear.  It says I should manually update the
'boot code' (boot manager?) *after* updating.  That doesn't make sense
because if this goes wrong, booting wouldn't be possible.  So
obviously, I would need to update the 'boot code' before updating.

So maybe I better remove a plug-in first.  But which one?

Trivial questions?  Maybe, maybe not.  It's certainly not trivial when
I update and the router doesn't boot anymore.

The update shouldn't even start before that issue is somehow resolved
first.  Or perhaps I'm understanding this wrong and nothing can go
wrong when I update because the information is entirely unclear?

Why are the release notes not starting with the important section?

> Apparently the rules didn't continue to work.

You're convoluting your experience with the technical facts in this particular case. It's futile to go into an argument like that.


Cheers,
Franco

I'm not convoluting anything.  IIUC, someone created this thread when he found that he can not enable or disable firewall rules after an upgrade anymore, and was advised to install a plug-in to fix that.  I haven't made this experience, but I made the experience that the firewall rules didn't work after the migration and had all to be redone.  After the painful experience, I was told that it would take many years to come before the migration would be necessary.  In spite of that, now apparently everyone who wants to be able to enable or disable firewall rules needs to do some kind of migration, even if it is only to install the plug-in.

Don't continue to pretend that it is an easy and harmless migration that wouldn't even require a big fat warning.  It is not.

This is not about technical details but about the user experience and about improving it.  From the technical details being clear to the developers, it doesn't follow that users are sufficiently informed even when they read the release notes.  Apparently, this very argument is futile here.  I wonder why that is.

The community has spoken. My contribution is that I am not recommending 26.7 for my clients. I'm running 26.1 in home lab.

What do you recommend instead?

https://github.com/opnsense/changelog/blob/master/community/26.7/26.7#L98

I don't see how that is unclear.  But there have been at least a dozen questions.


Cheers,
Franco

"All rules will continue to work regardless of the plugin being installed or not and are easily migrated using the given assistant."

Apparently the rules didn't continue to work.  That they are 'easily migrated' is definitely wrong.

"If you want to be on the safe side during the upgrade itself please remove the plugin before proceeding."

Which plugin is that?

Other than that, I'm finding the section with the 'Migration notes, known issues and limitations' the most important and relevant part of these release notes.

Thanks!  I don't think it can be unassigned, see pictures.

It seems to be working fine --- but is/was it a good idea to change the configuration file from the console first?

It seems that the instruction to do so may be a remnant from when there was no way to configure vnstat through the GUI.  Perhaps it interferes?

[!!! Upgrade your Bootloader after upgrading to 26.7 !!!]

And what is 'config.xml'?

Ehh... seriously... ?!?!

It's the file that holds your complete OPNsense configuration and can be easily downloaded for backup purposes via the webGUI so you can restore it in the current or new installation of OPNsense ;)

Yeah I download the configuration every now and then, but the file has a pretty long name I don't pay a lot of attention to other than always saving it in the directory designated for it.  Besides, 'config.xml' is a very generic name.  What do you think how many files with that name do you have?

And what are these items


o firewall: move config.xml default LAN allow rules to new rules GUI
o firewall: legacy rules pages move to plugin


in the change log supposed to tell me?  I'm not familiar with the format and the contents of a file with the generic name 'config.xml'.  I don't know where and how firewall rules are stored.  What's with 'default LAN'?  That entry doesn't tell me anything because it's incomprehensible and meaningless to me.

What are the 'legacy rules pages'?  I happen to have an idea because I migrated my rules.  After migrating, there were still the automatic rules left and I didn't dare to delete those.  Now I can't see them anymore, if they are still there.  Will they stay forever now?  If I hadn't migrated, that entry would also be completely meaningless to me.

Where do the release notes tell me that I can't en-/disable firewall rules without installing a plug-in?  That would have been a relevant information.

You have just hit a bad spot in time during the history of OPNsense in my opinion where a lot of migrations need to be done to be compatible with future source code upgrades or whatever it's officially called :
- ISC DHCP got moved to a plug-in.
Luckily migrating to either KEA or DNSmasq is pretty easy thanks to .CSV file exports for your Static DHCP Mappings.

Dunno, ISC DHCP is still there.  I was wondering if there is a way to migrate and hoping for one.  It would be very tedious and error prone to do it all manually.

Where in the release notes were we told that there is some kind of DHCP migration tool and how to use it?

- Firewall Rules (Leagacy) got moved to a plug-in.
Firewall Rules [New] is now the only Firewall Rules section and needs to be migrated indeed.
The official plan was before upgrading from 26.1 to 26.7 but luckily it's not that strict and can be done afterwards too!

A couple weeks ago I was told it'll be years before a migration is required.  I do hope it's still many years because if I'd have to migrate, I'd have to completely redo those installations because the migration tool doesn't work.  That would take a week or so.

- Port Forward just got renamed to Destination NAT.
So that was pretty easy :)

- Outbound NAT is going to be the new Source NAT from now on.
So this also needs migrating now.
The offical plan is during 26.7 and before upgrading to 27.1 next year, but maybe you can get away with doing that later on too... Dunno...

Yeah I noticed.  From my perspective, fortunately it was merely a change in naming.  I didn't do anything with source NAT.

And last but absolutely not least :
!!! Upgrade your Bootloader after upgrading to 26.7 !!!
However this is a FreeBSD thing and not OPNsense specific ;)

Upgrade the bootloader?  Was that mentioned anywhere?  Doesn't that work automatically?  How would I upgrade it?

Once we get all this stuff behind us I am sure the updates/upgrades will be a lot less hassle than they seem to be now :)

The firewall rules were a nightmare, and that was handled badly by not telling us that we don't need to migrate and that the migration will break all the rules.  Instead it was made to appear as a harmless migration, using that tool, taking a look at the export to see if it looks messed up or not, and importing it.  It looked fine and all the rules were broken after the import.

A big fat warning is required, but there was none, and there still doesn't seem to be one.

Good luck! with all of the above...

Thanks, to you too.

This is neither a consumer product nor a commercial solution. This is the community edition of an enterprise firewall. If you choose to use it, you need to educate yourself and read the release notes of every update before pressing the button. Community support relies on users doing their homework. Expecting enterprise-grade stability and active hand-holding without carefully reading the changelog misses the point of a community edition.

That the idea of 'community' --- whatever that idea might be --- is attributed to something doesn't mean that everyone who comes in contact with that something has to be educated to such an extend as, for example, to be able to read the very source code of all the components of a product like OPNsense.  Communities running (school) busses to transport residents (kids) usually do not require such residents (kids) to able to rebuild the engines of the very busses in order to use them, and they don't require the users of the busses to follow all the maintenance logs in detail in case a bus might have a flat tire or needs an engine rebuilt.  They don't even expect something like that from the bus drivers.

'Community' is a horribly overinflated word since a long time now such that it has become virtually meaningless.

There is only so much anyone can do.  Don't expect the bus drivers to rebuild the engines.  Don't expect the manufacturers of the busses to produce perfect busses the engines of which never fail.  And when someone suggests how the bus could be easier to use, don't just ignore his suggestion, and don't blame the user as uneducated and not sufficiently competent to use the bus.

Make it so that the bus driver can easily change a flat tire.  Make it so that everyone can understand the timetable.

Fortunately I don't need to use busses.  Last time I tried there wasn't even a timetable.  If there had been one, I probably wouldn't have been able to understand it.  These timetables are not for 'normal users'.  Busses suck.

Some people are not aware, but it bears repeating:

In the context of the BSD licenses, the "as is" provision means that the software is provided without warranties, and the authors generally disclaim liability for issues, failures, or damages resulting from its use, including operational errors.

We try our best to describe the changes.

of course

And you're doing a fantastic job.  All the OPNsense installations I've to do with have been working 100% reliably without problems for years now, and that is quite an accomplishment.  Thank you.

The code is open so it also documents the changes in a straightforward (but still very technical) way. If you want more you may want to consider participating in this process: the documentation is open and the changelogs are open too.


Cheers,
Franco

Let me suggest this:

What if you were to point out the changes that may possibly break stuff for users or hide things more clearly, plus a link to the release notes, instead of presenting the relatively short note and the long list of detailed changes that is being shown when someone is about to update?

That might give room for the real important stuff, and when someone wants to know all the details they would still be easy to find.

The long list seems to me like it might be good information for developers and users who have been following the development rather closely --- and such users are probably interested only in a particular issue or change and not the whole list anyway.

I don't think you can expect from the 'normal user' to be familiar with all the details, and it doesn't --- and shouldn't --- matter what all the details are.  But, for example, it matters to the 'normal user' to know that it's a bad idea to use the firewall rule migration tool and/or that the 'normal user' needs to install a certain plug-in when he wants to en-/disable firewall rules after the update.  Why aren't we told such things?  Why are we being told things that don't mean anything to us instead?

Perhaps it might help to make updates less confusing :)

Well, it says Identifier wan for the WAN interface, and Device is pppoe0.  That's already three different designations.  It's not any better at other places but even worse because I have an interface titled Interfaces: [FORPPPOE] which turns into forPPPoE when I copy and paste the title --- while the interface is actually named forPPPoE.  But that interface has an identifier of opt5 (what is that for??), and at the same time, the device is bce0.  Why the hell does that have to be made so confusing?

And I'm just noticing that that interface is not enabled, which is probably bad because IIUC, it means that the interface settings are not being applied.  But I don't remember why I didn't enable it and what exactly it is for.  There must have been some reason that I didn't enable the interface when I set things up.

So pppoe0 is bce0.  Should that interface be enabled or not?  How is it even possible to use an interface that is disabled?

And how do I tell if vnstat is configured right?

Installing the os-vnstat plugin tells you to put your default interface into it's configuration file before starting the service.  So I looked at what 'vnstat --iflist' said and put pppoe0 as interface.

Then when I looked at the GUI to start the service, I could pick the interfaces, so I picked the WAN interface.  There is no pppoe0 interface in that list.  Now the configuration has a bunch of entries and still starts with 'Interface pppoe0'.  I didn't find any other 'interface' lines in the file.

Is it supposed to be like this or is this now a misconfiguration?

No, like I said before, I read the release notes that are being displayed before updating.  I don't remember seeing anything mentioned about such a plugin.

If you want ppl to read stuff you need to show it there.

Quote from the release notes:

o firewall: move config.xml default LAN allow rules to new rules GUI
o firewall: legacy rules pages move to plugin

Was that shown in the notes that are being displayed when updating?

I used to read the whole list of details and stopped doing that a while ago because the items in the list effectively don't tell me anything.  Even if I did read them, I wouldn't remember them.  And even if I did read and remembered them, I still only have a choice between updating and not updating.  Not updating isn't really an option, so what's the point of all these items?

In any case, how do you expect users to conclude from these items that they need to install such a plugin when they find out that they suddenly can't en-/disable firewall rules and either migrating and/or the plugin are suddenly mandatory?

And what is 'config.xml'?

I've said before that this was handled badly, and now it seems being handled even worse.

If you're not able to read and understand release notes before updating critical components, OPNsense may not be the correct solution for you. Consider switching to a consumer product.

Obviously, I can't read or understand what I'm not being presented with.  It also matters what the presentation is like.

BTW, that was an utterly stupid remark.  You need to first assess the requirements before suggesting what product to use.  Plus you're implying that OPNsense is not intended to be user friendly while 'consumer products' are, which is not true.  Consumer products aren't intended to be user friendly or useful or anything at all, they are only made to generate profits.

Do you think it is helpful to suggest to use a different product when someone suggests how OPNsense could be more user friendly?

When I look at the statistics and it says 'cleared: 1970-01-01T01:00:02', what does that mean?

It means it was cleared before time synchronization took place. 1970-01-01 is the beginning of Unix time.

That's kinda what I thought.

So how do I find out how many times a PPPoE connection was reset during a given period of time?

None of those would show any data for the last month, for example ...

I don't want to capture all the data like netflow seems to do --- and why would I need to limit it to the WAN interface(s)?  Is there no way to get decent reports from all the gathered data?

Imagine I want to be able to just look at the dashboard and see that information, plain and simple, without any further ado except maybe adding a widget for it.  It should be just there.

vnstat shows that. and recent releases also contain a vnstat widget for the dashboard: https://forum.opnsense.org/index.php?topic=51328.0

Cool, I'll try it out :)

ATM, I'm wondering why the counters of the parent interface of one of the VLAN interfaces which is a WAN interface are so high while the counters of the VLAN interface itself only show some IPv4 blocked packages and all other counters are 0.  When I look at other VLAN interfaces, thier counters look ok.

When I look at the statistics and it says 'cleared: 1970-01-01T01:00:02', what does that mean?