OPNsense Forum
Archive => 18.1 Legacy Series => Topic started by: mestafin on March 12, 2018, 09:15:24 am
-
Hi,
I do appreciate the willingness of the OPNsense team to fix issues and to respond to any bug reports in a timely (weekly) manner, but it also create some uneasiness.
When is 18.1 Production Series stable enough to install on a critical production site?
Is there not a need for a more "stable" release with a list of known issues that can be used on production sites provided you can live with the the known issues?
The frequent 18.1 releases is a just too "bleeding edge" for me, or am I just too conservative?
-
Agreed. I was actually thinking to propose this myself.
18.1.4 would be the first version in the 18 series that I think could be cataloged as a strong RC.
If I could have a say in this, for a release, I would confirm as bugs & fix these, in this order:
1. Installer issues: without a doubt, there are some -> first user impressions -> can't install -> can't use
2. Dependencies: verify & fix
3. PPPoE issues: no net -> no use in many cases
4. Routing issues involving VLANs -> can't confirm yet if there are any; personally, I didn't get to test VLANs just yet with OPNsense; many people that use custom firewalls/gateways will probably want this to work flawlessly
5. Other stuff I probably don't know about
Calling these service releases beta versions or release candidates has an added benefit:
1. People will accept the fact there are bugs in these versions
2. OPNsense's public image and trustworthiness will not be so affected if major things break during this development phase (if it's the case)
3. Frequent beta's and RCs gives a very positive feeling to users
But, this recipe might need/involve a few things:
1. Users to test betas and RCs in various environments
2. Do not retire the previous stable version until a new stable version is out and confirmed as stable for some time
3. Beta/RC users to report issues on Github, not the forum (mandatory)
4. The rest of things can be dynamically integrated over the course of implementation
-
Hi,
I do appreciate the willingness of the OPNsense team to fix issues and to respond to any bug reports in a timely (weekly) manner, but it also create some uneasiness.
When is 18.1 Production Series stable enough to install on a critical production site?
Is there not a need for a more "stable" release with a list of known issues that can be used on production sites provided you can live with the the known issues?
The frequent 18.1 releases is a just too "bleeding edge" for me, or am I just too conservative?
You mean something like a community edition and a business edition (with some license fees)?
-
Hi there,
When is 18.1 Production Series stable enough to install on a critical production site?
When is it secure? Or stable? The answer depends on the expectations. Some agree to production status, some disagree to whatever is answered or shipped. If it works for you it's enough. :)
Is there not a need for a more "stable" release with a list of known issues that can be used on production sites provided you can live with the the known issues?
Not within an open source scope. There was an idea for https://forum.opnsense.org/index.php?topic=3408.0 but that hasn't been made a reality yet.
BTW: Updates in 18.1.x are not rolling, that's why we have two major releases per year.
Cheers,
Franco
-
When is it secure? Or stable? The answer depends on the expectations. Some agree to production status, some disagree to whatever is answered or shipped. If it works for you it's enough. :)
This is how we usually do it (1 year plan):
- finish scheduled bugfixes (in a specific order)
- finish scheduled features (feature complete)
- finish adding new automated tests
- tests & bugfixes
- code freeze
- start a new set of automated & manual testing
- if no critical issues found, release a beta with release notes (known issues included)
- some more tests & bugfixes
- include user's feedback (bugs, no features)
- release another beta, probably a RC
- more tests & bugfixes (internal + user's feedback)
- release the major version (first stable)
- bugfixes (in parallel, new features development on another branch)
- code freeze
- more tests
- release a SR (second stable)
- release another beta (from the new branch)
- release another SR, but only if necessary (old branch, third stable)
-
I see your point. Is it realistic to do this twice a year?
Smaller iterations are "safer" as less code is turned over in the new version. Larger iterations are "safer" because less code is turned over in the old version.
Extrapolating from feedback over the years, two releases a year seem rushed or premature. In other software, the same happens in 2-3 year cycles. Sometimes I wonder if anyone likes change at all. And if 80% is ok any way, we must go back to the other 20% and keep iterating.
Maybe this isn't appropriate for commercial software, or maybe that is the point to offer a real "USP" to open source that tries to not lack behind?
The point is: tweaking 18.1.x or 18.7.x is not going to solve the issues that others are seeing. We need to talk about these issues from a broad perspective that includes time and effort on both sides. E.g. another stable version in open source terms is not viable.
Cheers,
Franco
-
It's not the numbers that matters, it's how you do it, IMO.
I certainly have no experience with opensource projects. But I don't think that really matters, minor details here and there, the audience is the same. I could be wrong of course.
I'm trying to help, not to judge. So here it goes:
Frequent release cycles are fine as well, if you could define (the OPNsense way) "stable". You have to define it at some point and stick to it if it works. If not, dig deeper and find a way.
I know you rely heavily on FreeBSD and their releases and commits, which is also fine as long as you stick to your defined "stable". These three months with you guys have been a new experience for me, and coming from the outside, I somehow feel that "stable" is not yet defined. Maybe you don't want too, it's one way to macro manage a product, which sometimes works in such a dynamic and ever changing software. But it's a security product at the end of the day, it's not just a web interface on top of an existing OS, even if it's opensource. The concept of "stable" is most probably arguable, but I don't think it cannot be defined, the OPNsense way.
I know it's difficult to work with people, everybody is an expert in everything and everybody knows better. I also know the reality is totally different and only a handful of people actually deliver something really valuable. However, I learned the hard way that clearly defining some stuff is for the greater good and allowing users to do that for you (or try) is not necessarily that good. The issue here I think is exactly that we are having this conversation (IMO). We shouldn't :-)
Personally, I am not convinced that a clearly defined stable channel is not viable. Call the other frequent releases something else, don't call them alpha, beta, sr, rc etc, invent something else, like feature release or anything else, whatever works is fine and pin it to the forum and/or website.
Ok, I'll stop talking about this as it is a sensitive area and I probably already said too much. Sorry if I made anybody uncomfortable, I meant well. I'll keep trying to help, as always. Take into consideration whatever you think is worthy, or don't, no hard feelings here :-)
-
Believe me, the guys feeling responsible can go fine with critism :) Just go ahead, if 1% of it will respected you influenced the process by x percent :P
If franco and the other core devs would only go stable this firewall would be without any features.
I'm way no programmer and have no idea of it. I can do some basic if/then/else and perfectly copy+paste, and with this I've done around 12 plugins. The bad side is I'm a network engineer, I'm not that good when it comes to Q&A and that's why there will be probably some bugs with updates.
-
I like the two major releases per year model. But reserve the yy.1/7.x (dot x releases) for security and critical functionality only. That would be my definition of "stable".
As the product becomes more mature and development beat rates slows then maybe yearly major releases.
-
Stable is good, features are good. Why not combine these somehow so that everybody is free to choose whichever? For production, one could choose the stable release, for features, one could choose the less stable but more feature rich release. When some features get stable enough, add them to the stable release and so forth. Damn it, i'm talking again :p
-
Good idea! But this is extra work and I don't know who should do this since everyone does the development on their spare time. Only Deciso works for it, but since OPNsense is responsible for their income I'm not really sure they will give their extra time for a stable distribution for free.
-
Hi,
I do appreciate the willingness of the OPNsense team to fix issues and to respond to any bug reports in a timely (weekly) manner, but it also create some uneasiness.
When is 18.1 Production Series stable enough to install on a critical production site?
Is there not a need for a more "stable" release with a list of known issues that can be used on production sites provided you can live with the the known issues?
The frequent 18.1 releases is a just too "bleeding edge" for me, or am I just too conservative?
You mean something like a community edition and a business edition (with some license fees)?
Oh hell no! I can't stand using crippled or beta distros because I cannot pay for a fully functional release copy, Endian which I had used for months is exactly this way. I wound up sending patches and changes but did I get access to the stable version with all the features? Nope. I was just testing beta quality yet crippled software.
Anyone can switch their updates to go from stable to bleeding edge in OPNsense, it's a feature recently added. These ARE stable releases. The community edition model would be akin to being forced to use the development version unless you are paying thousands and no other choice. Anyone looking for something 100% stable and bug free, get back to me in about 5 billion years just before the Sun is at the end of it's Star life and tell me how it went.
I'm fine with it just as it is. Two majors with minor security and fixes updates between. We just got 18.1, we have some new features. We likely will not see any changes in features if at all until this Summer. Even software costing tens of thousands a year for licenses has unknown bugs that may show up at some point for someone. The openness and powerful feature set where we all get the same stable version whether we are a Corp with a million dollar maintenance contract with Deciso, is exactly why I looked into OPNsense after years of Linux firewalls and a stint with PFsense. I'm getting up to speed with the differences between BSD and Linux but many programs by CLI are the same.
-
The frequent 18.1 releases is a just too "bleeding edge" for me, or am I just too conservative?
Nope, you just misread the entire situation and badly. These ARE production ready releases, unless you changed your settings in updates to use development snapshots which are bleeding edge testing versions. What version do you think Deciso is using on client's networks? Some of these clients being major players with scaled up networks most of us couldn't even imagine having. At least seven that I know of.
The premise that something is only stable if there are infrequent security and bugfix updates is very misguided. If this were true, IPCop which is in stasis and has not had a cvs commit in nearly two years would be the most stable distribution ever conceived. Actually the IPCop.org domain only has several months left before it will be discontinued. This distro is not small time nor is 18.1 a testbed for something else. You are not a guinea pig unless you switch your update settings to devel snaphots. This can easily be reversed.
I really hope silly ideas that will ruin everything that brought so many of us here are not considered.
-
Good idea! But this is extra work and I don't know who should do this since everyone does the development on their spare time. Only Deciso works for it, but since OPNsense is responsible for their income I'm not really sure they will give their extra time for a stable distribution for free.
Yep, I'm aware.. but because I don't know anything about their resources, I popped the idea :-) I would help, but I'm not a BSD/Linux guy. Very limited knowledge.
-
When is it secure? Or stable? The answer depends on the expectations. Some agree to production status, some disagree to whatever is answered or shipped. If it works for you it's enough. :)
This is how we usually do it (1 year plan):
- finish scheduled bugfixes (in a specific order)
- finish scheduled features (feature complete)
- finish adding new automated tests
- tests & bugfixes
- code freeze
- start a new set of automated & manual testing
- if no critical issues found, release a beta with release notes (known issues included)
- some more tests & bugfixes
- include user's feedback (bugs, no features)
- release another beta, probably a RC
- more tests & bugfixes (internal + user's feedback)
- release the major version (first stable)
- bugfixes (in parallel, new features development on another branch)
- code freeze
- more tests
- release a SR (second stable)
- release another beta (from the new branch)
- release another SR, but only if necessary (old branch, third stable)
Tests tests and more tests, the Sun will burn out before anything bug free is ever released by anyone. If Architecture was as complex as millions of lines of code with each fix possibly creating another problem no one will run into for maybe months or years, a single woodpecker could destroy civilization. A Stable version of Windows often has 60,000 known bugs. OS X had nearly that many when it came out. Even so it was considered stable and most people were not affected by those bugs much of the time.
I like the OPNsense model exactly as it is. Other offerings from Companies that would have you think they are ironed out and bulletproof and cost a lot of money to use for even 50 users per year, are simply not telling you everything. These guys are very transparent and forthcoming and weird obscure problems like I had recently are taken seriously. I've not seen any dstro polish up and improve in three years like this one has.
-
Other offerings from Companies that would have you think they are ironed out and bulletproof and cost a lot of money to use for even 50 users per year, are simply not telling you everything.
I work with 3 of these. Man, I'm telling you, you are right, we are full of s**t. All marketing departments are selling big fat lies 24/7 and deceiving users as much as they can! For the last 15+ years!
-
I wouldn't go that far. I cannot blame someone for providing a usable service to a paying customer and creating a livelihood in the process. You guys are not full of it and likely would not knowingly and unscrupulously put someone's network at grave risk but would not want to talk about every little zit to the people who are paying you to worry about it. They pay you, you worry about the little zits and prevent the big festering boils or worse, festering carbuncles (a clustered group of boils, ewwww), I'll take a zit or several over a festering carbuncle anytime and be happy about it.
-
Hi,
Given the responses to my original post, clearly illustrates the point that there are different opinions about this.
Decisio BV is closely associated with OPNsense and is sponsoring the development. They also sell consulting services and hardware linked to OPNsense.
Clearly, it is in the best interest of Decisio to make sure that OPNsense, even as an open source project, is successful.
The iXsystems business model for Freenas with an open source community edition and a paid commercial edition, seems to be working very well.
The real money is in servicing enterprise level clients or relative bigger businesses and for these clients, stability is critical.
Before I started the our OPNsense pilot project as an option to replace our EOL Cisco and CheckPoint firewalls, I talked to a lot of people in the IT security field. One of the things I asked was - is it time to move away from proprietary solutions to open source? The answer to this can best be summarised as "... There are some good open source solutions available, but they are not enterprise-ready, they are not stable enough. You can use it for a small branch office, but not in your corporate data centre".
Unfortunately, after about 4 months of using OPNsense, I am starting to agree with that opinion.
Another point, there are some very good open source software available which are enterprise-ready and stable. It is therefore possible to create a development process to deliver great open source software with growing feature sets, without sacrificing stability in the process.
Clearly, more discussion and thinking are needed ...
-
That's a bit stretched... :-) I'm referring to stable vs enterprise.
I personally never deployed any software in mission critical environments without having enterprise grade support in the first place. And never heard anybody doing so. Doesn't really matter what kind of software it is, as long as i get enterprise support, i don't even care. But as an unwritten rule, you will only get this kind of support from very successful, expensive and high quality software/hardware developers/manufacturers. Many times with worldwide coverage. That's what enterprise usually means.
So I'm definitely not talking enterprise here. Just good old SMB stable.
-
Hi,
Before I started the our OPNsense pilot project as an option to replace our EOL Cisco and CheckPoint firewalls, I talked to a lot of people in the IT security field. One of the things I asked was - is it time to move away from proprietary solutions to open source? The answer to this can best be summarised as "... There are some good open source solutions available, but they are not enterprise-ready, they are not stable enough. You can use it for a small branch office, but not in your corporate data centre".
Unfortunately, after about 4 months of using OPNsense, I am starting to agree with that opinion.
If it gives you a warm fuzzy feeling paying for it, then by all means do so. I'm not wowed by the word Enterprise unless it's a Starship. A three man company can be an Enterprise, by that definition a fifty dollar firewall is Enterprise ready. What you suggested in your original post completely contradicts why OPNsense exists in the first place. It forked for a specific set of reasons and many of us are here for some of if not all of those reasons and because we just plain like the firewall compared to many others we have tried, some giving you a free trial before shelling out a lot of money to keep it. We also have no problem with learning how to use it properly for the application of such. Name one person or entity that had their network compromised due to OPNsense and not because of how they botched the setup of OPNsense.
Endian has the exact business model you seek. It was originally forked from IPCop years ago.
-
Did I mention compliance? Audits?
That's also a major player in enterprise-grade stuff...
-
That's a bit stretched... :-) I'm referring to stable vs enterprise.
I personally never deployed any software in mission critical environments without having enterprise grade support in the first place. And never heard anybody doing so. Doesn't really matter what kind of software it is, as long as i get enterprise support, i don't even care. But as an unwritten rule, you will only get this kind of support from very successful, expensive and high quality software/hardware developers/manufacturers. Many times with worldwide coverage. That's what enterprise usually means.
So I'm definitely not talking enterprise here. Just good old SMB stable.
I agree with your view on mission-critical environments and enterprise support. That is not what I were advocating and I am not expecting OPNsense and Decisio to compete with companies like Cisco.
My use case for our pilot project is a small subsidiary with 2 servers running a few vm's - a typical SMB with less than 10 people. Despite this, I would still prefer the sysadmin responsible for their IT, to have stable firewall software that he can expect to work.
Let's be honest, the 18.1 release was promoted as the way to go and the 17.7 series was immediately marked as legacy. That implies that the 18.1 release is at least stable and production-ready without big regressions. Judging by the feedback in this forum, basic features did not work as expected or had issues.
As someone else pointed out, it would have been better to keep the 17.7 series as production and rather release 18.1 as beta or release candidates to indicate that there may still be bugs.
-
If it gives you a warm fuzzy feeling paying for it, then by all means do so. I'm not wowed by the word Enterprise unless it's a Starship. A three man company can be an Enterprise, by that definition a fifty dollar firewall is Enterprise ready. What you suggested in your original post completely contradicts why OPNsense exists in the first place. It forked for a specific set of reasons and many of us are here for some of if not all of those reasons and because we just plain like the firewall compared to many others we have tried, some giving you a free trial before shelling out a lot of money to keep it. We also have no problem with learning how to use it properly for the application of such. Name one person or entity that had their network compromised due to OPNsense and not because of how they botched the setup of OPNsense.
Endian has the exact business model you seek. It was originally forked from IPCop years ago.
We do not evaluate and select software on whether we pay for it or not or whether it is called "enterprise-ready". If we did that, we would never have selected OPNsense for a pilot evaluation.
I am also not advocating a specific business model. I was responding to a statement made that it is doubtful if Decisio will sponsor more and testing and quality control.
The point that I raised in my first post, was that it may be time to consider a different release model that makes a distinction between a software release that is stable and can be deployed in a production environment, even for a SMB, and a release of new software that may still have some bugs
-
If OPNsense would be enterprise grade (not ready, grade), nobody woud every pay a dime for software :)
My personal mission is to make OPNsense competeable with UTM's like Sophos and this is way more to go than I expected but we are in progress and it's fun too :D
-
Take Mikrotik as an example. Awesome devices, brilliant software (also pretty complicated). Although one might think they are enterprise grade, they lack the support and so very few enterprises adopted them. Also lack the features of OPNsense.
But, their update/release channels are as in the freshly captured (attached) image from my Mikrotik (bugfix only, current, release candidate and development). This gives users plenty of options to choose from. And a warm feeling that there is a stableish channel, so no need to be that nervous about available updates. Sure, they have plenty of bugs as any other software, but most of them are in the development and rc channels. They are not opensource, but not enterprise either, putting both OPNsense and Mikrotik fighting for similar target audience.
-
But, their update/release channels are as in the freshly captured (attached) image from my Mikrotik (bugfix only, current, release candidate and development). This gives users plenty of options to choose from. And a warm feeling that there is a stableish channel, so no need to be that nervous about available updates. Sure, they have plenty of bugs as any other software, but most of them are in the development and rc channels. They are not opensource, but not enterprise either, putting both OPNsense and Mikrotik fighting for similar target audience.
Separate channels are all I am asking for
-
Separate channels are all I am asking for
Yep, me too (if possible), as I said before...