Messages

He means have you tried to emulate a different ethernet NIC type.

Anyway how did you go with this? I am seeing "errors out" on a VLAN interface on a virtualised firewall. Wondering if you found a solution? I noticed that the original post was prior to the release of 21.1 which contained a fix of "Fix stability and reliability issues with regard to vmx(4), vtnet(4), ixl(4), ix(4) and em(4) ethernet drivers."

Did you upgrade to 21.1, and did that fix your issue?

Yes, correct, change to mirror "Deciso (HTTPS, NL, Commercial)", select OpenSSL, Business and enter subscription. Save and check for updates. :)

That's perfect - thank you very much. :)

I have community edition 20.1.4 setup as a trial install at a particular site. It is looking good and we want to switch to the newly-released 20.4 Business Edition.

Question - what's the easiest way of going about this, if we had a license ready to go?  Can the existing install be upgraded, or do I have to download the Business Edition installer and start again? If so... can the config I have for the community edition be imported into the business edition?

I am hoping I can just put in a license key, and enable the change in the GUI, and it downloads whatever it needs to from the Internet. Looking forward to your replies. Thanks.

I last tried OPNsense seriously around the time of 19.1 and was amazed that the installer had multiple lockups.

A couple of years on from that with 21.1 I am trying again, and am gobsmacked that the same installer bugs still exist:- once when choosing "guided install" and again at the swap partition size.   Each time I have to go "CTRL-C" and log back in again as installer/opnsense. Incredible.

I am on standard ESXI 6.5 with latest patches.  UEFI.  Nothing unusual. Sure you can get around it searching around the Internet and eventually stumbing on the multiple CTRL-C re-login workaround (if you dig deep), but this is just not good enough for any software which wants to be taken seriously. I can imagine thousands of techs trying out OPNsense for the first time and abandoning it right there, being scared off from the very start at the quality of the software, or lack thereof.... which is exactly my experience now on multiple ocassions!

Will "Migrate bsdinstaller to bsdinstall" fix this long-standing and very important issue? If it does, then it can't come soon enough? Also will that include install on ZFS? I am OK with UFS installs for my virtualised environments, but there's no way I'm putting UFS on my stand-alone firewall boxes, when others offer ZFS as root install options.

There are no plans to add any non-functional abstractions to the rule listing.


Cheers,
Franco

Anything that improves the readabiity and maintainability of firewall rules is not in fact "non-functional". Rule seperators and/or groupings serve a very important function, at least from a human perspective (and the web GUI is, by its very nature created for humans to use). In my 25+ years in networking I've worked on Checkpoint, Fortinet, Netscreen and Juniper firewalls for national Internet Service Providers and guess what?! - they all have rule seperation/grouping. It's far from being a pfSense thing - it's an industry-wide thing.

pFsense's approach is OK - better than nothing - but even that could do with some improvement. Individual filter rules should be programatically linked to the categories they fall within, and ideally have the ability to toggle collapsing of groups, reorder groups (drag and drop) and toggle group rules on and off. Basically, they act as "first-class citizens" in the whole scheme of things. This is one thing that seperates the commercial enterprise firewall offerings from more hobbyist/enthusiast ones.

As a bare minimum though, there should be text seperators to visually indicate logical groupings of rules. This lack of functionality is one of the main reasons why I conitnue to favour pfSense over OPNsense.

This issue makes me think of the late 90's novel by Allan Cooper entitled, "The Inmates Are Running the Asylum", which details how programmers ruin interface design by thinking that they know what's best for the end-user (and to no-one's surprise, they don't)

+1 :)

We can't solve your host shuffling interfaces in the guest.

Well for some reason, OPNsense doesn't like it when shuffling between a couple of interface types on the WAN. It blows away the IP address. No idea why. Maybe that kind of bug report feedback is helpful to you? If you're rather ignore it - that's up to you. It's likely you have more pressing bugs to sort out with this product.

If you just want to be here to complain I'll spend my time helping others instead of reading rants. Even if you are not happy with that at least one other person can be.

Please respect my decision as much as I respect yours to barf all over this forum.

You can do what you like. But do you think it is a good idea, as a OPNsense representative (?!) to belittle your users on public forums, and refer to their feedback and day-long efforts of testing and feeback to make the product better for the good of all, as "barf". Seriously?!  This is unbelievable stuff.

Anyways you probably (again for the second time) didn't read what I actually wrote. At the bottom of my last post I said I would offer suggestions for fixing this pain point and improve the product.

While you were busy thinking of ways to slander me, I was writing constructive feeback ("barf" in your own words). Since I had already written this "barf", then here's a hot serve. You probably won't read it, but hopefully a mature OPNsense developer will.

====
My suggestions to improve this product for the pain points I've raised are as follows:

1. Obviously fix the WAN interface remapping (re-Assignment) bug. I can assist by providing details of my environment [edit: probably not now I've been disrespected]
2. Fix the display output! Plainly put - it's severely lacking. For example it says this after I reboot into the new interface types (all previous interfaces changed to new types - from vtnet to vmx in this example):

DATA (vtnet0)   ->
TEST (vtnet3)   ->
WAN1 (vtnet1)   ->
TEST2 (vtnet0_vlan444) ->

When it should be something like:
DATA (vtnet0 - missing)   -> 192.168.27.1/24
TEST (vtnet3 - missing)   -> 10.10.10.1/24
WAN1 (vtnet1 - missing)   -> 5.5.5.5/30
TEST2 (vtnet0_vlan444 - missing) -> 20.20.20.1/24

NOTE: Interfaces missing. Please shut down and correct hardware, or "Assign interfaces" (option 2.) for the interfaces listed as missing, in order to prevent the loss of their configuration data! This arose because these interfaces have "Prevent interface removal" set in the Interface configuration, and can no-longer be found by the system

Finally, when rebuilding VLAN interfaces (as one has to do on the new parent interface), for sure you can still have your scary warning (inherited from pfSense), of  "WARNING: all existing VLANs will be cleared if you proceed!", but since that's not actually true, how about improve it to be actually factual? For example:

WARNING: All non-permanent VLANs (VLAN interfaces not protected by "Prevent interface removal") will be cleared if you proceed!" Any VLAN Interfaces with status of "missing" should be Assigned (option 2.) after building new VLANs, in order to prevent the loss of VLAN Iconfiguration data"

Much better!

=====

Just set "Prevent interface removal" in the respective interface settings. Any polite forum search would tell you.

franco.
(1) You assumed I posted without first searching (incorrect), and
(2) You replied with a suggestion I explicity mentioned as not working.  ::) I know this is the Internet where people rarely read beyond the headline but still... 

@Gcon
and if without hysterics?
what did you do and when did you lose the configuration?
i don't have esxi at hand so i quick test it on hyper-v (switch intrefaces type from hn to de):
set "Prevent interface removal" on LAN and WAN (hn0, hn1 interfaces), power off vm, delete nic-s (hn0,hn1), add another nic-s (de0, de1). boot opnsense, go to 1)Assign Interfaces, assign new de0\de1 to LAN\WAN, reboot and continued working with old config.
so?

Fright - thanks for replying with your experience that wasn't trolling, and showed you actually read what I wrote - I do appreciate that.
I'm glad it works for you but when I switch between vmx and vtnet interfaces and reassign the "WAN" interface, it always gets reset and loses its IP address....and yes "Prevent interface removal" is always set. The static IPv4 I put in gets blown away and goes back to DHCP/DHCP6. I went several times from vmx1 to vtnet1 and back again. Each time with a "save" and "apply" and even a reboot to make sure the IP stuck, it was impossible to remap between interfaces and not lose the IP address info.

So there is definitely some sort of bug with remapping the WAN interface to different NICs - at least in the environment I have setup (not hyper-V).

**The good news is that other interfaces can be remapped successfully.**  (both major interfaces and subinterfaces / VLAN-tagged interfaces). So why did I think this wouldn't work for any interface / subinterface? Well there are several reasons:

1) I tried the "WAN" first and it was no good. If it failed for that, then what hope did the other interfaces have? Very little I thought.

2) After the reboot, into the boot up with new interface types, I am greeted with something like this....
========================
*** <HOSTNAME REMOVED> OPNsense 20.7.2 (amd64/OpenSSL) ***

DATA (vtnet0)   ->
TEST (vtnet3)   ->
WAN1 (vtnet1)   ->
TEST2 (vtnet0_vlan444) ->

<FINGERPRINT REMOVED>

<HOSTNAME REMOVED> (ttyu0)
========================

I originally had a couple more WAN interfaces and half a dozen tagged interfaces - lots of config actually -  but you get the idea - none of the interface IPv4 addresses show.

3) Upon building VLANs you get shown this scary warning:
"WARNING: all existing VLANs will be cleared if you proceed!"

...which gives the user the impression that you HAVE TO START FROM SCRATCH! (sorry was that too hysterical?)  A proper warning would also add "NOTE: This excludes VLANs explicitly set to 'Prevent Interface removal' .... but that is assumed knowledge, but it shouldn't have to be.

So I was there basically thinking this,  "OK I tried to remap the WAN - but that's lost! Damn! Now all the interfaces are sitting there laughing at me at the console with no IP addresses, and to rub salt into the wound... it's saying even if I attempt to rebuilt my VLAN tagged interfaces - it's going to blow everything away... if it hasn't already!". GRRRR!! x 1000.

But what actually happens is that only the WAN gets lost, (and it only seems to be the WAN IP addressing... thankfully the NAT and filter rules are still there). OK I will offer some insightful sugggestions in the next post which can help this project improve....

OPNsense is super horrible when it comes to changing interface types! SIMPLY. HORRIBLE.

I have a lab setup where I am using VMware Paravirtualsed Ethernet v3 NICS (vmxnet3) and then I found out that tagged Ethernet frames weren't working in the lab environment (GNS3 VM on ESXi 6.7) so changed to Paravirtualised Network I/O (virtio-net-pci) NICs and then rebooted. FYI the VMware NICs are vmx* in OPNsense and Virtio NICs  are vtnet* in OPNsense .... and the vtnet* type did work with tagged frames but here's the thing...

Upon reboot, instead of OPNsense going "err.. all your old NICs are gone.... help me match up the previous NICs to new ones",  it just says "yeah screw you buddy and your previous config that you worked so hard at and for so long - it's all gone now!! HA HA HAAAAAAA!!!!"

OPNsense devs - this is terrible. You need a cleaner separation of logical interface to physical interface mappings. Then if/when the physicals change, you prompt for this and do an **assisted remap**, and then the customer can get up and running again with **NO LOSS OF CONFIGURATION**.  You don't just blow their config away!!!!!grrrrrr >:(

And no the "prevent inteface removal" doesn't help. That just stores the old interface data (with old interface types ... but does only a partial job at that) - it doesn't help with remapping to new interfaces / interface types.  I don't know if you guys do much testing in virtualised environments but if you do - you can't really be pushing the boundaries of what you can do in them. Otherwise you'd have sorted this out well before now.

pfSense actually can and does do interface remappings. i.e. it handles this sanely. Here's a tip.... when you fork software... you are supposed to *improve* it.

Why does this matter? It matters because I have a full lab environment modelling two firewalls, three routers and three internet connections plus LAN switches and hosts. I have firewalls on pfSense right now and I could take the production configs with minimal massaging, and easily adapt them to the lab setup with minimal fuss.  I want to be able to rebuild the pfSense lab firewalls with OPNsense as a Proof Of Concept (P.O.C.) with full production-ready pre-built configs and test everything. The idea is to work up the OPNsense configs in the lab until they are production-ready, and then rip the configs off the lab and take them into production. But with OPNsense being completely unable to handle interface type changes - I can't for the life of me see how this is going to work! I'll probably mean I will abandon any attempts to go to OPNsense - which means no support contract money for them. This stupidity has most likely cost them a customer.

Now you can fire back with all the ad hominems that you like but I couldn't care less. If there are massive problems with software I'll point it out - every single time. Constructive criticism is helpful - you should appreciate that I've taken the time to test the software, find the issue, compare and contrast to competitors (e.g. pfSense), and effectively communicate the issue to you and offer suggestions on how to fix it.

But yeah - it stinks.

OK so have been a bit busy but "that other firewall brand" starting with a "p" doesn't have the sawtooth CPU issues after install (see screenshot), so reluctantly I am going with their product (I have to deliver the new firewall by year's end).

I look forward to OPNsense01 20.1 to see the new features and hopefully this mysterious CPU issue is gone!  I will keep the OPNsense 19.7.7-amd64 VM with this issue on the ESXi host (they have different IP addresses and don't clash) so if any of you OPNsense devs want to get in touch and have me to run any tests I'll be more than happy to help out.
-Gcon

I don't think you'd want to have a single FQDN (firewall.domain.com) with multiple A records representing your WAN links.  DNS servers randomise the order of the IPs returned in a lookup, so it becomes non-deterministic. Even if the order didn't matter, then the DNS client randomly picks an IP - usually the first one - and then tries that. There's no guarantees that that the client will try the second IP - that would depend on how the DNS client application is programmed.  The test would be to try it in a lab and see what happens. It all sounds a bit messy though.

OPNsense 19.7 has release notes "IPsec Route based mode (VTI)". I'm looking into that feature to see what it provides as I used to use Virtual Tunnel Interfaces extensively with Cisco back when I was using those edge devices for VPN and found them to be quite handy - especially when used with GRE headers and running RIPv2 over the top.

The large CPU usage seems to be due to python 3.7. I wish the next version of OPNsense was out already so I could compare to that.

This weekend I'm going to clone the OPNsense VM appliances and install the clones with the latest pfSense CE and painstakingly copy the extensive firewall rules etc and see how it compares. I'll be at it all weekend, but it seems like it's just something i have to do in order to get a baseline comparison going. Would much prefer to stick with OPNsense as the Web interface is so much nicer to my eyes, and how my brain works. Will see how it goes.

Hi peoples.  Is this sawtooth CPU pattern going to be an issue if I put this into production?

It's the latest OPNsense 19.7 as a VM on a dual CPU hex-core Intel x5650 on ESXI 6.5U3 / Dell R710 (lightly loaded).

The first image is a new build where it's not passing traffic but listening in with promiscuous mode, which I seem to need to get CARP to work properly.

The second image is where I cloned the VM to another host configured exactly the same with same hardware specs, except it's a non-production box running ESXi 6.7U3, and there's absolutely zero external traffic going to the Opensense VM as the uplink is unplugged - so doesn't seem to be triggered by network traffic. In both cases interrupt CPU % doesn't get over a few percent anyways.

Other details - setup as 4 vCPU, 8GB RAM, 120GB HDD (4x 15k RPM RAID10 SCSI 3.5") and vNICs are VMXNET 3  (9 of those), and a LSI SAS SCSI controller.  BIOS is EFI, so I could up the resolution of the console.

Not identical but similar patterns of spiking shy of 30%. Is this normal?
Thanks. Gcon

"Which would you choose and why?

HP ProCurve 2810-48G 48 port 10/100/1000 or a Dell PowerConnect 2824 24 port 10/100/1000?"

I know it's off topic, but can I ask why you're limiting the choice to these two? I use TP-Link and UBNT gear personally, but you already burned the poster suggesting those  ::)

Thanks guys!

I had already tried the following before posting and it didn't help:
set hw.pci.enable_msix=0
boot

...and...
set hw.pci.enable_msix=0
set hw.pci.enable_msi=0
boot

Today (after a good night's sleep) I tried this:
set hw.pci.enable_msix=0
set hw.pci.enable_msi=1

...and it worked!  Seems obvious in hindsight. Hope this helps a future searcher.

Hi peoples.

I am super desperate and will pay someone to help me out here. I have had a high number of interface interrupts causing high CPU and I thought I'd try putting the following in /boot/loader.conf.local  (as recommended elsewhere):

hw.pci.enable_msix=1
hw.pci.enable_msi=0

Unforunately I cannot now boot as all my PCI devices "fail to allocate an interrupt"!

"Safe mode" is no help to me (yeah not very safe after all)
#3 loader prompt - have tried setting loads of things to no avail.

You'd think this would be easy but it's an absolute nightmare! A shame the website I got this from failed to mention the dangers of this. I have had days of building a new firewall config... now am left feeling absolutely shattered by this small yet devestating issue as I fear it might all be lost. I'm absolutely gutted.

Please help....I beg of you... please! :-[