Messages

16

Hardware and Performance / Re: Poor Throughput (Even On Same Network Segment)

« on: October 27, 2020, 12:16:51 pm »

What i find out when opnsense used in a virtualized environment its uses only one core only. The hw socket detection is faulty in case.

net.isr.maxthreads and net.isr.numthreads is always returns 1.
But it can be changed in the tunables too.
This also needs to change net.isr.dispatch from "direct" to "deferred".
This gives me massive performance boost on gigabit connection, but still not perfect. The boost comes with overhead too. But only in fbsd 12. With 20.1 what is still based on fbsd 11 its lightning fast

17

Hardware and Performance / Re: Poor Throughput (Even On Same Network Segment)

« on: October 27, 2020, 12:13:06 pm »

Would it be possible to install a stock FreeBSD 13 kernel?  Maybe they fixed the regressions.  I'm wondering if it has something to do with HBSD compile flags for security.

Unfortunatelly this is not so easy. You cant use a precompiled kernel from an another system. It wouldn't boot.
You have to compile from source, but newer kernel means newer headers and libraries in dependency. The compilation process could failed at some point. The only solution what could work is cherry pick the fix only and implement to the original kernel source tree and compile. But this needs work too.
I was an android kernel developer many years back so i know experiencing with the kernel is always risky.

Wouldnt it be easier to do it the other way round?

Make OS work with FBSD13? To eliminate any remnance of bad plugin code?

They just switched to fbsd12 i dont think fbsd13 will be adapted soon. But you have the point.

18

Hardware and Performance / Re: Poor Throughput (Even On Same Network Segment)

« on: October 27, 2020, 08:53:09 am »

Would it be possible to install a stock FreeBSD 13 kernel?  Maybe they fixed the regressions.  I'm wondering if it has something to do with HBSD compile flags for security.

Unfortunatelly this is not so easy. You cant use a precompiled kernel from an another system. It wouldn't boot.
You have to compile from source, but newer kernel means newer headers and libraries in dependency. The compilation process could failed at some point. The only solution what could work is cherry pick the fix only and implement to the original kernel source tree and compile. But this needs work too.
I was an android kernel developer many years back so i know experiencing with the kernel is always risky.

19

Hardware and Performance / Re: Poor Throughput (Even On Same Network Segment)

« on: October 26, 2020, 03:32:57 pm »

Just keep using 20.1 with all the security related caveats and missing features. I really don't see the point in complaining about user choices.


Cheers,
Franco

I did rollback, everything is fine. The network speed is around 800mbit again (gigabit internet), with 20.7 this was just 500-600mbit. Speed is important here, i dont care about missing features i dont use any. Im not sure about the security caveats. freebsd 11 is no less secure. Until this issue not fixed i stay with 20.1.x. This servers used in production enviroment, i dont have time and oppurtunity to use these as a playground. This was exactly the same reason why i abandon using pfsense. They importing untested kernels and features and the core system become unstable and after an upgrade i have fears what will gone wrong. Opnsense did right for now, i hope the devs fix this or at least we have some workaround. The speed is not the only issue. I have to disable IPS/IDS and sensei too because its cause system freeze. I basicly neglected my firewalls. I know this is still in testing phase but 20.7 is 4 almost 5 months old now and still unable to use this features properly. And we paid for the sensei which is unusable now. This is not acceptable. So yes, i take the "risk" and did rollback wherever i can...

20

20.7 Legacy Series / Re: Testing 20.7 but being able to rollback to 20.1 without install. Possible?

« on: October 26, 2020, 03:16:27 pm »

I did multiple rollback currently. HW and virtualized too.
Just backup the config from 20.7.x reinstall a 20.1 import the config and update to the 20.1.x latest version.
I tested with openvpn client server, ipsec tunnels, etc. Everything is working fine.
I dont think its possible without reinstall.

21

Hardware and Performance / Re: Poor Throughput (Even On Same Network Segment)

« on: October 26, 2020, 01:32:54 pm »

It should talk about this, so maybe offtopic but still.
Half year release model, so im updated since recently, 20.7 is almost half year old now, we are close to the 21.1 now, when 20.7 will be obsolate too. You're right about that a critical system software should wait for adapting new releases. So even the 21.x series should use freebsd 11 and wait for upgrading to 12 until it will be stable. A firewall is not a good place to experiencing and making the first step.

But i can say something what is not offtopic.
Disabling net.inet.ip.redirect and net.inet.ip6.redirect, increasing net.inet.tcp.recvspace and net.inet.tcp.sendspace also kern.ipc.maxsockbuf and kern.ipc.somaxconn helps a little. Still have perfomance lost but not that bad.
I attached my tunables related config.

22

Hardware and Performance / Re: Poor Throughput (Even On Same Network Segment)

« on: October 26, 2020, 12:50:54 pm »

With proxmox using vnet adapter the speed is fine, but using pfsense based on freebsd 11 works fine with vmxnet3 too.
So the issue is with the HBSD and the vmxnet adapter. I dont understand why opnsense based on a half dead OS. HBSD is abandoned most of the devs. Just drop it and use the standard freebsd again.

FreeBSD 12.1 has the same issues ..

Yes, but the pfsense current stable branch still using freebsd 11.x not 12. I think they are on point. Not a good idea switching to a newer base OS if its still have many issues. Now i have to roll back to opnsense 20.1 everywhere where i upgraded to 20.7. And the issue is not just with the vmxnet. After i upgrade to 20.7 one of my hw firewall with EFI boot, the OS no longer boot but freezed during the EFI boot. Its also a freebsd 12 related issue, i already figured out.

23

Hardware and Performance / Re: Poor Throughput (Even On Same Network Segment)

« on: October 26, 2020, 10:27:47 am »

With proxmox using vnet adapter the speed is fine, but using pfsense based on freebsd 11 works fine with vmxnet3 too.
So the issue is with the HBSD and the vmxnet adapter. I dont understand why opnsense based on a half dead OS. HBSD is abandoned most of the devs. Just drop it and use the standard freebsd again.

24

20.7 Legacy Series / 20.7.3 vmxnet 10Gbps adapter via tagged VLAN slow performance

« on: October 22, 2020, 10:08:50 am »

Hi!

I have multiple opnsense virtual machine with vmxnet adapter's. All of them is 10Gbps connection.
The internet access is 1/1 Gbps. I can measure when im directly connected to with my laptop.
I use 802.1q vlan tagging on the virtual machines using simply static ip addressing.
Every opnsense vm produce the symptoms like the bandwidth reduced to around 600/600Mbps.
This issue only occured since upgrade to 20.7. It was fine with the 20.1.
My friends also experiencing this issue, it seems its some kind of overhead because we experienced this:
1000/1000 connection measured around 600/600
500/500 connection measured around 300/300
300/300 connection measured arounf 200/200

Non of them is pppoe so its not mtu issue. We use the standard 1500byte on everywhere.
If i put a soho mikrotik device instead of the opnsense it can be translate the full gigabit without an issue.
Anybody knows anything about this?

update: its the same result with e1000e adapter too using vmware.
update: using proxmox with virtio works fine.
update: the same setup working perfect with pfsense which is based on Freebsd 11 not HBSD.
It seems the HBSD is the source of all problems...

Thx!

25

20.7 Legacy Series / Re: 20.7 & VMXNET3 to VMX Order

« on: October 20, 2020, 11:29:24 pm »

This is an old probably a decade old issue. And its related to the freebsd, nothing to do with the opnsense.
Honestly freebsd getting worse and worse every year. Im not sure its a good idea using as a base for anything anymore. Opnsense have a lot of issue recently what is mostly related to the freebsd.

26

20.7 Legacy Series / Re: Call for testing: netmap on 20.7

« on: August 28, 2020, 12:48:55 pm »

The bandwidth issue is because the netmap and the kernel. The main issue is the way too paranoid "hardened" bsd project. I use freebsd on other systems too, this whole project is starting to get annoying and its more like pain in the ass. Much more sideeffect than benefit. I really dont like this where the freebsd developement is going.

27

20.7 Legacy Series / Re: Call for testing: netmap on 20.7

« on: August 14, 2020, 12:22:21 pm »

@mb

Is 20.7.1 fixing the netmap issues adressed in the test kernel or would it set me back to the state before?

No, read the changelog. That is not fix the netmap issues.

"From the reported issues we still have more logging quirks to investigate and especially Netmap support (used in IPS and Sensei) is lacking in some areas that were previously working. Patches are being worked on already so we shall get there soon enough. Stay tuned."

That could mean anything...

Yes but one thing is certain. The fix is still not ready

28

20.7 Legacy Series / Re: Call for testing: netmap on 20.7

« on: August 13, 2020, 08:42:09 pm »

@mb

Is 20.7.1 fixing the netmap issues adressed in the test kernel or would it set me back to the state before?

No, read the changelog. That is not fix the netmap issues.

29

20.7 Legacy Series / Re: Performance problem after 20.7 upgrade

« on: August 07, 2020, 07:56:29 pm »

This probably related to the other issues what is related to the netmap+kernel issue.

30

20.7 Legacy Series / Re: Upgrade from 20.1 -> 20.7 failed when IPS/IDS enabled.

« on: August 07, 2020, 07:54:56 pm »

If i understand correctly this is not the opnsense issue but the freebsd and the kernel.
This "hardened" BSD project is a way to overkill for everyday use. Its cause trouble trouble trouble.