OPNsense Forum
English Forums => Hardware and Performance => Topic started by: glubarnt on January 31, 2020, 07:46:41 pm
-
Hi everyone,
I am in the process of migrating from a dd-wrt-Router to a virtualized setup with OPNsense.
I installed it three days ago with 19.7, updated today to 20.1.
As I mentioned, OPNsense runs as a VM. The hypervisor is kvm on CentOS 8. CPU is a AMD EPYC 7282.
OPNsense gets two cores and 4GB ram.
It also gets one nic via virtio, which is a bridge. Another nic is passed directly from the hypervisor to the vm via macvtap.
Directly after the installation I noticed that the cpu was at 100% constantly, which left me wondering, because the system was not doing anything.
Looking at top on the shell, I see that unbound constantly has 6-10% cpu-usage and python 6-8%.
All other processes are below 0%.
Still, CPU has 36-45% user load, 30-45% system load and 5-40% idle.
Where could that load be coming from?
Looking in the webgui, I see a command "[rand_harvestq]" hovering around 30-50% CPU, while [idle{idle: cpu0}] is at the top most of the time.
Any clues what could the problem could be here?
Any help is much appreciated.
-
I saw similar issues on a fresh virtualized install. In my case, I was also seeing pflog0 promiscuous enabled/disable messages spamming the logs many times per second. This seemed to be related to IPV6 unable to pull a prefix delegation on the WAN interface of the OPNsense VM.
Try disabling IPV6 on WAN and see if this clears up? If so, it's likely related to the issue I saw in my LAB.
-
Hi,
thanks for your tip.
To my surprise it actually helped a little.
The load goes from 0% to 50% now, according to the dashboard.
That is better, but still not great for a system not serving any clients yet :D
-
Since the process rand_harvestq is using a significant amount of CPU, you could investigate changing the value of "kern.random.harvest.mask".
The default setting in OPNsense is kern.random.harvest.mask=2047
root@OPNsense:~ # sysctl kern.random.harvest
kern.random.harvest.mask_symbolic: UMA,FS_ATIME,SWI,INTERRUPT,NET_NG,NET_ETHER,NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED
kern.random.harvest.mask_bin: 000000000011111111111
kern.random.harvest.mask: 2047
The UMA (universal memory allocator also called zone allocator) has a potentially high rate. I don't know if UMA acts any differently on a VM vs hardware.
You could determine if CPU load decreases when lowering kern.random.harvest
kern.random.harvest = 2047 OPNsense default
kern.random.harvest = 1023 don't use UMA
kern.random.harvest = 511 FreeBSD default
kern.random.harvest = 351 max throughput according to some documentation that I found
You can set it using sysctl in a shell for testing, but I found to set the value permanently, I have to use the GUI: System -> Settings -> Tunables and add kern.random.harvest.mask
Disclaimer: If any of my information is incorrect, please correct me.
Good Luck
-
What machine type and nic driver type are you passing to the vm from KVM?
-
I think I read somewhere that this is related to IPv6 tracking in an interface where v6 is not available, where disabling v6 solved this.
-
Thanks a lot for the additional input.
Since the process rand_harvestq is using a significant amount of CPU, you could investigate changing the value of "kern.random.harvest.mask".
The default setting in OPNsense is kern.random.harvest.mask=2047
root@OPNsense:~ # sysctl kern.random.harvest
kern.random.harvest.mask_symbolic: UMA,FS_ATIME,SWI,INTERRUPT,NET_NG,NET_ETHER,NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED
kern.random.harvest.mask_bin: 000000000011111111111
kern.random.harvest.mask: 2047
The UMA (universal memory allocator also called zone allocator) has a potentially high rate. I don't know if UMA acts any differently on a VM vs hardware.
You could determine if CPU load decreases when lowering kern.random.harvest
kern.random.harvest = 2047 OPNsense default
kern.random.harvest = 1023 don't use UMA
kern.random.harvest = 511 FreeBSD default
kern.random.harvest = 351 max throughput according to some documentation that I found
You can set it using sysctl in a shell for testing, but I found to set the value permanently, I have to use the GUI: System -> Settings -> Tunables and add kern.random.harvest.mask
Disclaimer: If any of my information is incorrect, please correct me.
Good Luck
I tried the tunable and I think it also brought the load down a little bit more, but reading this comment:
What machine type and nic driver type are you passing to the vm from KVM?
I looked up what machine type I created and by accident, it is a i440FX machine, which is not what I want.
The nic gets passedthrough, driver is virtio.
Anyways, as this is not a system that serves any traffic yet, I am gonna go ahead and resetup, because migrating i440FX to PC-Q35 is nasty and I do not want to mess it up.
I will keep you posted on the status.
Thanks again for the input and ideas.
-
I use i440fx and virtio with seabios because it works the best for me after testing all the different combinations, and uses the least cpu. I think trying to use q35 would be really hard.
-
I agree, because of some bug, Q-35 does not seem to be possible at the moment.
I think I am gonna leave this machine running for some time and play around with the kern.random.harvest.mask.
-
So,
after setting kern.random.harvest.mask to 511 and observing the load of the system for a few days, I think this is solved now.
Without serving any traffic the system sits at 0%-8% load, which is fine for me.
Thanks a lot for your help.