OPNsense on a standalone vmWare host 100% CPU

[Styx]

Hi there,
I'm using pfSense vmWare since years and i want to try to replace it with OPNsense instead for lots of reasons ^^.
Actually, i've got 2 VM with OPNsense in HA.

Each VM is using 100% of the CPU. I had to limit CPU frequency, but it isn't a solution.
I tried :

• Restarting VM
• Verifying that there were not surallocation on the hypervisor
• Adding CPU and RAM
• Checking that network cards are "e1100" on vmWare and that settings in OPNsense for Hardware CRC/TSO/LRO are disabled
• Resintalling os-vmware Tools (wich is recognized by the vmWare)

I take a look in the System > Activity , there is only the process "idle" taking early all the CPU ?



What can i do ?,

Thx for your help and sorry for my english!

EDIT : there aren't additionnal plugins (other than vmware Tools) and this is the same on a single VM without HA

[phoenix]

There is no such process as "idle", that is the state of the processor and shows you that the cpu is idle i.e. nothing is consuming any cpu resources.

[Styx]

Thanks for your answer.

There is no such process as "idle", that is the state of the processor and shows you that the cpu is idle i.e. nothing is consuming any cpu resources.

I would like so.

For me, the idle task is used for process accounting, and also to reduce energy consumption. As far i know, in Linux, one idle task is created for every processor, and locked to that processor; whenever there’s no other process to run on that CPU, the idle task is scheduled.

Anyway, like i've explained before, if the frequency of the VMs is not limited, they will take 100% of the CPU from the ESXi host (as you can see in my previous post, the VM FW1 takes immediately 100% of the host CPU when I suppress the 200Mhz limitation like on FW2).



I don't think this is normal...

[Supermule]

What version of OPNsense and can you post pictures of your VM's CPU performance??

[Styx]

What version of OPNsense and can you post pictures of your VM's CPU performance??

Virtualized OPNsenses 20.1 / HA
vmWare ESX standalone host 7.0

[Supermule]

309 mhz is VERY low.....

[Styx]

309 mhz is VERY low.....

Yes, when a VM is virtualized, there is a gap between ressource provided and really used.
Each pfsense VM are using between 0,06 and 0,1GHz with no limitations.
On the OPNsenses, like I explained before, if I let "unlimited", the VM will take all CPU from all the cores of the esxi host without consideration and will be as slow as actually.

So, like I said, it's not the solution but I set limitations on CPU to 0,3 GHz (where a pfsense use 0,06GHz).

Anyway, remove limitations doesn't make any changes apart stressing the esx host CPUs.

What can I do?

Thanks.

[Supermule]

I think youre wrong...

How many vm's are on that server??

You can easily set the shares on the Vcpu's so that the performance doesnt take a hit like you say.

[Styx]

I think youre wrong...

How many vm's are on that server??

You can easily set the shares on the Vcpu's so that the performance doesnt take a hit like you say.

Only OPNsenses VM are slow in my case even ihfthe esx host deliver all cpu to the OPNsenses VMs without reason.

There are 5 VM (2 Pfsense, 2 OPNsense anf one Windows Server 2019) on the esx host.
I have shut down all Pfsense and windows VM, remove limitations : the esx host and the OPNsenses VM are at 100%.

I may be wrong but I don't think that all vCPU assigned to the VM need to run at 100% on i7 2,7gHz.

Is it possible that there is something to check on OPNsense default parameters or VM parameters which can explain this CPU consumption?

[Supermule]

If you see CPU idle on OPNsense and thats close to 100% then everything is fine. It does nothing so it schedules idle time...

How many cores do you have?

[Styx]

If you see CPU idle on OPNsense and thats close to 100% then everything is fine. It does nothing so it schedules idle time...

How many cores do you have?

You're right. It's the only clue visible on diagnostic page. I shouldn't have mentioned it.
8 cores and I can't explained that cpu goes to 100% with only 2 VM. I believe there is something to change. I've also tried to convert thin to thick storage with FW1 but same result...

[Supermule]

Can you post the CPU performance graph over time on the VM if you click on the performance tab??

Otherwise send me a teamviewer ID and code, I will log on and investigate if that works for you?

[Styx]

Here it is, i just start the VM :



ESXTOP :

Code: [Select]

9:50:28pm up  6:22, 621 worlds, 7 VMs, 8 vCPUs; CPU load average: 0.71, 0.73, 0.79
PCPU USED(%):  12 4.0  17  25  12 105  41  60 AVG:  34
PCPU UTIL(%):  10 4.6  12  15 9.7  70  28  41 AVG:  24
CORE UTIL(%):  13      26      78      68     AVG:  46

        ID        GID NAME             NWLD   %USED    %RUN    %SYS   %WAIT %VMWAIT    %RDY   %IDLE  %OVRLP   %CSTP  %MLMTD  %SWPWT
     38282      38282 FW1                10  258.86  171.40    0.00  828.48    0.02    0.29   29.53    0.15    0.00    0.00    0.00


• NWLD = 10. There are a lot of suscribed VM wich are not ON. It explains the value of %OVRLP
• %USED = %RUN + %SYS - %OVRLP. Percentage of physical CPU core cycles used. In fact, i have to improve the load of the Guest OS
• %RUN if it represent the percentage of total scheduled time, the system is in contention or overloaded
• 100%= %WAIT + %RDY + %CSTP + %RUN. As far as i know %WAIT represent the percent of time that the VM have to wait for a VMkernal activity. Increase storage performance can resolve this in case of latency but the Datastore is currently on Local SSD. To go further : this percentage also includes %IDLE. The theoretical maximum of %WAIT is (NWLD*100). It's nothing wrong when there is a high value because the idle time is included in %WAIT.

So i don't know if there is something on OPNsense or on vmWare but i don't understand why this fork doesn't work well...Maybe should i try to reinstall it in x86 ?

[Supermule]

Can you decrease the number of cores to 1 and check if there is an update to Opnsense?

Latest is 20.1.4...

[Styx]

I've decrease the number of cores on the VM and OPNsense is now updated to 20.1.4, it seems to keep the same behavior .... 
Maybe do i test with the devel one ?