Messages

1

23.7 Production Series / Re: PPPoE drop/disconnect, requires a reboot to fix

« on: November 29, 2023, 08:02:02 pm »

can you switch out the 82574 for something newer?  I had stability issues with the em driver and an 82576 - went away going to something which is more current ( igb, ixgbe etc )

2

Hardware and Performance / Re: Suggestions for troubleshooting slow nat performance (throughput)?

« on: September 18, 2023, 11:25:48 am »

found a block diagram for a 'typical' h77/z77 board - the 2 pci slots are off a pci/pcie bridge chip and the upstream link is pcie3x1 - that should be heaps for 4 x gbe links ( presuming the bridge chip isn't utter rubbish )

I'd check core loading on the firewall when running a speed test from a lan client to wan and see if you're saturating a single core ( ssh in to opnsense, run 'top -P' and watch the 'interrupt and 'idle' column ).

I dug up the pro/1000 mt dual specs pdf - no mention of rss on it at all that I can see, freedBSD used to have a separate 'emx' driver with rss support for some of those older cards but it doesn't list the 82546, just newer variants ( https://man.dragonflybsd.org/?command=emx&section=4 ). Suspect you need a new NIC

3

Hardware and Performance / Re: Suggestions for troubleshooting slow nat performance (throughput)?

« on: September 18, 2023, 11:10:13 am »

PCI is very bandwidth constrained compared to PCIe - and with 4 gbe nics on PCI you're almost certainly running into those constraints

I'd replace those old pro1000 cards with either a couple of iintel i21x series cards (if you only need a couple of ports ) or if you really need 4 ports something like an i350-t4 - again well supported NICs for freeBSD.

you'll also probably want to look at enabling RSS once you've got the cards replaced - the i7-3770 has pretty low single-core perf by modern standards and without RSS you can end up limited by that 

4

Zenarmor (Sensei) / Re: Throughput drops massively with option selected (even in bypass)

« on: September 12, 2023, 03:02:21 am »

https://forum.opnsense.org/index.php?topic=24409.0

5

Zenarmor (Sensei) / Re: Throughput drops massively with option selected (even in bypass)

« on: September 07, 2023, 08:17:11 pm »

Hi,

With regards to Zenarmor, we have identified two steps that can help improve its performance. Firstly, we will be implementing a feature that pins a random core to the system. Currently, it is pinned to CPU 1 when you check the box. Secondly, we are working on providing multicore support to the engine, which is expected to be shipped in November."

Let me know if you need any further assistance.

the original req for the 'do not pin' option ( which was made by me ) was to increase perf on modern multi-core cpus ( and espec under esxi ) - and up to the latest release it did the 'right' thing ( didn't pin to any core, let the scheduler assign the process at will )

there was a regression in the new version where 'do not pin' stopped doing that and instead right now pins the eastpect process to cpu0 ( which is an even WORSE option than the default of pining cpu1 if you haven't got rss enabled as cpu0 tends to get hammered by interrupts )

Again I have an open req to revert to the previous behaviour open - how come your devs are instead deciding 'pin to a random cpu' is a better fix ( rule #1 - don't fight the kernel's scheudler - it generally knows more than you )?

Anyway - very much looking forward to full multicore support

for the op - if yout want to revert to previous ( and what I consider 'correct' behaviour) save the script below and create a cron job to run it every 10 or 30mins ( eastpect doesn't get restarted often ).

Also consider turning on RSS as it definitely helps performance in vm's with a lot of vcpus assigned

#!/bin/sh

eastpect_instance_0=`ps -ax | awk '$5 ~ "eastpect" && $0 ~ "Instance 0"  { print $1 }'`

echo "Eastspect Instance 0 pid=" $eastpect_instance_0

#echo "current cpu affinity"
cpuset -g -p $eastpect_instance_0

# change affinity to all cores
cpuset -l ALL -p $eastpect_instance_0

#echo "new cpu affinity"
cpuset -g -p $eastpect_instance_0

exit 0

6

Zenarmor (Sensei) / Re: 'do not pin engine packet processors to dedicated CPU cores' broken

« on: August 10, 2023, 12:05:04 am »

logs etc sent

note that I re-enabled the cron job I used to have prior to the GUI option to un-pin, so if the logs include the current cpuset output for eastspect it'll show it using 0-7 due to the cron job 'un-pinning' the process - without the cron job easpect pins to cpu 0 if 'do not pin' is selected, and defaults back to cpu 1 if 'do not pin' is unticked

7

Zenarmor (Sensei) / 'do not pin engine packet processors to dedicated CPU cores' broken

« on: August 09, 2023, 10:37:31 pm »

despite having the option 'do not pin engine packet processors to dedicated CPU cores' selected I can see that eastpect is running ONLY on cpu 0 ( and watching core loading with htop whilst stress testing confirms this )

cpuset -g on the easptect pid gives this :

Eastspect Instance 0 pid= 61127
current cpu affinity
pid 61127 mask: 0
pid 61127 domain policy: first-touch mask: 0

if I manually set the cpu mask to all cores using cpuset I see what I would expect and much more even cpu utilisation at high loads

Eastspect Instance 0 pid= 61127
current cpu affinity
pid 61127 mask: 0, 1, 2, 3, 4, 5, 6, 7
pid 61127 domain policy: first-touch mask: 0

If I un-tick 'do not pin' it sets affinity to cpu 1 as previously, so looks like in the new version there's been some sort of regression in the 'do not pin' code ( ticking it now basically pins the eastpect process to cpu 0 - which is obviously NOT what is intended )

8

Hardware and Performance / Re: usb ethernet ASIX AX88179A

« on: June 25, 2023, 09:52:36 pm »

...next time buy an optiplex SFF and you can add PCI NICs, problem 100% solved ;-)

Is there a recommendation for a micro or USFF device than can take a single or dual nic?

Intel do NUC models with dual lan ports ( latest ones are dual 2.5gbe ) - not as cheap as the topton etal boxes, but reliable, have proper support with BIOS updates and are rated for 24/7 operation by Intel

9

Hardware and Performance / Re: Opnsense/Zenarmor Performance problem.

« on: June 15, 2023, 09:41:12 pm »

if it's a sg330v2 it's got an i5-6500 in it - which is easily double the single core perf of the xeon 2420 ( whilst puilling a fraction of the power ) - plus of course it's got faster RAM which will again help ( ddr4 vs ddr3 )

10

Hardware and Performance / Re: How to Troubleshoot a PC that shuts off under high load

« on: June 14, 2023, 09:32:38 pm »

first thing I'd try is a different known good NIC

Also the pro/1000s are pretty ancient - ideally you want something newish and with good driver support ( pair of i210/i219s or maybe an i350-t2/t4 or even an x540-t2/x550-t2)

11

Hardware and Performance / Re: Opnsense/Zenarmor Performance problem.

« on: June 12, 2023, 09:45:39 pm »

presumably those are 2420 (v0) in the g8 - like most of those old xeons that have terrible single-core perf, and zenarmor is currently single threaded - you'd need something with about double the single core perf to not bottleneck at <1gbps ( if you want to run zenarmor at full 10gbps that's going to be an even bigger problem - until it's multi-threaded I finding a cpu with high enough single core perf is an issue )

12

Hardware and Performance / Re: opnsense with Intel X710-DA2 Bandwidth low

« on: May 10, 2023, 08:28:37 pm »

first thing to try would be running opnsense bare-metal rather than virtualised - it's the only way to tell if the issue is the hypervisor and your use of vmnics, or you're actually running into hardware limits ( I presume you've already watched cpu load whilst running tests to check you're not bottlenecking on a single core)

13

Zenarmor (Sensei) / Re: V1.13 after upgrade does not block web sites

« on: May 07, 2023, 10:29:53 pm »

I'm seeing very high RAM usage to (+/-80% of 4GB) with high usage related to what appears to be ZenArmor services

main culprit seems to be elastisearch - eastpect itself has a big footprint but very little actually resident

CPU   PID USER       PRI  NI  VIRT   RES▽S CPU% MEM%   TIME+  Command
  7 56969 elasticsea  52   0 4721M 2150M S  0.0 26.4 32:18.91 /usr/local/openjdk8/bin/java -Xms2g -X
  0 39443 root        20 -20 18.8G  421M S  0.0  5.2 10:40.59 eastpect: Eastpect Instance 0
  0 36637 root        20 -20  861M  202M S  0.0  2.5  0:43.57 /usr/local/sensei//bin/eastpect -D
  0 39574 root        52 -20  858M  201M S  0.0  2.5  0:00.00 eastpect: Eastpect Streamer Instance

14

23.1 Legacy Series / Re: Error with GPT partitions on VMWare ESXi 7.0

« on: March 06, 2023, 08:31:33 pm »

what controller type did you choose when creating the vm - vmware paravirtual or LSI logic SAS?

when adding a vm in esxi7 and choosing 'freebsd 13 64-bit' as the OS ( which is of course the right choice for opnsense ) the default controller setting is 'vmware paravirtual' - however this won't work with the bootable image, you need to change it to 'lsi logic sas', run the install and THEN once the install is complete and you're booting off the vm disk ( as opposed to the bootable .iso ) you can then change to vmware paravirtual

15

Zenarmor (Sensei) / Re: Is Zenarmor looking into the NETMAP bug?

« on: February 25, 2023, 07:54:05 pm »

in my experience the best way to deal with freeBSDs intel driver issues ( which are beyond opnsense and zenarmour's control - issue is mainly with Intel themselves ) is simply to virtualise opnsense using esxi and use the vmx drivers - opnsense/zenarmour is solid using vmx ( vmx supports netmap native )

The esxi ix/ixl drivers are super reliable ( as you'd expect ) - i225/i226 is supported by the community driver