OPNsense
  • Home
  • Help
  • Search
  • Login
  • Register

  • OPNsense Forum »
  • Profile of opnfwb »
  • Show Posts »
  • Messages
  • Profile Info
    • Summary
    • Show Stats
    • Show Posts...
      • Messages
      • Topics
      • Attachments

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

  • Messages
  • Topics
  • Attachments

Messages - opnfwb

Pages: 1 [2] 3 4 ... 8
16
Hardware and Performance / Re: OPNsense 4x slower than PFSense on same hardware
« on: January 29, 2020, 06:53:02 pm »
Yes, I see what you mean with pfSense being able to sustain a higher throughput under the same circumstance.

Unfortunately, I'm not sure if this is useful data because it isn't telling us how fast either solution can actually route packets. In actual use, we'd be using pfSense or OPNsense as a firewall/router setup and we would want to see how quickly they can push traffic through themselves, rather than serving traffic directly through one interface.

I suppose the last things I would check would be to verify that OPNsense and pfSense both have OpenVMTools installed/running. And run a 'top -aSCHIP' on an SSH console for both of them and see what their CPU usage is when running your transfer test. Watching them under load may reveal a bottleneck, especially on the OPNsense router since that one seems to be under performing in your tests.

Finally, if you feel like digging in a bit more, I would recommend doing a test using Proxmox client and server VMs, and have two switches inside Proxmox. One can be used for the WAN port and one switch is a private switch with no physical uplinks, we'll use this for the LAN port. Doing this method will allow you to simulate actual routing performance of both solutions and you won't need an extra physical client.

17
Hardware and Performance / Re: OPNsense 4x slower than PFSense on same hardware
« on: January 29, 2020, 04:40:42 pm »
Just to confirm, does your setup look like the below diagram? OPNsense is not hosting the client or server portion of iperf, correct?

18
Hardware and Performance / Re: OPNsense 4x slower than PFSense on same hardware
« on: January 19, 2020, 10:37:43 pm »
A quick reply regarding the OPNsense CPU utilization. In my case this seemed to be related to DHCP6 being enabled out of the box. I'm not sure if OPNsense was trying to delegate a prefix to the LAN side over and over and causing high CPU usage on unbound? My logs are filled with this:

Code: [Select]
kernel: pflog0: promiscuous mode disabled
kernel: pflog0: promiscuous mode enabled

I was seeing these events spamming the logs constantly every second. As soon as I disabled DHCP6 on WAN, these errors went away and idle CPU usage on OPNsense returned to normal.

Here are the results of a current iperf3 test, using the same VMs described in my post above. These throughput numbers are much more consistent now that OPNsense has normal CPU usage.
Code: [Select]
Accepted connection from 192.168.1.232, port 4084
[  5] local 192.168.1.231 port 5201 connected to 192.168.1.232 port 24664
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec   185 MBytes  1.55 Gbits/sec
[  5]   1.00-2.00   sec   323 MBytes  2.71 Gbits/sec
[  5]   2.00-3.00   sec   315 MBytes  2.64 Gbits/sec
[  5]   3.00-4.00   sec   344 MBytes  2.88 Gbits/sec
[  5]   4.00-5.00   sec   316 MBytes  2.65 Gbits/sec
[  5]   5.00-6.00   sec   357 MBytes  2.99 Gbits/sec
[  5]   6.00-7.00   sec   353 MBytes  2.96 Gbits/sec
[  5]   7.00-8.00   sec   349 MBytes  2.93 Gbits/sec
[  5]   8.00-9.00   sec   356 MBytes  2.98 Gbits/sec
[  5]   9.00-10.00  sec   345 MBytes  2.89 Gbits/sec
[  5]  10.00-11.00  sec   305 MBytes  2.56 Gbits/sec
[  5]  11.00-12.00  sec   348 MBytes  2.92 Gbits/sec
[  5]  12.00-13.00  sec   341 MBytes  2.86 Gbits/sec
[  5]  13.00-14.00  sec   343 MBytes  2.87 Gbits/sec
[  5]  14.00-15.00  sec   331 MBytes  2.77 Gbits/sec
[  5]  15.00-15.04  sec  14.8 MBytes  3.04 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-15.04  sec  0.00 Bytes  0.00 bits/sec                  sender
[  5]   0.00-15.04  sec  4.81 GBytes  2.75 Gbits/sec                  receiver

19
Hardware and Performance / Re: OPNsense 4x slower than PFSense on same hardware
« on: January 19, 2020, 08:48:02 pm »
Here are my numbers. Both of these are fresh out of the box installs, OPNsense 19.7.9 and pfSense 2.4.4p3, both are X86_64.

Hypervisor Specs:
VMware ESXi 6.7u3
2x Intel Xeon E5620
All VMs are running open-vm-tools, including the firewalls

Specs on both firewall VMs are as follows:
2x CPU
4GB RAM
2x VMXnet3 NICs (one WAN, one LAN)

I have two other VMs running as iperf3 server and client. The "server" VM is on the WAN side of these firewalls, the client VM is on the "LAN" side. This is to test traffic throughput of the router itself. Never try to run these tests with the router/firewall acting as a client or server, you will not get accurate results.

pfSense 2.4.4p3:
Code: [Select]
Accepted connection from 192.168.1.230, port 56492
[  5] local 192.168.1.231 port 5201 connected to 192.168.1.230 port 45828
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec   314 MBytes  2.64 Gbits/sec
[  5]   1.00-2.00   sec   459 MBytes  3.85 Gbits/sec
[  5]   2.00-3.00   sec   407 MBytes  3.41 Gbits/sec
[  5]   3.00-4.00   sec   393 MBytes  3.30 Gbits/sec
[  5]   4.00-5.00   sec   351 MBytes  2.94 Gbits/sec
[  5]   5.00-6.00   sec   372 MBytes  3.12 Gbits/sec
[  5]   6.00-7.00   sec   424 MBytes  3.55 Gbits/sec
[  5]   7.00-8.00   sec   410 MBytes  3.44 Gbits/sec
[  5]   8.00-9.00   sec   443 MBytes  3.71 Gbits/sec
[  5]   9.00-10.00  sec   393 MBytes  3.30 Gbits/sec
[  5]  10.00-11.00  sec   448 MBytes  3.76 Gbits/sec
[  5]  11.00-12.00  sec   428 MBytes  3.59 Gbits/sec
[  5]  12.00-13.00  sec   404 MBytes  3.39 Gbits/sec
[  5]  13.00-14.00  sec   419 MBytes  3.51 Gbits/sec
[  5]  14.00-15.00  sec   445 MBytes  3.73 Gbits/sec
[  5]  15.00-15.04  sec  16.1 MBytes  3.26 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-15.04  sec  0.00 Bytes  0.00 bits/sec                  sender
[  5]   0.00-15.04  sec  5.98 GBytes  3.42 Gbits/sec                  receiver

OPNsense 19.7.9 (no tuning, Unbound using lots of CPU)
Code: [Select]
Accepted connection from 192.168.1.232, port 15150
[  5] local 192.168.1.231 port 5201 connected to 192.168.1.232 port 46858
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec   304 MBytes  2.55 Gbits/sec
[  5]   1.00-2.00   sec  88.9 MBytes   746 Mbits/sec
[  5]   2.00-3.00   sec   371 MBytes  3.11 Gbits/sec
[  5]   3.00-4.00   sec   164 MBytes  1.38 Gbits/sec
[  5]   4.00-5.00   sec   420 MBytes  3.52 Gbits/sec
[  5]   5.00-6.00   sec  79.4 MBytes   666 Mbits/sec
[  5]   6.00-7.00   sec   400 MBytes  3.36 Gbits/sec
[  5]   7.00-8.00   sec  97.7 MBytes   820 Mbits/sec
[  5]   8.00-9.00   sec   403 MBytes  3.38 Gbits/sec
[  5]   9.00-10.00  sec   399 MBytes  3.35 Gbits/sec
[  5]  10.00-11.00  sec   104 MBytes   872 Mbits/sec
[  5]  11.00-12.00  sec   374 MBytes  3.14 Gbits/sec
[  5]  12.00-13.00  sec  74.0 MBytes   621 Mbits/sec
[  5]  13.00-14.00  sec   289 MBytes  2.42 Gbits/sec
[  5]  14.00-15.00  sec   135 MBytes  1.13 Gbits/sec
[  5]  15.00-15.04  sec  3.24 MBytes   675 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-15.04  sec  0.00 Bytes  0.00 bits/sec                  sender
[  5]   0.00-15.04  sec  3.62 GBytes  2.07 Gbits/sec                  receiver

OPNsense 19.7.9 (set unbound to use Quad9 DoT using forwarding mode)
Code: [Select]
Accepted connection from 192.168.1.232, port 58840
[  5] local 192.168.1.231 port 5201 connected to 192.168.1.232 port 16760
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec   214 MBytes  1.80 Gbits/sec
[  5]   1.00-2.00   sec   268 MBytes  2.25 Gbits/sec
[  5]   2.00-3.00   sec   312 MBytes  2.61 Gbits/sec
[  5]   3.00-4.00   sec   315 MBytes  2.64 Gbits/sec
[  5]   4.00-5.00   sec   273 MBytes  2.29 Gbits/sec
[  5]   5.00-6.00   sec   259 MBytes  2.17 Gbits/sec
[  5]   6.00-7.00   sec   201 MBytes  1.69 Gbits/sec
[  5]   7.00-8.00   sec   279 MBytes  2.34 Gbits/sec
[  5]   8.00-9.00   sec   311 MBytes  2.61 Gbits/sec
[  5]   9.00-10.00  sec   120 MBytes  1.01 Gbits/sec
[  5]  10.00-11.00  sec   237 MBytes  1.99 Gbits/sec
[  5]  11.00-12.00  sec   298 MBytes  2.50 Gbits/sec
[  5]  12.00-13.00  sec   322 MBytes  2.70 Gbits/sec
[  5]  13.00-14.00  sec   291 MBytes  2.44 Gbits/sec
[  5]  14.00-15.00  sec   303 MBytes  2.54 Gbits/sec
[  5]  15.00-15.03  sec  8.95 MBytes  2.26 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-15.03  sec  0.00 Bytes  0.00 bits/sec                  sender
[  5]   0.00-15.03  sec  3.92 GBytes  2.24 Gbits/sec                  receiver

As we can see, OPNsense does seem to have some throughput limits out of the box. Still, I am seeing much higher throughput values than you are so it's important to make sure your tests are using servers/clients on the WAN and LAN sides of the firewall.

Finally, here's a screenshot of what a 'top -aSCHIP' looks like on the OPNsense 19.7.9 VM, you can see the high CPU usage for some reason with unbound. You may want to check if your OPNsense VM exhibits the same high CPU behavior, as that can also take away from the overall throughput.


20
19.7 Legacy Series / Re: Python 3.7 using max memory and filling disk space
« on: December 15, 2019, 07:22:24 pm »
I did a System/Firmware/Health audit and below are the results of this:

Code: [Select]
***GOT REQUEST TO AUDIT HEALTH***
>>> Check installed kernel version
Version 19.7.7 is correct.
>>> Check for missing or altered kernel files
No problems detected.
>>> Check installed base version
Version 19.7.7 is correct.
>>> Check for missing or altered base files
No problems detected.
>>> Check for and install missing package dependencies
Checking all packages: .......... done
>>> Check for missing or altered package files
Checking all packages: .......... done
***DONE***

At this point I'm stumped. Not sure what caused this? I've run memcheck on the firewall as well and have no errors, so I don't think its hardware related. It has been stable for months until the odd Python CPU usage and disk fill yesterday.

21
19.7 Legacy Series / Re: Python 3.7 using max memory and filling disk space
« on: December 15, 2019, 06:17:09 pm »
When I manually run that command I receive back "status" : "ok"

This is a stock install, I think I started with an early version of 19.7 and have upgraded it as updates were made available through the OPNsense repos. I have not made any customizations to packages and have only installed two packages available on the mirrors, WOL and Smart.

One thing I also wanted to add, the "python3.7.core" file that was filling the disk was not removed on a reboot. I shutdown the firewall gracefully and powered it back up. When it came up, the disk space was completely exhausted and I had to manually remove the "python3.7.core" file to get the space back.

22
19.7 Legacy Series / Re: Python 3.7 using max memory and filling disk space
« on: December 14, 2019, 08:56:17 pm »
Here's the output of my filter_tables.conf file.


23
19.7 Legacy Series / Re: Python 3.7 using max memory and filling disk space
« on: December 14, 2019, 07:21:40 pm »
Thank you for the response. This is a fairly basic install and I don't use aliases. Here's a screenshot of my firewall aliases table.

The only packages I have installed are wake-on-lan and smart tools. I'm not using Suricata or Sensei on this install.

24
19.7 Legacy Series / Re: Python 3.7 using max memory and filling disk space
« on: December 14, 2019, 05:48:56 pm »
Looks like Python is completely filling the disk. I tried restarting all of the services shown on the dashboard homepage, this did not have any impact on the disk fill. I'm shutting down the firewall now, these are the last screen caps I grabbed before shutdown.


25
19.7 Legacy Series / Re: Python 3.7 using max memory and filling disk space
« on: December 14, 2019, 05:31:56 pm »
Replying 2nd time to add more attachments due to size limits.

26
19.7 Legacy Series / Python 3.7 using max memory and filling disk space
« on: December 14, 2019, 05:31:27 pm »
I noticed this morning that Python 3.7 seems to be maxing out my memory and is filling up the disk space on my SSD. I have not made any config changes to the firewall for the last week, the last config change that I made a week ago was to change the firewall optimization from "standard" to "aggressive". The firewall has been very stable for weeks without rebooting until today. It is currently passing traffic for now.

Is there a reason why Python 3.7 would fill the disk like this? This seems like a major problem in how it is functioning.

Firewall specs are as follows:
Intel J3455 (bare metal install)
16GB RAM
120GB SSD
Dual Broadcom NICs
OPNsense 19.7.7-amd64
FreeBSD 11.2-RELEASE-p16-HBSD
OpenSSL 1.0.2t 10 Sep 2019

Below are the screenshots showing the problem.

27
General Discussion / Re: Installation of Opnsense Minipc with Intel atom E3845 and quad core
« on: December 11, 2019, 05:05:02 am »
There was another user that posted here a few days ago with a similar issue. See this thread for a work around, it's related to a FreeBSD bug.

https://forum.opnsense.org/index.php?topic=11869.0

28
Hardware and Performance / Re: Large sawtooth CPU pattern at idle
« on: December 11, 2019, 05:03:42 am »
Unfortunately, I've noticed the same behavior since upgrading to the 19.7.x series. There was a thread on it early on when 19.7 was first released but I think it fell off the radar.

I know there was a push to migrate from Python 2.7 to 3, and it was theorized that this caused some of the usage increased. However, I've seen a significant increase in CPU usage and a noticeable delay in the webGUI when clicking and opening pages in OPNsense. It doesn't seem to impact the throughput or functionality of the system as a firewall, but it's been a step backwards in terms of usability.

Sorry I don't have a solution but, I'm also seeing the same thing for many months now with 19.7. Link to the 19.7 CPU usage thread for reference: https://forum.opnsense.org/index.php?topic=13507.0

29
General Discussion / Re: System stuck at "Booting... \"
« on: December 06, 2019, 05:48:14 am »
I recalled reading about this on the last version. This seems like the issue you are experiencing. I would try the fix listed in this thread and see if that helps? It looks to be a bug inherited from FreeBSD.

https://forum.opnsense.org/index.php?topic=11869.0

30
19.7 Legacy Series / Re: Zotac CI329 Nano - Error 19 during installation
« on: December 02, 2019, 03:32:39 am »
The error messages mention OPNsense Nano. Are you able to duplicate these same errors when using the full version of OPNsense and booting that off of a USB stick?

Pages: 1 [2] 3 4 ... 8
OPNsense is an OSS project © Deciso B.V. 2015 - 2021 All rights reserved
  • SMF 2.0.17 | SMF © 2019, Simple Machines
    Privacy Policy
    | XHTML | RSS | WAP2