Messages

If I enable Gateway Monitoring: System > Gateways > Configuration > edit WAN Gateway configuration > Uncheck Disable Gateway Monitoring
does this add any noticeable overhead (CPU load) to low-powered firewalls?  (like my ancient APU2D4)

Thank you for the clarification. I didn't know if the setting had any effect on a non-NUMA CPU.
I apologize for the noise.

OPNsense 24.7.5
This update also disables NUMA by default which can bring a boost in network throughput on affected systems. 
system: default to vm.numa.disabled=1


OPNsense 25.1.1
execute: sysctl vm.numa.disabled and the value is vm.numa.disabled: 0
I've added "vm.numa.disabled" to Tunables and verified that vm.numa.disabled="1" is in /boot/loader.conf
After reboot, the value is still vm.numa.disabled: 0

Since I have an APU2D4, I would like to explicitly disable NUMA.
Am I missing something?

[IF]

By chance, did you verify that hint.uart.0.at="isa" (& hint.uart.1.at="isa") were in /boot/device.hints before upgrade? To verify that "isa" is the correct value?
Also, was hint.uart.0.at="isa" (& hint.uart.1.at="isa") in /boot/loader.conf after you added the values in Tunables and saved?

If the upgrade worked correctly, there is the possibility that the web interface is working. Did you try to connect to web interface?
For me, watching the console simply confirmed that the upgrade worked.

IF the web interface is working, you could possibly go to System:Settings:Administration and enable Secure Shell.
Login with Secure Shell and you should be able to manually edit/boot/device.hints and check that hint.uart.0.at="isa" is in /boot/loader.conf

Good Luck!

I've been using an APU2D4 since OPNsense 19.1 and upgrading OPNsense along the way to 25.1 (current). Still working fine, but FreeBSD hardware support might be questionable in the relatively near future.

possible OPNsense hardware:
   maybe Intel N150 CPU - starting to be available and would be a huge improvement (maybe, "overkill" but the same CPU power usage)
   4 port i226 network
   serial port or USB for console access
   don't need (or want) WiFi
   no cooling fan

Does anybody have experience with the (inexpensive) Chinese mini PC manufacturers?  Topton, CWWK/Changwang, HUNSN, SJRC, HKUXZR, etc
Many of the models from different manufacturers appear to be identical, at least, the cases and specs look the same.

quality of hardware?   decent or the typical low-quality chinese stuff?
updating BIOS?   requirement to be able to update BIOS to either AMI or Coreboot (preferred)

I was thinking that I would buy a barebones box, purchase decent quality memory and SSD, flash BIOS to current AMI or Coreboot. I really don't trust software coming from China even though it's probably the generic AMI BIOS.

If anybody has better recommendations, I'm very interested.

I successfully upgraded my ancient APU2D4 (with serial port console) on my home network.

THANK YOU for the fix provided in this discussion thread! Otherwise, the upgrade would have caused some panic for me when the console would have stopped working.

In Tunables, I simply added hint.uart.0.at = isa (and hint.uart.1.at = isa). This added the settings to /boot/loader.conf and superceded the values in /boot/device.hints. Just for completeness, I updated the values in /boot/device.hints after the upgrade.

The upgrade from 24.7.12_4 to 25.1 was smooth. THANK YOU, Franco!

On a side note, I will start investigating new hardware for my firewall. That's a discussion for the Hardware and Performance forum.

I was configuring my network optimization based on information from https://calomel.org/freebsd_network_tuning.html

I disabled flow-control (dev.igb.0.fc=0  # (default 3)) as described in this section of sysctl configuration:

# Intel i350-T2 igb(4): flow control manages the rate of data transmission
# between two nodes preventing a fast sender from overwhelming a slow receiver.
# Ethernet "PAUSE" frames will pause transmission of all traffic types on a
# physical link, not just the individual flow causing the problem. By disabling
# physical link flow control the link instead relies on native TCP or QUIC UDP
# internal congestion control which is peer based on IP address and more fair
# to each flow. The options are: (0=No Flow Control) (1=Receive Pause)
# (2=Transmit Pause) (3=Full Flow Control, Default). A value of zero(0)
# disables ethernet flow control on the Intel igb(4) interface.
# http://virtualthreads.blogspot.com/2006/02/beware-ethernet-flow-control.html
#
dev.igb.0.fc=0  # (default 3)

Admittedly, on my lightly-loaded home network, it probably makes no difference whether flow control is on or off.

I'm just trying to get best performance out of my little apu2d4 firewall.

I have an apu2d4.
Upgraded from 22.7.11 to 23.1 with no apparent errors.
However, after upgrade, networks (LAN/WAN) didn't work. No response.
Working on the console, I finally tried to manually stop and start the network interfaces.
"ifconfig igb0 down" then "ifconfig igb0 up".  (same for igb1)  Both interfaces started working.
After a reboot, I still needed to stop/start the interfaces to get working.

Previously, I did everything I could think of to optimize network performance including setting dev.igb.X.eee_control=0 in Tunables.
Using the recommendation earlier in this discussion, I deleted dev.igb.X.eee_control from Tunables.
Now, after reboot, network interfaces are starting and working normally.  I'm keeping dev.igb.X.fc=0

Just telling my experience to possibly help others.

My firewall is a PC Engines APU2 and I keep up with PC Engines on GitHub (https://pcengines.github.io/)

Does anybody know about miczyg, Michał Żygowski ?
There have been no updates for PC Engines since late August 2022.
The last activity on GitLab for Michał was on September 30, 2022.
I know that he has worked very hard on PC Engines CoreBoot.
Is he OK or is he taking some time away from PC Engines CoreBoot ?

Gary7

FWIW, today I saw that 'Reporting'->'Health'->'System'->'States' had the wrong display and it started when I upgraded to 22.1. I've had a couple of reboots since.

I searched the forum and found this discussion.
My home firewall is a very simple 1 WAN & 1 LAN on an APU2.
My speed is exactly the same as pre-22.1 ( a little over 200 Mbps, my full Internet speed )
I'm running Unbound with blocklists and a few Firewall:Rules:LAN, i.e.spamhaus_drop
Nothing inbound.
I've set several tunables to try to get max performance.

I just noticed that Reporting:Settings still has Round-Robin-Database enabled (from pre-22.1). Should I shut-off RRD graphing backend ?
Since I have only 1 WAN, I believe that I can safely shut-off RRD graphing backend.
[Update] Looks like I need to have RRD graphing enabled to display Health graphs.

As always, Franco, you do outstanding work.
Thanks

Using a shell, can you run "top" and see if you have a process that running at 100% ?

On my little APU2 after upgrade, unbound was running at 100% and never dropped to idle.

In Unbound DNS: Blocklist, I removed all of the blocklists and even disabled DNSBL.
When I applied the changes, unbound started and dropped to idle right away.
Then, I enabled DNSBL and added blocklists.
I add several blocklists, apply, and unbound starts, adds the blocklists, and drops to idle relatively quickly.
However, there is one very large (50 MB) blocklist that seems to cause constant 100% load for unbound. I'll investigate further this weekend.

Just a couple of thoughts based on my sysadmin experience.
1)This will require some manual monitoring of CPU load, but could you see if the CPU spikes are occurring randomly or. possibly, almost exactly on the minute? i.e. a cron job running on the minute.
2) Is your WAN interface connected to anything? Did you give it a static address? Is it possible that the O/S is trying to configure your WAN interface with DHCP at regular intervals?

Good luck.

I thought that if_bridge was required for OPNsense routing and/or firewall functions.

I am only using a single WAN and a single LAN interface.

In my (misguided) attempts to get maximum performance out of OPNsense, I have a some questions about the need for certain loaded kernel modules.

I have a VERY simple home configuration: no in-bound traffic, no high-availability(CARP), no iPSEC, no tunneling of any kind, no LAGG, no PPP, and no VLAN

Is there any advantage (or disadvantage) to not loading certain modules since I won't be using them?

carp_load="NO"      #Common Address Redundancy Protocol (CARP)
if_enc_load="NO"    #encryption needed for IPSEC
if_gif_load="NO"     #generic tunnel interface
if_gre_load="NO"    #Generic Routing Encapsulation
if_lagg_load="NO"   #link aggregation and link failover
if_tap_load="NO"    #Ethernet tunnel software network interface (for virtualization?)
if_tun_load="NO"    #tunnel driver (user process ppp)
if_vlan_load="NO"   #IEEE 802.1Q VLAN network interface

As a test, I added this to /boot/loader.conf.local and rebooted.
I know that it's reading these local settings because the order of modules displayed by kldstat changes.
kernel modules moved down in the list:
13    1 0xffffffff82a2e000     6890 carp.ko
14    1 0xffffffff82a35000      d7a if_enc.ko
15    1 0xffffffff82a36000     4bba if_gre.ko
16    1 0xffffffff82a3b000     a230 if_lagg.ko
17    1 0xffffffff82a46000     30c1 if_tap.ko
and modules not loaded
    if_gif_load
    if_tun_load
    if_vlan_load

Apparently, carp, enc, gre, lagg, and tap are getting loaded later during boot.
On the dashboard, I'm getting a CARP error. Since I don't use CARP, I'm ignoring it.

Is there any possibility there would be lower kernel overhead by not loading these modules? Other, than some slight reduction of in-memory kernel size?

As a side note, FreeBSD 13 has the possibility of a VERY nice performance increase due the improvements in if_bridge and other optimizations.

I took a look at the cloudflare.com SSL certificates.
Here is the list of addresses, Common Names, and Subject Alternative Names (SAN)

Cloudflare SSL certificates

Addresses: 1.1.1.1  &  1.0.0.1
Common name: cloudflare-dns.com
                SAN: DNS Name=cloudflare-dns.com
                        DNS Name=*.cloudflare-dns.com
                        DNS Name=one.one.one.one
                        IP Address=1.1.1.1
                        IP Address=1.0.0.1
                        IP Address=162.159.36.1
                        IP Address=162.159.46.1
                        IP Address=2606:4700:4700:0000:0000:0000:0000:1111
                        IP Address=2606:4700:4700:0000:0000:0000:0000:1001
                        IP Address=2606:4700:4700:0000:0000:0000:0000:0064
                        IP Address=2606:4700:4700:0000:0000:0000:0000:6400


Addresses:  1.1.1.2  &  1.0.0.2
Common name: security.cloudflare-dns.com
                SAN: IP Address=2606:4700:4700:0000:0000:0000:0000:1112
                        IP Address=2606:4700:4700:0000:0000:0000:0000:1002
                        DNS Name=security.cloudflare-dns.com
                        DNS Name=*.security.cloudflare-dns.com
                        IP Address=1.1.1.2
                        IP Address=1.0.0.2

Addresses:  1.1.1.3  &  1.0.0.3
Common name: family.cloudflare-dns.com
                SAN: IP Address=2606:4700:4700:0000:0000:0000:0000:1113
                        IP Address=2606:4700:4700:0000:0000:0000:0000:1003
                        DNS Name=family.cloudflare-dns.com
                        DNS Name=*.family.cloudflare-dns.com
                        IP Address=1.1.1.3
                        IP Address=1.0.0.3