Messages

i guess my experience has been the exact opposite with emulated mode. they improved emulated mode to not be as awful as it previously was, but i don't recall any statements saying they closed the gap. their own documentation states:

https://www.zenarmor.com/docs/guides/deployment-modes#b-with-emulated-netmap-driver

Code Select Expand

Be noted that emulated driver is not as performant as the native netmap driver.
i think if you have a sh*t drive like realtek, sure, but the netmap implementation in the intel driver is solid imo.

i would say i found similar over igb, igc, ixl, ax (the opnsense hardware amd 10g driver not the freebsd usb driver)

emulated mode i always got high latency spikes and bad bufferbloat tests. native mode i can get an "A" without any shaping, but i was constantly running into kernel messages about transmit full, etc. this is with a DEC3920 and 1 gbps WAN, so hardware is overkill.

i do

Code Select Expand

dev.ax.0.iflib.override_nrxds=4096
dev.ax.0.iflib.override_ntxds=4096
and my bufferbloat score moves to "A" and never see any of those transmit full messages in dmesg:

Code Select Expand

Notice kernel 440.906116 [4335] netmap_transmit ax0 full hwcur 902 hwtail 1016 qlen 909

None of those would show any data for the last month, for example ...

I don't want to capture all the data like netflow seems to do --- and why would I need to limit it to the WAN interface(s)?  Is there no way to get decent reports from all the gathered data?

Imagine I want to be able to just look at the dashboard and see that information, plain and simple, without any further ado except maybe adding a widget for it.  It should be just there.

vnstat shows that. and recent releases also contain a vnstat widget for the dashboard: https://forum.opnsense.org/index.php?topic=51328.0

i tried this and couldn't get the custom application to hit. i assume i am setting things up wrong.

i tried adjusting the custom app like 10x different ways. when i go back to the live sessions after updating custom app / policy, i still just see the entry allowed (green shield)

Custom apps in ZA are broken. And for some time now.

I already did open a ticket (21.10. 2025) for it, but it's in a backlog.

Regards,
S.

this makes sense, but i tried also making some other custom apps, and none of those worked either. like real simple ones, by IP address or just a port, so try and see if any custom app was working. seems like you said, custom apps just doesn't work and they apparently have no tests around that.

i tried this and couldn't get the custom application to hit. i assume i am setting things up wrong.

i tried adjusting the custom app like 10x different ways. when i go back to the live sessions after updating custom app / policy, i still just see the entry allowed (green shield)

yea i was able to get almost 2.5 gbps on n150 protectli, which cpubenchmark.net shows 9100 has better single and multi-threaded performance

i had the same random cloudflare rate limits: https://forum.opnsense.org/index.php?topic=52181.0

i also reported this a few months ago. nothing was done or even acknowledged

interface names to use the OPNsense names vs the FreeBSD interface names: https://github.com/opnsense/plugins/pull/5566

Code Select Expand

pkg -d update
Updating OPNsense repository catalogue...
DBG(1)[18410]> PkgRepo: verifying update for OPNsense
DBG(1)[18410]> Pkgrepo, begin update of '/var/db/pkg/repos/OPNsense/db'
DBG(1)[18410]> (fetch) Request to fetch https://us-east.opnsense-update.deciso.com/4409c341-bd35-4332-9b15-39ffd854a63b/FreeBSD:14:amd64/26.4/latest/meta.conf
DBG(1)[18410]> (fetch) Fetch: fetcher used: https
DBG(1)[18410]> (fetch) Request to fetch https://us-east.opnsense-update.deciso.com/4409c341-bd35-4332-9b15-39ffd854a63b/FreeBSD:14:amd64/26.4/latest/data.pkg
DBG(1)[18410]> (fetch) Fetch: fetcher used: https
OPNsense repository is up to date.
Updating SunnyValley repository catalogue...
DBG(1)[18410]> PkgRepo: verifying update for SunnyValley
DBG(1)[18410]> Pkgrepo, begin update of '/var/db/pkg/repos/SunnyValley/db'
DBG(1)[18410]> (fetch) Request to fetch https://updates.zenarmor.net/opnsense/FreeBSD:14:amd64/26.4/769b66af-4dce-43d9-81a0-cdac469aef9d/meta.conf
DBG(1)[18410]> (fetch) Fetch: fetcher used: https
pkg: An error occurred while fetching package: Unknown error
DBG(1)[18410]> (fetch) Request to fetch https://updates.zenarmor.net/opnsense/FreeBSD:14:amd64/26.4/769b66af-4dce-43d9-81a0-cdac469aef9d/meta.txz
DBG(1)[18410]> (fetch) Fetch: fetcher used: https
DBG(1)[18410]> (fetch) Request to fetch https://updates.zenarmor.net/opnsense/FreeBSD:14:amd64/26.4/769b66af-4dce-43d9-81a0-cdac469aef9d/data.pkg
DBG(1)[18410]> (fetch) Fetch: fetcher used: https
pkg: An error occurred while fetching package: Unknown error
DBG(1)[18410]> (fetch) Request to fetch https://updates.zenarmor.net/opnsense/FreeBSD:14:amd64/26.4/769b66af-4dce-43d9-81a0-cdac469aef9d/data.tzst
DBG(1)[18410]> (fetch) Fetch: fetcher used: https
Fetching data.tzst:   0%
DBG(1)[18410]> (fetch) Request to fetch https://updates.zenarmor.net/opnsense/FreeBSD:14:amd64/26.4/769b66af-4dce-43d9-81a0-cdac469aef9d/packagesite.pkg
DBG(1)[18410]> (fetch) Fetch: fetcher used: https
Fetching packagesite.pkg:   0%
SunnyValley repository is up to date.
All repositories are up to date.
if i try and curl that URL i get error code: 1015

Code Select Expand

curl --verbose --location https://updates.zenarmor.net/opnsense/FreeBSD:14:amd64/26.4/769b66af-4dce-43d9-81a0-cdac469aef9d/meta.conf
*   Trying [2606:4700:20::681a:dad]:443...
* Immediate connect fail for 2606:4700:20::681a:dad: No route to host
* connect to 2606:4700:20::681a:dad port 443 from :: port 0 failed: No error: 0
*   Trying [2606:4700:20::ac43:4ad1]:443...
* Immediate connect fail for 2606:4700:20::ac43:4ad1: No route to host
* connect to 2606:4700:20::ac43:4ad1 port 443 from :: port 0 failed: No error: 0
*   Trying [2606:4700:20::681a:cad]:443...
* Immediate connect fail for 2606:4700:20::681a:cad: No route to host
* connect to 2606:4700:20::681a:cad port 443 from :: port 0 failed: No error: 0
* Host updates.zenarmor.net:443 was resolved.
* IPv6: 2606:4700:20::681a:dad, 2606:4700:20::ac43:4ad1, 2606:4700:20::681a:cad
* IPv4: 104.26.13.173, 172.67.74.209, 104.26.12.173
*   Trying 104.26.13.173:443...
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* SSL Trust Anchors:
*   CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / X25519 / id-ecPublicKey
* ALPN: server accepted h2
* Server certificate:
*   subject: CN=zenarmor.net
*   start date: Jun  2 00:12:03 2026 GMT
*   expire date: Aug 31 01:11:53 2026 GMT
*   issuer: C=US; O=Google Trust Services; CN=WE1
*   Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA256
*   Certificate level 1: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA384
*   Certificate level 2: Public key type EC/secp384r1 (384/192 Bits/secBits), signed using ecdsa-with-SHA384
*   subjectAltName: "updates.zenarmor.net" matches cert's "*.zenarmor.net"
* OpenSSL verify result: 0
* SSL certificate verified via OpenSSL.
* Established connection to updates.zenarmor.net (104.26.13.173 port 443) from 71.33.134.33 port 37694
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://updates.zenarmor.net/opnsense/FreeBSD:14:amd64/26.4/769b66af-4dce-43d9-81a0-cdac469aef9d/meta.conf
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: updates.zenarmor.net]
* [HTTP/2] [1] [:path: /opnsense/FreeBSD:14:amd64/26.4/769b66af-4dce-43d9-81a0-cdac469aef9d/meta.conf]
* [HTTP/2] [1] [user-agent: curl/8.20.0]
* [HTTP/2] [1] [accept: */*]
> GET /opnsense/FreeBSD:14:amd64/26.4/769b66af-4dce-43d9-81a0-cdac469aef9d/meta.conf HTTP/2
> Host: updates.zenarmor.net
> User-Agent: curl/8.20.0
> Accept: */*
>
* Request completely sent off
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/2 429
< date: Tue, 07 Jul 2026 20:36:16 GMT
< content-type: text/plain; charset=UTF-8
< content-length: 17
< retry-after: 162
< cache-control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< expires: Thu, 01 Jan 1970 00:00:01 GMT
< referrer-policy: same-origin
< x-frame-options: SAMEORIGIN
< report-to: {"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=ZmUN3cyrFXqk%2B1qHTk7Ldc8NjauZHxDaMxXTuxWL1gtrmKFKgDn9h0LH0Cdn%2BgIGmb%2BExZJRJ8gDW0V%2FVR5OPO6bF7%2BPmNehWHiNDZMQybw9ot93lAsoaVlUjcu6LsuJ6ITjWZeS"}]}
< nel: {"report_to":"cf-nel","success_fraction":0.0,"max_age":604800}
< server: cloudflare
< cf-ray: a179a3f369b2e675-DEN
<
error code: 1015
* Connection #0 to host updates.zenarmor.net:443 left intact
HTTP/2 429  so it looks like i am hitting some sort of cloudflare rate limiting... from my quantum fiber home internet...

Thanks for this widget.  I like it!!

I would like to recommend an option to reverse the sort order.  So when the widget placed as a small window, the newest hours/days/months are visible at the top of the widget window.

yes, i could add sort order option. thanks for the feedback

- when you select which interfaces to monitor you have the choice within the different names of the interfaces and when it's displayed on the dashboard you have the name of the underlying device. i would prefer to have the interface name.

yea, that makes sense, i'll put up a PR to use the OPNsense interface name instead of the underlying FreeBSD device name.

- perhaps a layout with where you can select 2 or more interfaces to display could be a plus. But i guess it there will troubles to display that correctly. Or maybe spawn that to a widget per interface.

i'll probably skip this due to the complexity and trying to keep things simple

this finally landed in 26.1.11

Code Select Expand

***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 26.4.1 (amd64) at Sat Jun 20 16:47:18 MDT 2026
Strict TLS 1.3 and CRL checking is enabled.
Fetching subscription information, please wait... done
Fetching changelog information, please wait... done
Updating OPNsense repository catalogue...
Fetching meta.conf: . done
Fetching data.pkg: .......... done
Processing entries: .......... done
OPNsense repository update completed. 938 packages processed.
Updating SunnyValley repository catalogue...
pkg: An error occurred while fetching package: Unknown error
pkg: An error occurred while fetching package: Unknown error
Fetching data.tzst: .... done
Processing entries: .. done
SunnyValley repository update completed. 15 packages processed.
All repositories are up to date.
Checking for upgrades (2 candidates): .. done
Processing candidates (2 candidates): . done
Checking integrity... done (0 conflicting)
Your packages are up to date.
***DONE***
like i am not sure i ever remember a single zenarmor update without some sort of issue.

when RSS is enabled,

Code Select Expand

net.inet.rss.enabled = 1
the kernel will override dispatch and always make it hybrid (so you can leave that tunable out if you are enabling RSS), you can verify my claim with netstat -Q

Code Select Expand

net.isr.dispatch = hybrid
---

i tested a protectli VP2440 with N150 with i226 with Zenarmor and i was able to get 2.5 gbps

using the x710 10g interface without Zenarmor, i was able to get 9.46 gbps

so i would think that N100 could easily do 2 Gbps without IDS/IPS.

the tunables i used were (including installing the Intel Microcode Plugin):

Code Select Expand

dev.hwpstate_intel.0.epp = 10
dev.hwpstate_intel.1.epp = 10
dev.hwpstate_intel.2.epp = 10
dev.hwpstate_intel.3.epp = 10
dev.igc.0.fc = 0
dev.igc.1.fc = 0
hw.acpi.cpu.cx_lowest = C2
hw.ibrs_disable = 1
net.inet.tcp.soreceive_stream = 1
net.isr.bindthreads = 1
net.isr.maxthreads = -1
vm.pmap.pcid_enabled = 0
vm.pmap.pti = 0

note that i think the intel microcode plugin + vm.pmap.pcid_enabled = 0 is pretty much the gold standard with the N100/N150/N305 cpus, as they have a bug in the freebsd kernel otherwise...

i did update my i226 firmwares to v2.32 tho i dont think that mattered.

over the weekend, saw these in dmesg

Code Select Expand

[2148079] ax0: unable to obtain hardware mutexes
[2148079] ax1: unable to obtain hardware mutexes
[2148080] ax0: unable to obtain hardware mutexes
[2148080] ax1: unable to obtain hardware mutexes
[2148081] ax0: unable to obtain hardware mutexes
[2148081] ax1: unable to obtain hardware mutexes
[2148082] ax0: unable to obtain hardware mutexes
[2148082] ax1: unable to obtain hardware mutexes
[2148083] ax0: unable to obtain hardware mutexes
[2148083] ax1: unable to obtain hardware mutexes
but things still seem to be fine and moving along :)