Messages

There seems to be some cloudflare wrong-doings. Either i'm told I've been limited or

Code Select Expand

Error 1000 Ray ID: a20430019928d268 • 2026-07-24 16:09:08 UTC
DNS points to prohibited IP

Code Select Expand

Error 1015 Ray ID: a204460fdb0cd268 • 2026-07-24 16:24:11 UTC
You are being rate limited


ok - I swear to the sacred master boot record, I will check forum right before I fire up any upgrade other than minor version. Big thanks to the ZA team (https://forum.opnsense.org/index.php?topic=52385.0)

yes, indeed, any message (btw posted some 20 minutes ago) basically saying be aware that it is not working yet, while pushing the new 26.7 to the users seems very valid...  not to offend anybody >:-)
I too vote for the warning saying at least: "hey noob users, this is a major upgrade, pay attention to your repos". And not just spitting some "unable to update repo" messages to the console.  What about some preflight analysis? OK i get it - recently I was working out some upgrades from some ancient opnsense to the latest, everything went smoothly to the point when pkg segfaulted. I did not surender, I had a chance to take my backup and implant it to the new instalation.
But this in fact rolling out a product without proper testing... just saying

no, not realy. Not completely purging the idea of reinstall, but as it is not my main opnsense box i'll wait till the release of the new ZA. To my defense I should say i'm aware of importance of having backups and snapshots. But my current VM setup does not do it automaticaly and I have not expected this to be a mayor upgrade, breaking the update proccess instantly - even when downloading and updating it's own packages... As I have not been warned anywhere :] and suddenly - during new pkgs install - Unable to update repository SunnyValley. I thought this is not a beta anymore...

well - what should I do to reinstall os-sensei plugin to let it work, waiting for ZA to bring up the new version?

Went from 26.1.11_6 to 26.7 - now at 26.7 without working sensei (my mistake to remove sensei itself not only remove/add repo)

Code Select Expand

pkg update -f
Updating OPNsense repository catalogue...
Fetching meta.conf: 100%    179 B   0.2kB/s    00:01
Fetching data.pkg: 100%  327 KiB 334.4kB/s    00:01
Processing entries: 100%
OPNsense repository update completed. 928 packages processed.
Updating SunnyValley repository catalogue...
Unable to update repository SunnyValley
Error updating repositories!


Code Select Expand

pkg -d update
Updating OPNsense repository catalogue...
DBG(1)[21388]> PkgRepo: verifying update for OPNsense
DBG(1)[21388]> Pkgrepo, begin update of '/var/db/pkg/repos/OPNsense/db'
DBG(1)[21388]> (fetch) Request to fetch https://pkg.opnsense.org/FreeBSD:15:amd64/26.7/latest/meta.conf
DBG(1)[21388]> (fetch) Fetch: fetcher used: https
DBG(1)[21388]> (fetch) Request to fetch https://pkg.opnsense.org/FreeBSD:15:amd64/26.7/latest/data.pkg
DBG(1)[21388]> (fetch) Fetch: fetcher used: https
OPNsense repository is up to date.
Updating SunnyValley repository catalogue...
DBG(1)[21388]> PkgRepo: verifying update for SunnyValley
DBG(1)[21388]> PkgRepo: need forced update of SunnyValley
DBG(1)[21388]> Pkgrepo, begin update of '/var/db/pkg/repos/SunnyValley/db'
DBG(1)[21388]> (fetch) Request to fetch https://updates.zenarmor.net/opnsense/FreeBSD:15:amd64/26.7/latest/meta.conf
DBG(1)[21388]> (fetch) Fetch: fetcher used: https
DBG(1)[21388]> (fetch) Request to fetch https://updates.zenarmor.net/opnsense/FreeBSD:15:amd64/26.7/latest/meta.txz
DBG(1)[21388]> (fetch) Fetch: fetcher used: https
DBG(1)[21388]> (fetch) Request to fetch https://updates.zenarmor.net/opnsense/FreeBSD:15:amd64/26.7/latest/data.pkg
DBG(1)[21388]> (fetch) Fetch: fetcher used: https
DBG(1)[21388]> (fetch) Request to fetch https://updates.zenarmor.net/opnsense/FreeBSD:15:amd64/26.7/latest/data.tzst
DBG(1)[21388]> (fetch) Fetch: fetcher used: https
DBG(1)[21388]> (fetch) Request to fetch https://updates.zenarmor.net/opnsense/FreeBSD:15:amd64/26.7/latest/packagesite.pkg
DBG(1)[21388]> (fetch) Fetch: fetcher used: https
DBG(1)[21388]> (fetch) Request to fetch https://updates.zenarmor.net/opnsense/FreeBSD:15:amd64/26.7/latest/packagesite.tzst
DBG(1)[21388]> (fetch) Fetch: fetcher used: https
Unable to update repository SunnyValley
Error updating repositories!


hmm as the last ( ??? ) resort I've fired os-sensei package reinstall and now it show session details. Not sure what went wrong

I was pushed by Zenarmor gui to upgrade to 1.11. After successful upgrade when I select anything from pie chart graph in the Blocks tab and select Session details it shows no sessions.
In fact I've already posted same question to sunnyval bug report:

Connections,Threats,partly Web - select Pie chart piece - session detail - show list of sessions
Blocks - select Pie chart piece - session detail - shows empty list of sessions (Loaded records: 0 / 0)
DNS - select bar graph - no sessions in the list either
TLS - shows (green) allowed and (red) blocked sessions

I've done configuration-reset reporting and erase reporting data and deleted that log file and everything in the /usr/local/sensei/log/archive as well (I was personally stunned by the full hdd). After opnsense restart I've got enough free space to run elasticsearch and zenarmor again - but it does not show the above mentioned sessions

What I've done so far:
-several times tryed to restart reporting
-reseted reports and let it delete all the data
-removed elasticsearch pkg and let it reinstall

I can see Alerts - Blocked Conversations Heatmap and Alerts - Blocked Local Hosts and Reasons noprob but I can see no session details

I'm still using OPNsense 21.7.8-amd64 (performance issues with 22.x discused elsewhere here)

I believe I've found something. I turned on the "Log packets matched from the default pass rules put in the ruleset"  on the both installations - so I hope I have the same settings.

on the 21.7 working fast I have this in live fw log

Code Select Expand


vmx_2 Mar 2 11:38:39 10.0.0.20:52258 81.0.212.201:443 tcp PROXY redirect
vmx_2 Mar 2 11:38:39 10.0.0.20:52257 88.86.101.2:443 tcp PROXY redirect
vmx_2 Mar 2 11:38:39 10.0.0.20:52256 81.0.212.201:443 tcp PROXY redirect
vmx_2 Mar 2 11:38:39 10.0.0.20:52255 52.210.251.145:443 tcp PROXY redirect
vmx_2 Mar 2 11:38:39 10.0.0.20:52254 52.210.251.145:443 tcp PROXY redirect
vmx_2 Mar 2 11:38:39 10.0.0.20:52253 77.78.109.86:443 tcp PROXY redirect
vmx_2 Mar 2 11:38:39 10.0.0.20:52252 77.78.109.86:443 tcp PROXY redirect
vmx_2 Mar 2 11:38:35 10.0.0.20:52251 142.251.36.67:443 tcp PROXY redirect
vmx_2 Mar 2 11:38:26 10.0.0.20:52250 52.210.251.145:443 tcp PROXY redirect


but on the OPNsense 22.1.2

Code Select Expand


vmx_2 2022-03-02T11:31:51 10.0.0.20:52049 81.0.212.201:443 tcp let out anything from firewall host itself
vmx_2 2022-03-02T11:31:51 10.0.0.20:52049 81.0.212.201:443 tcp PROXY redirect
vmx_2 2022-03-02T11:31:51 10.0.0.20:52048 88.86.101.2:443 tcp let out anything from firewall host itself
vmx_2 2022-03-02T11:31:51 10.0.0.20:52048 88.86.101.2:443 tcp PROXY redirect
vmx_2 2022-03-02T11:31:51 10.0.0.20:52047 18.203.13.148:443 tcp let out anything from firewall host itself
vmx_2 2022-03-02T11:31:51 10.0.0.20:52047 18.203.13.148:443 tcp PROXY redirect
vmx_2 2022-03-02T11:31:51 10.0.0.20:52046 18.203.13.148:443 tcp let out anything from firewall host itself
vmx_2 2022-03-02T11:31:51 10.0.0.20:52046 18.203.13.148:443 tcp PROXY redirect
vmx_2 2022-03-02T11:31:51 10.0.0.20:52045 77.78.109.86:443 tcp let out anything from firewall host itself
vmx_2 2022-03-02T11:31:51 10.0.0.20:52045 77.78.109.86:443 tcp PROXY redirect
vmx_2 2022-03-02T11:31:51 10.0.0.20:52044 77.78.109.86:443 tcp let out anything from firewall host itself
vmx_2 2022-03-02T11:31:51 10.0.0.20:52044 77.78.109.86:443 tcp PROXY redirect


PBR rule has quick ticked on and sloppy states set.

I think firewall sends data to the proxy as well to (default) routing - for example
one logged packet to proxy: seq   875304436
next line: seq   875304436 (let out anything from firewall host itself)

I don't see any let out anything from firewall host itself rules in the live log view on the 21.7.x installation.

Am I missing something on the 22.x? As I wrote - I have to installations as VMs - I have recently upgraded both, but after this difficulties, I revert my main rig to 21.7 and used the other one as a guinea pig for 22.1

I've wiresharked both installations from my test client and I see many TCP out-of-orders and TCP duplicates (mainly ACK, but also some PSHs). I tried to analyze it by L2 headers - to see what host sends the data - it seems req goes to set default GW mac and data returns from the proxy. Without the PBR rules, req goes to default gw mac and returns the same way. But with the 22.1 as the dGW I see many TCP retransmits and out-of-order ACKs etc.

it's unchecked in the both versions - needles to say proxy is not running inside opnsense - it's solitary instalation in the vm. opnsense has only one WAN

If I disable PBR rule data goes to opnsense dGW and it's fast up and down. If I set proxy in my browser - data goes fast as well. But when I use the PBR rule - data should (should - but see the test below) go to the proxy and then out via it's dGW - download speed is ok, but upload is only 10Mbps.
And all it started with 22.1 upgrade. I let my other opnsense instance upgraded to 22.1 and reinstalled my main rig to 21.7.8 and my speeds are ok.

In fact just now I did this test - client with dGW as (I'm using iptraf-ng on the proxy):
opnsense 21.7.8 - I can confirm hundreds of Mbps (ie. 914/920Mbps) up and down going through the proxy iface (10k pps both sides)

opnsense 22.1_1 - I can read hundreds Mbps out from proxy (10k pps both sides) and then 40Mbps multistream upload (rychlost.cz) on the client, but only <500kbps (about 50pps) on the proxy in (ie. output of client).

So with opnsense 22.1 - data seems to go up elsewhere - but why and how?

no - proxy has it's own dGW - it's not sending data back to opnsense to route it out to the internet. What's more weird is that with 21.7.8 it works fast - I had no reason to question my PBR rule so far

1) data without policy based route  -  full speed (outgoing from router)
2) data send directly to proxy via browser/system setting - full speed
3) data targeted to proxy GW - download full, upload 10Mbps

OPNsense 22.1.1_1 - not solved

I have recently switched from 21.7.8 to 22.1 just to found out upload via our PBRed squid proxy is poor (just 10Mbps per single server). We use proxy for http/https inspection and slight http cacheing, terminating ads and malware connections - so realy no real MITM for users.

Not changed my configuration - so my rule only changes gateway on the matching packets. This worked fine for some time - as my download and upload speed was about right for ISP speeds (1Gbps).  But when upgraded to 22.1 it went down and some web services are malfunctioning (ie. uploading PDF forms to the goverment servers etc. timeouts etc.)

My opnsense runs on ESXi 6.7U2 host with intel I210 and I350 with igbn driver. opnsense has vnics as vmxnet3. LANs are on a single vswitch with esxi tagged vlans, wan is on a separate vswitch. All MTUs set to 1500. All offloading in opnsense disabled. Zenarmor for all the LAN ifaces and surricata for WAN. I have one interface with VLANs in opnsense - parent interface assigned and enabled. Only tunables were queue settings for vmx, but it have been dropped recently. I know (from this phorum as well) it has been replaced by dev.vmx.0.iflib.... keys anyway.

I have tested all connections with iperf3 - client to fw, client to proxy, fw to proxy with similar results ~900Mbps TCP. While testing UDP to proxy - everything went ok, but testing against opnsense printouts

Code Select Expand


iperf3: OUT OF ORDER - incoming packet = 54887 and received packet = 0 AND SP = 277090
iperf3: OUT OF ORDER - incoming packet = 221983 and received packet = 0 AND SP = 277226
iperf3: OUT OF ORDER - incoming packet = 178360 and received packet = 0 AND SP = 277228
iperf3: OUT OF ORDER - incoming packet = 229939 and received packet = 0 AND SP = 281851
iperf3: OUT OF ORDER - incoming packet = 266111 and received packet = 0 AND SP = 282071
iperf3: OUT OF ORDER - incoming packet = 222533 and received packet = 0 AND SP = 282784
iperf3: OUT OF ORDER - incoming packet = 208849 and received packet = 0 AND SP = 282826
iperf3: OUT OF ORDER - incoming packet = 258064 and received packet = 0 AND SP = 283012
iperf3: OUT OF ORDER - incoming packet = 258554 and received packet = 0 AND SP = 283081


When I set up the client to directly use the proxy - everything works fine (930/930Mbps in inet tests). But if I put it via opnsense (ie. transparent mode) I get 930Mbps/8Mbps (8Mbps single server, about 30Mbps multiple servers).

So I went haywire and looked everywhere - but as I have test installation avaible on the another server (actually upgraded to 22.1 as well) I was able to check the behaviour is the same from the other one.

Later today I've grabbed 21.7 ISO and reinstalled it over my test instalation cloned VM, imported config and tested it. Now my incoming and outgoing rates on proxy can easily reach 900Mbps.

So I think I sorted out my infrastructure, cables, VLANs, proxy instalation and it's settings (deb10) and firewall rules. But I understand opnsense underwent mayor OS version change - but the same configuration is not working well anymore.

What are the settings I should look at? Do I miss something? Not realy used to FBSD systems - but I did not find anything useful -  actualy not here...

I'm going to let my testing installation VM clone run 21.7 (I must remember to take snapshots/backups before opnsense upgrade - this is not for the first time I got screwed a little bit) and run my main rig with 22.1 - in case somebody helps me rule it out...

THX