Messages

I was never able to shake loose enough free time to troubleshoot this properly. I just uninstalled Zenarmor, re-installed, and set the data retention periods to 24 hours. Hopefully that resolves and I can actually rely on the web filtering stuff again.

I had the Zenarmor section of the OPNsense admin console open and I was served a Zenarmor ad (For, like, OSX features? I don't run OSX and I don't care.). Is this normal? I find it rather distasteful.

So I'm investigating if that is actually the correct way to set the gateway IP.

Going off of what is in the official docs, I did not use the DNS server's IP address as the gateway. I instead did the -1 trick off of the Tunnel Address used in the WG Instance setup. However, the docs say to enter the -1 address in the Instance config itself and in newer OPN versions that setting is actually in the Gateway setting under IP Address.

I find this part of the setup to be really confusing. Particularly in the docs where it says that IP address is essentially arbitrary. I cannot figure out why I cannot get Gateway Monitoring to work using the Endpoint Address.

Also, even if I Reset State, I often have to reboot the firewall in order to get the firewall to route traffic out any of the WG tunnels.

For reference, I seem to be able to LB across 3 different Mullvad tunnels now. That may sound excessive however I'm playing region + ASN routing games with various sites and this is the most simple solution I have been able to formulate.

As soon as i remove the second gateway, disable the second wg interface and disable the second wg instance, everything works perfectly fine again.

I took some screenshots of my config: Screenshots

Do you have any idea for a solution to my problem?

Thank you very much for your help :)

That is hands-down the best and most up-to-date tutorial I have found on the topic. However, like you, mine breaks when I try to add a 2nd tunnel -- eventual goal is LB over multiples.

The biggest difference I see between the linked tutorial and some of the other guides I have found is that Andrew is using the IP address of what is listed as the DNS server in the WG config. Multiple Gateways with the same gateway IP address is not going to work. OPN won't even allow that as a config.

So I'm investigating if that is actually the correct way to set the gateway IP.

Unfortunately I'm back to nightly shutdowns.

I'm looking at what I should install on OPNSense and in my network for log and performance monitoring. Stand by...

Hi,

This is a temporary partition used to store trace logs for device updates and other activities. To identify the largest folder, navigate to the `/usr/local/zenarmor/run/tracefs/{interface_name}` directory and use the command `du -sh *`. Once identified, you can remove the folder if necessary. Zenarmor retains this data for troubleshooting purposes.

Alternatively, you can increase the partition size by going to Settings > Reporting & Data > Database and adjusting the Tracefs Partition Size. It is recommended to set it to 150 MB to adequately store trace data.

Confirming... I bumped up default values for Memory Disk Size and Tracefs Partition Size significantly and ZenArmor was still running as of this morning. So that's a good sign.

I didn't find much in the docs about doing this for small installations. Some somewhat related material for enterprise deployments.

Thanks for your help, @sy.

Sorry, for some reason I'm not getting notification emails from the forum about watched threads. Just happened to login for another reason and see this...

Re RAM disk, sorry I'm not a BSD person and so I'm not totally sure how to check about this and I'm not finding docs on it either. I don't **think** I have RAM disks except perhaps for this:

    tmpfs                        102400      252   102148     0%    /usr/local/zenarmor/run/tracefs

In the web console under Settings/System/Misc none of the RAM disk options are enabled. I did just enable swap there however.

This is as been ongoing issue for several months now and I'm not sure what to make of it. My Zenarmor Home shuts down every night due to what appears to be a false-positive warning about disk usage on the OPNsense box. In my monitoring I see no indication that any of the volumes on the firewall are going above 50%.

Other than this being a bit of a hassle -- restart every morning -- it also means that my family web filtering is not effective unless I stay on top of things daily.

Anyone else seeing this?

Previous update was in early February but I just updated as I submitted this post. If I don't say otherwise, assume the problem persists even with today's updates applied.

# OPNsense details:

Type opnsense
Version 25.1.3
Architecture amd64
Commit 6aa1d97b1
Mirror https://pkg.opnsense.org/FreeBSD:14:amd64/25.1
Repositories OPNsense (Priority: 11), SunnyValley (Priority: 7), mimugmail (Priority: 5)
Updated on Fri Mar 21 08:39:02 MDT 2025
Checked on N/A

# Zenarmor details

os-sensei (installed) 1.18.6 207MiB 3 SunnyValley Enterprise Security Extensions for OPNsense (ZENARMOR)
os-sensei-updater (installed) 1.17 4.03KiB 3 SunnyValley OPNsense ZENARMOR Plugin Updater
os-sunnyvalley (installed) 1.4_3 2.44KiB 3 OPNsense Vendor Repository for Zenarmor (a.k.a Sensei, Next Generation Firewall Extensions)
os-sensei-agent 1.18.6 115MiB 3 SunnyValley ZENARMOR (Sensei) Connectivity Agent for Cloud Central Management


thx

I see a few threads here which look pretty useful for helping validate some (too?) clever things I'm trying to do with Wireguard site-to-site VPNs and I will dive into those shortly. However, what I'm not finding is a lot of collateral on how to do low-level troubleshooting at the CLI. That is going to be the highest fidelity approach in some situations because the management console presents only a subset of the available info for debugging.

Does anyone have any suggestions on a resource? Primarily just the usual FreeBSD docs?

thx

+1 from me.

Until I get a more infra-wide dashboard in place I have same report up on a spare monitor and would love to ignore spikes from re-scaling the Y axis.

Rich Stevens uses tcpdump to show how the protocols work in action. Absolutely brilliant.

I'm pretty comfortable with IP/UDP/TCP itself. Is there a similarly great book you're recommend on advanced networking topics with FreeBSD?

the best tool to troubleshoot wireguard would be the good old tcpdump.

tcpdump -i wg0

Well, that is a tool I've definitely spent time getting to know. Thanks for the advice.

I've been trying to set some of the Outbound NAT and other firewall rules for logging but I'm not seeing much in the console. Perhaps I'm not looking in the correct place or perhaps there's a way to get at that info from the command line?

Another approach I'm planning to take is set up an internal log aggregator. I've been meaning to this for some time anyway.

So what I've noticed is that even in DEBUG the Wireguard Log can sometimes report that the tunnel is up but it doesn't seem to be functioning for me and I don't see routes for it at the CLI either. Entirely possible that I am able to auth to the remote VPN endpoint and then something about routing or forwarding is still goofed up.

On that theory, I've been opening 2 browsers when configuring new tunnels and comparing them step-by-step to known-good configs for working tunnels. Hasn't produced working tunnels in most cases.

Howdy, folks.

So I'm having a LOT of problems getting multiple WG VPN tunnels set up to Mullvad and then doing selective routing over them. Some of it, I would say, is due to lack of observability as to the state of the tunnels. In short, I miss the old Status tab which would at least let me know I had configured the tunnel correctly before I start playing with selectively routing of traffic over them.

What advice do folks here have as to troubleshooting WG VPN (and maybe related pf configurations) from the CLI?

For reference, I have tons of experiencing doing pretty complicated gateway setup on Linux going all the way back to ipchains days. Not so much with *BSD and pf. So, I can learn is what I'm saying. :D