Messages

Also interesting, when I do "pkg remove cloudflared", then remove the "/usr/ports/net/cloudflared" folder, "pkg install cloudflared" does not recreate this folder anymore, weirdly.

That's expected. The files in /usr/ports/* are files used to create pkg files from source, not the other way around. pkg does not manage any of the files in /usr/ports/. As long as you didn't delete /usr/ports/.git then you can use git to restore the folder you deleted:

Code Select Expand

cd /usr/ports
git checkout /usr/ports/net/cloudflared

Since your cloudflared Makefilewas out of date, you also need to update your ports tree (this is why your Makefile was for 2023.10.0_2, why it wanted to use go1.20 to build, and why make couldn't find 1.24 in your ports tree):

Code Select Expand

cd /usr/ports
git pull

Or you can delete and re-initialize your ports tree with:

Code Select Expand

cd ~
opnsense-code -r ports
opnsense-code ports

Deleting your ports tree does not deinstall any packages you built from ports. Once a package is built, that package is installed/deinstalled via pkg (that is, make deinstall is a helper that calls pkg remove with the correct arguments)

Thanks, I'll try that. Traefik seemed nice since I'm already using it on other systems. I guess I searched for "traefik on opnsense" and I should have just searched for reverse proxy options...

I see that HAProxy is also an option and uses the os-acme-client, which I already use.

[Current Setup, without Traefik plug]

Current Setup, without Traefik plug
My current setup is pretty standard. I have 80 and 443 forwarded to an internal host. On that host I run traefik and some docker containers.

• 80 -> internalhost1:80
• 443 -> internalhost1:443

But I'd like to a second internal host, also running some services. And I'd like to do this without running either on non-standard ports.

Desired setup, with os-traefik-maxit
I've installed traefik from mimugmail's repo. I was planning something like this:

• http://(.*) -> https://(.*)
• https://newservice.example.com -> https://internalhost2/
• https://oldservice1.example.com -> https://internalhost1/
• https://oldservice2.example.com -> https://internalhost1/
• https://oldservice3.example.com -> https://internalhost1/

I wonder if anyone has set up something similar. I'm running into some roadblocks right out of the gate and I wonder if anyone has solved them or has suggests.

1. Traefik entry points look like

Code Select Expand

address = ":443", but that will conflict with the local opnsense webui. IS there some way to dynamically use the WAN ip address as the bind address in a configuration file like this? I don't think traefik allows selecting a bind adapter. I won't need traefik on OPNSense listening on any local IPs.

I guess one solution might be that I could have traefik listen on non-standard ports like 127.0.0.1:8443 and then use a port forwarding rule in the OPNSense firewall config.

2. One reason I like Traefik is because of how easy it is to manage TLS certificates. I use DNS challenge with Digital Ocean, but that requires an environment variable

Code Select Expand

DO_AUTH_TOKEN is set. I don't think traefik lets me put this in the traefik.toml file. Is there a way to set environment global variables on OPNsense so that a service like traefik will inherit that in its launch shell?

A dot is merely a second, not a size unit. You can do the math here how fast the download is.

Ok, good info. I'm not sure what happened. Space was slowly filling up, it had 300MB free when I rebooted, but I watched that drop from 325MB. But I do use bectl for boot environment snapshots, and I see that both /tmp, /var/tmp, and /var/log are included, so I think the shrinking space was simply because "a zfs snapshot existed" and "temp and log files are being written"

I'll try the upgrade again later.

Mine's currently doing it. It ran all night. Got over 10k dots (half that count is spaces)...

Code Select Expand

cat /tmp/pkg_upgrade.progress | grep 'Fetching base' | cut -f2 -d\: | wc
       1       1   20858

IDK what file name is used during the download process, but I can't find base-24.1.2-amd64.txz anywhere on the system. Poked around a bit with `du` and don't see anything absolutely massive. /var/log (mostly flowd.log*) and /usr are both pretty big, but I think that's generally expected.

Unfortunately on reboot it didn't come back online, so now I'll have to drag a monitor over there and figure out where in the boot process it's hanging.

I wanted Pi-hole but didn't want to buy a raspberry just for that, so I used the pi-hole provided Virtual Machine amd-64 image and ran it as a VM on an ESXi host. All fine.

ESXi is no longer free 😭

Pihole is a classic example of Linux centric development. No way (currently) it's going to run on FreeBSD.

Not my or OPNsense's fault.

Patrick, this is an incorrect statement. I have Pihole running on TrueNAS Core Virtual Machine using bhyve.

That's still not running PiHole on FreeBSD. That's still installing PiHole on Linux. But yeah, as long as your opnsense box has enough resources, running Linux on bhyve is doable. Still can't use the PiHole installer scripts on FreeBSD directly and modifying PiHole for BSD would be a huge effort. Running PiHole in a jail would be much more attractive than using bhyve.

What about the Linuxulator? I haven't used that since the Linux 2.6.26 days; I had thought it was abandoned, but the wiki says Linux 4.4 compatibility. Can that be used for Docker? Or is it more like WINE and too much of the Linux kernel API is still unimplemented?

Now the service doesn't crash anymore but it may not be the best solution if you don't want to have the unbound listening on for example the WAN interface.

Set a Block rule for UDP 53 in your firewall for any interfaces you don't unbound listening to.

Derp, ok. I'm dumb.

Today I noticed in FreeBSD's handbook that bridges are affected by the packet filter. Specifically, pf treats packets flowing through the bridge as entering the physical port, not the bridge port. So for each interface, I added a Firewall Rule to OPNsense for "for this interface with source LAN NET and destination LAN NET, pass". And it worked!

So then I looked back at the guide and Step 6 is setup tunables so pf filters based on the bridge rather than the member ports.

So if anyone else has this problem... either add explicit firewall rules, or follow all the steps in the guide >_<

I followed the guide for Bridged LAN and it appeared to work, but it turns out it's only half working.

I have igb0 set as WAN, bridge0 set as LAN. Bridge0 contains igb1, igb2, and igb3 (though only igb2 and 3 are plugged in).

All connected links can talk to the router; DHCP works, hosts have internet access, etc. But hosts on igb2 can't talk to hosts on igb3.

Any suggestions on how to proceed?

I used beadm to take a snapshot, then I upgraded to 20.7. I'm pretty sure it was successful, but I guess I didn't pay too much attention.

Then I messed around with some bridge settings and locked myself out from the network. I used the local VGA console and used beadm to restore to prior to the upgrade. Things were working, so I deleted the broken 20.7 boot environment; figured I could just upgrade again.

Well now when I go to System -> Firmware -> Upgrade it claims no upgrades available. It doesn't show the 20.7 unlock button.

If I do Audit -> Health it shows a ton of checksum mismatches. I don't think that was the case before I attempted the upgrade, etc.


Is it possible that the something from 20.7 survived switching beadm to switch boot environments back to 20.1?

see: https://controlc.com/9c26862d

Code Select Expand

$ uname -a
FreeBSD router.local 11.2-RELEASE-p18-HBSD FreeBSD 11.2-RELEASE-p18-HBSD  f08b5f14327(stable/20.1)  amd64

No issues turned up in zpool scrub.

[Disable TLS 1.0 and TLS 1.1]

I don't normally have my WebUI accessible via the internet, but I allowed it through the firewall temporarily so I could test in on SSLLabs. It was surprisingly easy to get an A+ rating.

I'm using the ACME plugin to get a cert via Lets Encrypt. Everything else setup via System -> Settings -> Administration.

Disable TLS 1.0 and TLS 1.1

SSL Labs caps your rating at a B if you allow TLS 1.0 or 1.1. It looks like the only way to do this is by limiting the available ciphers. So limit to the following ciphers I've changed my ciphers from Default to the following:

Code Select Expand

# TLS 1.3
TLS_AES_256_GCM_SHA384
TLS_CHACHA20_POLY1305_SHA256
TLS_AES_128_GCM_SHA256

# TLS 1.2
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
TLS_DHE_RSA_WITH_AES_256_GCM_SHA384

This will prevent some archaic browsers from accessing the WebUI. Some examples:

• IE 11 works, but not IE 11 on some un-updated versions of Windows Phone OS
• Safari 9 on all platforms work, but Safari 8 and older will not work
• Android 4.4.2+ should work, but anything older will not

HSTS
Check the box for HTTP Strict Transport Security or you'll be limited to an A rating.

That's it!
Congrats! Your router now has a higher SSL rating than Amazon.com =D

Jose, that looks great. I agree with you that the BSD Install method looks best. It's more consistent for users (that's the method adopted by other projects based on FreeBSD such as FreeNAS and pfSense) and I expect it should be easier to maintain (as we can get changes from FreeBSD as they adjust their installer script).

I would be happy enough to contribute the ZFS installer to the OPNsense devs so they can update/modify/adapt as needed.  :)

There are currently not enough developers to implement all the desired features in OPNsense, but the source code is on github and all are welcome to contribute pull requests. It's spread across multiple repositories, but here's the repo for the installer. Would you be willing to create a pull request for this?

Having the same problem. I know with pfSense there's no solution. Really hoping opnsense has a work around as it seems like a more flexible platform. This is basically a requirement to use IPv6 on a home network if you want to host any services behind the firewall.