Messages

Hypothetically, let's say we find a security vulnerability in OPNSense that we want to (privately!) bring to the attention of developers to get it resolved quickly and quietly. Who should we send this data to, and how should we send it?

Thanks!
-Anomaly0617

Hi all,

We maintain a number of multi-site locations where we've used IPSec and fully-meshed the networks (ie: every location can talk to every other location). The obvious issue with this is the upkeep. Doing the math, if I have 11 locations, that means that every firewall has 10 IPSec tunnels. (Locations * (Locations -1)). So, in the instance of 11 locations, I'm maintaining 110 individual tunnels.

And this is where cloud-meshed VPN solutions (SD-WAN) enters the show. Instead of having to maintain 110 tunnels, I could maintain 11, and the cloud-meshed VPN solution would handle the routing. Or, worst case, I can manage the routes at the centralized console. Point is, the routing becomes easier.

For some of my locations, this is a non-starter. Those locations deal with protected information and an SD-WAN solution leaves too high a possible risk for someone to silently add their own node onto the network and then have time and access to all of the information they can vacuum up without our notice. Yes, I know that these solutions often offer multi-factor authentication at the cloud level. But even multi-factor is being hacked these days.

And this brought us around to ZeroTier. One of my engineers was complaining about what a pain in the rear maintaining all the tunnels was, and how SD-WAN would be so much better a solution. So I decided to give it a try with ZeroTier.

I picked a location that is more or less a "test site" - no issues if that site gets hacked, there's nothing there for them to find. And for the other site (and eventually, sites), I picked a few of our staff's home networks that are in a similar situation - if they get hacked... well, that sucks, but they have firewalls and backups.

I followed this guide and tried to keep the configuration as simple as possible. Get it working reliably first, and then start building rules and filters as necessary. But I didn't get that far.

My experience was that after

• updating OPNsense to the latest version
• getting the zerotier plugin installed
• connecting it to my zerotier portal through VPNs >> ZeroTier and using the API I was given from the ZeroTier portal
• adding the site and approving it in the portal
• creating the network interface and assigning it the appropriate IP address from ZeroTier (note: /24, not /32!)

I would see an uptime of about 45 seconds to 2 minutes, after which my network monitoring software would go from "all green" to "all red" pinging the remote hosts.

I tried adding managed route(s) (example: network 192.168.92.0/24 routes through 10.147.17.197).
I tried rebooting firewalls.
My experience did not change.

I read through the ZeroTier manual. No luck.

I not only disabled the IPSec tunnels that were in place before, I deleted them out of a few firewalls and rebooted. No change.

So, I'm now curious if I did something very obviously wrong, or if this experience is relatable to others? I'm open to the idea of an SD-WAN solution if it's stable, but I'm not going to sacrifice stability for convenience.

Opinions/Comments welcome!

Hi all,

I was referencing this documentation the other day to get up country-based IP filtering.

The documentation states "In OPNsense, goto Firewall:Aliases and select the GeoIP settings tab. Enter the URL you have created into the URL box and click Apply, and that's it."

When I go to Firewall:Aliases on an OpnSense v20.1 server, I see this tab. However, on a v19.7.6 firewall, there is no GeoIP tab. On this one, when you create an alias you can choose GeoIP as an alias type, and then from there select the countries you want to block. Then, in theory, you make a block rule in Firewall:Rules:Floating, selecting all of your outside interfaces, and then block anything with a source of the GeoIP Alias.

I see that in v20 I can still do this with the aliases, as type GeoIP.

So, I'm just looking for clarification. Is the Alias method OK to use? Is the GeoIP method the one we're supposed to use? I don't want to assume that the alias method is working to block inbound traffic from undesirable countries and then find out that it doesn't actually work without MaxMind and the GeoIP tab.

Thanks, in advance!

Just wanted to bump this thread in case it was overlooked. I've checked on multiple firewalls (20 or so) and the code is in every one of them that has been upgraded to the 20.x version.

Are you adding that Cron job the official way using actions.d?.

My guess is, probably not. I'm adding it via the console using crontab. There appears to be no GUI-based way to add a cron job that isn't from the pre-populated list. With that said, if there's a better way (actions.d) let me know how to do it and I'll change up my instructions to match once I try it. It would be really nice if I could get these to persist across upgrades!

Thanks!

Just checked and no, that doesn't resolve it for me. From what I can tell, at least on my installs, the fix seems to be in rebooting the firewall if it cannot ping a server on each of the networks and a publicly hosted website like google.com. The problem is, every time you upgrade the firewall, the crontab rule that kicks this script off gets removed, so I have to re-add it manually.

Thanks for the idea, though! It made me go through my Unbound configuration with a fine tooth comb!

Hi all,

In the latest release (20.1), there's a hard-coded style element on line 1132 of the output:

Code Select Expand


1129: <style>
1130:  tr.phase1_tr > td {
1131:      font-weight: bolder;
1132:      background-color: #FBFBFB;
1133:  }


Because this is defined after all the css files are rendered, this is breaking themes. For instance, on dark themes (rebellion, tucan) the background is near white and so is the font.

Can this be moved into main.scss or another css file so it can be changed by theme designers?

Thanks!
Anomaly0617

Not so far. I'm using my check_internet.sh script to get around the problem at the moment, but one thing I've noticed is that the cron job does not persist from upgrade to upgrade, so I have to manually put it back in.

It's unfortunate that you are seeing the same problem, but also reassuring that I'm not the only one. :-/

[Unchecked]

From Google Translate, since I'm terrible at German :) :

The rewrite rule belongs off and leave the location on / match and match type empty, since the VHosts are mapped with the server names via different HTTP servers.

Ok, with this advice I got one server proxy working. I'll attempt more later today. Here's what I did to make it work...

Services: NGINX: Configuration: General
-- Enable nginx: Checked

Services: NGINX: Configuration: Upstream Server (1 Entry)
-- Description: Server1
-- Server: 192.168.xxx.zzz
-- Port: 8091 (later moved to 80, after I got it working)
-- Server Priority: 1

Services: NGINX: Configuration: Upstream (1 Entry)
-- Description: com_siteb
-- Server Entries: Server1
-- Load Balancing Algo: Weighted Round Robin
-- Enable TLS: Unchecked
-- TLS: Servername override: [Blank]
-- TLS: Supported Versions: [Nothing Selected]
-- TLS: Session Reuse: [Checked] (I think by default)
-- TLS: Trusted Certificate: [Nothing Selected]

Services: NGINX: Configuration: HTTP(S): Location (1 Entry)
-- Description: com_siteb
-- URL Pattern: /
-- Match Type: None
-- URL Rewriting: [Nothing Selected]
-- Enable Security Rules: [Not Checked]
-- Learning Mode: [Not Checked]
-- Block XSS Score: [Blank]
-- Block SQL Injection Score: [Blank]
-- Custom Security Policy: [Nothing Selected]
-- Upstream Servers: com_siteb
-- Path Prefix: [Blank]
-- Cache: Directory: [none]
-- File System Room: [Blank]
-- Index File: [Blank]
-- Automatic Index: [Not Checked]
-- Basic Authentication: [Blank]
-- Basic Credentials List: [None]
-- Enable Advanced ACLs: [Unchecked]
-- IP ACL: [None]
-- Force HTTPS: [Unchecked]
-- Enable HTTP/2 Preloading: [Unchecked]
-- Pass Request To Local PHP Interpreter / Threat Upstream As FastCGI: [Unchecked]
-- (PHP) Router Script: [Blank]

Services: NGINX: Configuration: HTTP(S): HTTP Server (1 Entry)
-- HTTP Listen Port: 8091 (Later changed to 80)
-- HTTPS Listen Port: [Blank]
-- Server Name: 192.168.xxx.zzz
-- Locations: com_siteb
-- URL Rewriting: Nothing Selected
[Everything else left to defaults]


This section was completely deleted
Services: NGINX: Configuration: HTTP(S): URL Rewriting (1 Entry)
-- Short description (to display): root
-- Original URL Pattern (Regex): /
-- New URL Pattern: $1
-- Flag: Redirect

After a restart of NGINX, It routes! Thanks, Fabian!

Hi there,

In the past, I have done this with pfS and the Squid Reverse Proxy tool. But it's been a few years.

I've got OPNsense 19.7-amd64 up and running at a location. I'd like to set up a number of web servers on the LAN side and have NGINX reverse proxy the traffic in to them based on the headers.

I'm not looking to load balance at this point, just setting it so that:

http://sitea.com proxies in to http://192.168.xxx.yyy:80
http://siteb.com:8091 proxies in to http://192.168.xxx.zzz:8091
http://sitec.com proxies in to http://192.168.xxx.aaa:80
https://sited.com proxies in to https://192.168.xxx.bbb:443
https://sitee.com proxies in to https://192.168.xxx.ccc:443

So, I'm starting with one server, siteb.com above. I followed this tutorial, but I doubt I did it correctly. So here's what I've got....

Services: NGINX: Configuration: General
-- Enable nginx: Checked

Services: NGINX: Configuration: Upstream Server (1 Entry)
-- Description: Server1
-- Server: 192.168.xxx.zzz
-- Port: 8091
-- Server Priority: 1

Services: NGINX: Configuration: Upstream (1 Entry)
-- Description: com_siteb
-- Server Entries: Server1
-- Load Balancing Algo: Weighted Round Robin
-- Enable TLS: Checked
-- TLS: Servername override: [Blank]
-- TLS: Supported Versions: [Nothing Selected]
-- TLS: Session Reuse: [Not Checked]
-- TLS: Trusted Certificate: [Nothing Selected]

Services: NGINX: Configuration: HTTP(S): Location (1 Entry)
--Description: com_siteb
--URL Pattern: ^http://siteb.com
--Match Type: Case Insensitive Match ("~*")
--URL Rewriting: [Nothing Selected]
--Enable Security Rules: [Not Checked]
--Learning Mode: [Checked]
--Block XSS Score: [Blank]
--Block SQL Injection Score: [Blank]
--Custom Security Policy: [Nothing Selected]
--Upstream Servers: com_siteb
--Path Prefix: [Blank]
--Cache: Directory: [none]
--File System Room: [Blank]
--Index File: [Blank]
--Automatic Index: [Not Checked]
--Basic Authentication: [Blank]
--Basic Credentials List: [None]
--Enable Advanced ACLs: [Unchecked]
--IP ACL: [None]
--Force HTTPS: [Unchecked]
--Enable HTTP/2 Preloading: [Unchecked]
--Pass Request To Local PHP Interpreter / Threat Upstream As FastCGI: [Unchecked]
--(PHP) Router Script: [Blank]

Services: NGINX: Configuration: HTTP(S): URL Rewriting (1 Entry)
--Short description (to display): root
--Original URL Pattern (Regex): /
--New URL Pattern: $1
--Flag: Redirect

Services: NGINX: Logs: HTTP Access Logs: com_siteb
--Time: 22/Dec/2019:18:32:25 -0500
--IP: [Redacted, but it's mine]
--Username: -
--Status: 404
--Size: 5431
--Referer: http://siteb.com/
--User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36
--Forwarded For: -
--Requested Line: GET /favicon.ico HTTP/1.1

Services: NGINX: Logs: HTTP Error Logs: com_siteb
--Date: 2019/12/22
--Time: 18:32:24
--Severity: error
--Number: 35259#100189
--Message: *1 "/usr/local/etc/nginx/html/index.html" is not found (2: No such file or directory), client: [Redacted], server: com_siteb, request: "GET / HTTP/1.1", host: "siteb.com"

We haven't tried that yet. Running a multi-site business, management gets a little... testy... when we arbitrarily decide to take the network down for no perceivable reason. This may be something we test on a weekend in the early morning hours.

[System Information]

Hi all,

I've got a weird one, but I've now seen it at different locations and it's concerning.

Please note that this one is very similar to this post, also by me, from a while back.

First, some specs because well, everyone loves specs:

System Information

Code Select Expand


Name frasvrfw.{redacted}
Versions OPNsense 19.7.6-amd64
FreeBSD 11.2-RELEASE-p14-HBSD
OpenSSL 1.0.2t 10 Sep 2019
Updates Click to check for updates.
CPU Type Intel(R) Xeon(R) CPU X3450 @ 2.67GHz (8 cores)
CPU usage
Load average 0.53, 0.38, 0.28
Uptime 1 days 03:06:07
Current date/time Tue Dec 17 10:33:25 EST 2019
Last config change Mon Dec 16 3:31:41 EST 2019
State table size 1 % ( 8526/814000 )
MBUF Usage 1 % ( 7100/506546 )
Memory usage 13 % ( 1099/8144 MB )
SWAP usage 0 % ( 0/8192 MB )
Disk usage 2% / [ufs] (1.4G/101G)

Interfaces

Code Select Expand


   GUEST 1000baseT <full-duplex> 192.168.25.254
   LAN 1000baseT <full-duplex> 192.168.3.1
   PLC 1000baseT <full-duplex> 192.168.20.1
   Phones 1000baseT <full-duplex> 192.168.9.1
   Printers 1000baseT <full-duplex> 192.168.6.1
   SAN 1000baseT <full-duplex> 192.168.10.1
   SANBACKUPS 1000baseT <full-duplex> 192.168.16.254
   SECUREWIFI 1000baseT <full-duplex> 192.168.5.1
   SECURITY 1000baseT <full-duplex> 192.168.50.1
   WAN 1000baseT <full-duplex> {redacted}

Ok, with that out of the way, I've got a Dell PowerEdge R210 running as an OpnSense firewall/gateway between multiple networks (separated into VLANs and their own subnets) with the OpnSense firewall at the center.

We've now had this happen twice -- 16-Dec-2019 between 3:20 AM and 7:20 AM, and about a month ago where there was a 35 minute power outage.... so I'm no longer thinking this is a "fluke."

The trigger:
The WAN connection drops at the provider end (in other words, between the internet provider's fiber router and their closest node)

The symptom:
OpnSense stops routing all traffic -- even the internal traffic between internal subnets such as from the LAN to the PRINTER networks. This continues to occur even after the WAN connection is restored and the internet is available again.

The workaround resolution:
Reboot the firewall and all the problems go away... until the next time the internet connection drops.

In both instances, the power in the area has gone out, and our on-site battery backups and generator have kept the building up and running throughout. So my OpnSense logs do not show that the firewall has rebooted. But if I go to Reporting -> Health and look at any metric, ie:

Packets -> LAN
Packets -> WAN
Packets -> IPSec
Quality -> Gateway

They are all flat-lined between those times.

So the question is, what would cause OpnSense to stop routing traffic between internal networks when the WAN connection drops, and is there a way to fix it, short of setting up a check_internet script that reboots the firewall if it can't get to something really common like google?

Thanks in advance, all!

Hi there,

Just wondering if anyone is actively working on generating a pkg for zabbix4.4 -- I accidentally updated my Zabbix server to 4.4.1 (meant to just go to 4.2.x) and I'm seeing errors in the zabbix server log like:

11068:20191117:195807.381 cannot process proxy "[SiteID]": protocol version 4.0 is not supported anymore

So, totally my fault for upgrading too far, but since I did and it made some database changes, I'm now hesitant to go backwards to 4.2.

If someone can walk me through compiling from source on FreeBSD, I'm generally fluent in POSIX environments. I'd be happy to submit whatever I compile.

Thanks!
-Paul

EDIT: I found the script I've been testing/using to fix this issue. Here it is. Your mileage may vary...

(Mine is located as /root/check_internet.sh - this may not be the best place for the code to persist through updates.)

To implement this, you need to open an SSH session to your firewall, log in, and select "8. Shell" from the menu.

Once there, type

Code Select Expand

vi /root/check_internet.sh

The editor you are now in is called vi. Vi is very powerful, but not very user friendly. Here's what you need to do in order to get this going...

Type :i[Enter]. This puts VI in Insert mode.

Paste in the following code. I use PuTTY, and in PuTTY to paste you just right-click after you've copied the code.

Code Select Expand


#!/bin/sh

TMP_FILE=/tmp/inet_up

# Edit this function if you want to do something besides reboot
no_inet_action() {
    shutdown -r +1 'No internet.'
}

if ping -c5 google.com; then
    echo 1 > $TMP_FILE
else
    [[ `cat $TMP_FILE` == 0 ]] && no_inet_action || echo 0 > $TMP_FILE
fi


Now type [Esc] :wq [Enter]. This says "Exit Insert Mode, Command: Write, Quit" and Enter executes it.

After you create it, you need to run

Code Select Expand

chmod a+x /root/check_internet.sh
which makes it executable.

Normally I'd tell you to go to the Web UI and set up a new Cron Job, but OpnSense does not seem to have the ability to accept a custom command, so instead we're going to do this:

Code Select Expand

crontab -e

Again, you're in VI.

Type :i[enter], which puts the editor in "Insert" mode. Go to the bottom of the file and add a new line:

Code Select Expand

*/5     *       *       *       *       /root/check_internet.sh > /dev/null

Now type [Esc] :wq [Enter]. Again, this says "Exit Insert Mode, Command: Write, Quit" and Enter executes it.

What you told the cron (scheduler) to do is "run this command every 5 minutes". If the command succeeds, nothing will happen. If the command fails, it will automatically reboot the firewall.


FYI (because credit where credit is due) I found the base code and modified it from here.

The upside of this script is that if the interfaces on your firewall are not properly initialized or did not properly initialize after a reboot, this will reboot the firewall and give it another chance without you having the drive in to the office and intervene manually.

The downside of this script is, if Google appears to be down from the firewall, it's going to reboot your firewall every 5 minutes. So, be aware of the Pro's and Con's of this script.

Hope this helps. Please let me know if this resolved an issue for you!

Hi all,

Those of you that have been around the *Sense world for awhile probably know me. I've been a fan for a number of years, and have a lot of posts on the pf as well as the opnSense forums. Heck, I may even have a few on m0n0wall, since I go back that far on distros of this firewall.

If you're just now reading this, this is an awesome firewall solution and I've found nothing that compares to it in the commercial world. There's a balance of functionality and UI that I've never seen at this level in a commercial product. So what's below is more for the folks that have been running some version of *Sense for a little while and is not representative of the product of a whole.

Ok, disclaimer over. Now to the real topic:

I have somewhere in the neighborhood of 30-50 *Sense installations in production. These span everything from a terrible DSL connection in the middle of nowhere's-ville to a 200 x 200 Mbps redundant fiber connection in a controlled data center environment. I'm seeing this problem across the board, on all installations. That means everything pf 2.3.2 and newer, and I'm guessing everything opnSense 18.1.x and newer. In other words, I'm thinking this is a FreeBSD problem and not a *Sense problem.

Symptom:

The firewall is set up to reboot on a schedule. After the scheduled reboot, the firewall does not respond to anything remotely. ICMP traffic does not respond, packets are not forwarded, rules are not processed. It's as if the firewall is stuck in the default state of "deny all" for all packets.

Resolution A (a bad one):

Go to the console and log in if necessary. Reboot the firewall using the appropriate menu option. Let the firewall reboot. Check again and see if you can access the outside world, and the outside world can access you.

Resolution B (possibly better?)

Run a script that checks to see if the firewall can reach the gateway device (DSL modem, DOCSIS router, fiber router, etc.) and/or a common website (Google DNS, for instance). If it cannot, issue a reboot command. Possible side-effect: This could reboot the firewall in the middle of the day with no warning to your end users.

Anyone else run into this problem? The most common hardware we use are Dell PowerEdge R2{x}0's as firewalls, so its possible it is specific to those, but I believe I've also seen this behavior on the micro-firewalls we have running made by a reputable Amazon reseller ending in "LI". ;-)

DISCLAIMER: If you can't tell, this email has been formatted to avoid getting a nasty letter from that one company that bought one of the *Sense distributions and has been known for sending nasty letters to people trying to help.