Topics

1

18.1 Legacy Series / CARP - OPNSense slow!

« on: April 13, 2018, 08:18:47 pm »

I am configuring 2 other OPNSense Appliances, running 18.1.6, and I've followed the documentation on how to setup CARP, and have set it up as I have done on 2 other pair of systems.

This time however as soon as I enabled CARP the 2 systems became very slow. They have a dedicated interface and cable connected between them, they can ping eachother fine, but as soon as I change something on the main system its spinning for about 10 minutes before the change is reflected on the other appliance and before I can do anything else on the system, so its very painful.

I tried to look into the logs, but...it takes about 10min of waiting after I do something, and then the logs seem to have changed since v17x..., so not sure where to find all the infos. That said the CARP communication are shown in Green, its just very slow.
The system reacts normally again if I disable CARP.

Does anyone had a similar experience or any idea what can cause this ?

 

2

18.1 Legacy Series / 18.1.5 update and CARP lottery

« on: March 23, 2018, 07:12:10 pm »

I have two lines of firewalls on opnsense appliances, I have updated all 4 of them from 18.1.2 to 18.1.5, for two of them no problem but CARP was messed up for the other 2.
Apparently the CARP configuration got lost during the 18.1.5 update on the backup firewall, so I configured it again, the system was very slow after that but was ok again after a reboot.
The problem however remains with CARP, it has been stable with 18.1.2, and I haven't touched any hardware during the update, but now both system's 7 VLANs Master/Backup will change randomly after some time or a reboot.
for example I have about 3 VLAN's with Master/Backup as they should be and 4 are Master on the Backup and Backup on the Master, and this can change after some time or a reboot seemingly randomly.
Again this was all stable before the update, and its not the first time CARP gets unstable after an update, last time it went stable again after some time and another update, I just don't dare use this system fully in production yet without a backup connection :/

I have tried to revert to 18.1.4, but that didnt help so I updated again to 18.1.5.

Does anyone have any advice where to look and troubleshoot this  ?

3

18.1 Legacy Series / [Solved]CARP backup no connexion

« on: February 09, 2018, 01:29:12 pm »

I have setup 2 appliances with CARP initially on 17.7.7 (or a version close to that one), and at the time the backup appliance could only connect to the internet when the master was down, at the time I didnt know why, but it wasn't that bad as main purpose for the backup was to be available if the master is down, which worked. It was however annoying for updates as the master had to be disconnected for the backup to update.
However after some updates and up until 17.7.12 the backup was able to connect again, but now after I updated to 18.1.1 it cannot anymore, sometimes a couple of pings go through but most of the time it doesnt.

What seems to happen is that when a package is sent out from the VIP ip, it tries to go back to the master not the backup, so it never returns. Although there could be some logic as the backup should respond only if the master is down, why was it working for 6 months, and it is also working on the other 2 appliances in CARP which are behind the first layer of firewalls ?

Has a anyone an idea what could be wrong ?

4

18.1 Legacy Series / [Solved] cannot update from 18.1.1 to 18.1.2

« on: February 08, 2018, 07:31:35 pm »

I have four opnsense appliances running (2x2 in CARP). One pair updated fine from 17.7.12 to 18.1.1 and 18.1.2, the other pair updated fine to 18.1.1, but then I lost connectivity partially with the slave and the master can connect fine, but when I try to update to 18.1.2 it says that the repo cannot be found on that server or that there is no update available. I tried several mirrors, one of them (Switzerland) did offer the update and it was updating for a long time, I selected update again and then it showed it had updated to 18.1.2 but it doesnt seem that it has, as it is still mentioning that the system is running on 18.1.1.

I tried to reboot a couple of times but that didnt help.

This is the error message I get when I update from the console:
"updating OPNsense repository catalogue...
pkg-static: Repository OPNsense load error: access repo file(/var/db/pkg/repo-OPNsense.sqlite) failed: No such file or directory
pkg-static: http://pkg.opnsense.org/FreeBSD:11:amd64/18.1/latest/meta.txz: Not Found
repository OPNsense has no meta file, using default settings
pkg-static: http://pkg.opnsense.org/FreeBSD:11:amd64/18.1/latest/packagesite.txz: Not Found
Unable to update repository OPNsense
Error updating repositories!
A firmware update is currently in progress"

I have however just rebooted, so there should not be another update in progress ?!

Anyway has anyone an idea what can be done to solve this ?

 

5

17.7 Legacy Series / openvpn using external CA doesnt work

« on: November 23, 2017, 07:03:19 pm »

Using the latest 17.7.8 version of opnsense on opnsense hardware - https://www.applianceshop.eu/security-appliances/19-rack-appliances/opnsense-based/opnsense-quad-core-gen3-10gb-ssd.html

OpenVPN works fine when using a self generated CA and Certificates, the issue however is that we want to use our own CA and certificates, and this doesnt seem to work.
The issue seems to be that at SwissSign the server certificate and the user certificate are made from their respective intermediate CA (the intermediate CA is however made from the same root CA), so opnsense/openvpn seems to think that there is a mismatch.

Does anyone know if there is anything that can be configured to make it work ?

There is an issue on the pfsense forum from someone that has the same issue
https://forum.pfsense.org/index.php?topic=136116.0

And a description of the issue on the openvpn forum
https://forums.openvpn.net/viewtopic.php?f=6&t=25322

 

6

17.7 Legacy Series / Ghost dhcp reservations

« on: November 03, 2017, 11:59:21 am »

My setup has 2 lines of 2 CARPed firewalls running opnsense 17.7.6.
Some of my VLAN's show tens of dhcp reservations although I have never made them and the subnet hasnt been used before.
They show up some times on some vlans and then on others seemingly randomly.
The dhcp service is working fine otherwise.

This issue hasn't brought up any serious incident but Ii'd like to understand how this can happen.
Does anyone have an idea ? 

Also for the Opnsense dev's, it would be helpful to be able to select multiple reservations at once, its taking me forever to delete 50-60 reservations at a time..

7

17.7 Legacy Series / [Solved] routing from one interface to another

« on: October 31, 2017, 12:59:30 pm »

I'd like to route one physical interface to another and so far I haven't found a way to make it work.
My setup:
4 opnsense units (17.7.6), a 1st line of two in CARP then a 2nd line also in CARP. There is a DMZ switch behind the first line and a LAN switch behind the second line. The 2nd line is connected to the first line firewalls.  (on the fiber port IX1 to IX0)
There are a number of VLANs configured on the 1st and 2nd line firewalls and this is working fine.

I have however a guest wifi network going on another physical port from the 1st line of firewalls to the 2nd, on port igb3 on both firewalls, and this goes fine. The problem is that I then need to route the guest Wifi through the same fiber port (IX1) as the others to route it to the sole fiber cable going to the different floors, and I cannot get the traffic to go from the igb3 port to the IX1 vlan on the 2nd firewall.
If I ping from guest wifi IX1 VLAN outside or even to the igb3 port on the first firewall, and packet capture it, I can see the ping going to the first interface and coming back to the 2nd line of firewall on port igb3, but then it doesnt go to the vlan on IX1.
(I've setup a static route on the first firewall which routes the traffic back to the 2nd line for that network and that works fine)

So far I've setup a gateway with the IX1 vlan gateway and forced any traffic going to that network coming into the igb3 port on the second firewall to use that gateway, in the firewall rules for the igb3 interface, but it is apparently not doing it.
Does anyone know if this is the way to do this, or if it should be done differently ?

I also tried to setup a static route for that network in the system, routes section, but it wont let me as it mentions that there is already and interface configured for that network.

Does anyone have any advice on how to configure this ?

8

17.7 Legacy Series / [Solved] CARP Setup - xml file ?

« on: September 05, 2017, 10:14:32 pm »

I've followed this guide to setup CARP, but I havent been able to set it up successfully so far - https://docs.opnsense.org/manual/how-tos/carp.html

There is one catch there are two interfaces I should map differently, the hardware is pretty much the same except that one firewall has two additional SPF+ ports, so I need to map port 7,8 to 2,3, but I've been told thats doable by editing the xml file.
The problem is that I don't see anywhere in the documentation at what point and where you can export/import that xml file ?
I went through the steps of creating the same interfaces on both firewalls (only diff is wan/lan on port 2,3 instead of 7,8), setup the firewall rules to accept CARP packages, setup the CARP interfaces (they can ping eachother), setup the virtual ips, updates the outbound NAT, then no dhcp on these two firewalls, and then started to setup the HA sync part, but at this point I guess it couldnt work as I had to edit the xml file to have the correct mapping and I don't know where/how to do this ?
I tried save anyway to see if there may have been some clever abstraction of the underlying port and that the interface name may have been enough, but that didnt work - backup server not reachable.

I may be missing something obvious, but it would be helpful if someone could point me in the right direction.

fw CARP logs after hitting save:
"
Sep 5 21:53:17   mainfw kernel: carp: demoted by -240 to 0 (pfsync bulk done)
Sep 5 21:53:17   mainfw kernel: carp: demoted by 240 to 240 (pfsync bulk start)
Sep 5 21:52:38   mainfw kernel: carp: demoted by -240 to 0 (pfsync bulk fail)
Sep 5 21:51:28   mainfw kernel: carp: demoted by 240 to 240 (pfsync bulk start)
"
using the latest 17.7.1 production version.

9

17.7 Legacy Series / [SOLVED] Rules priorities

« on: August 21, 2017, 11:04:09 am »

Regarding Firewall rules Priorities, floating rules seem to be prioritised over Interface rules.

How about group interface rules, are those checked before or after the interface rule ?

Thx!

10

17.7 Legacy Series / SOLVED - IDS/IPS conf/issue

« on: August 10, 2017, 12:52:56 pm »

I've checked some documentation about IDS/IPS like here: https://docs.opnsense.org/manual/ips.html
But so far I haven't been able to get IDS/IPS to work.

I've enabled it (tried both pattern matchers) only on the main gateway and with a minimal set of rules from abuse.ch, but every-time I enable it I cannot connect anymore to the internet. It seems to block everything.
The firewall logs show that the pings are allowed to pass and I don't see any related logs in the IDS Alerts, only Allowed actions for other traffic, also I don't see anything in the packet capture (even in promiscuous) so at this point I don't know why all connections are blocked as soon as I enable the IDS/IPS ?

Does anyone have an idea what could be wrong, or instructions ?

11

17.7 Legacy Series / [SOLVED] Firewall rules delayed activation

« on: August 04, 2017, 01:33:48 pm »

This may have nothing to do with the update, but it seems that whenever I create a rule it becomes active some time later (15min maybe). Like when I set a rule to block all, I can still access, and when I set it to allow again after a while, it is blocked, and then it is allowed again after some time.

Is this how it is supposed to work ? and if so is there a way to flush rules so that they are applied right away ?

12

17.7 Legacy Series / MIT Kerberos Access Server ?

« on: August 03, 2017, 04:36:56 pm »

There was a mention of MIT kerberos 5 in the 17.1.2 release.
Is this exclusively to support AD access ? I'm asking because we are using LDAP/Kerberos (MIT v5) and I was wondering if it was possible to authenticate  through MIT Kerberos as well ?

Else we are also using SAML2 and OAuth2 (through keycloak), but I don't see these options listed at all..

13

17.1 Legacy Series / Firewall rules - cannot set destination port

« on: June 08, 2017, 06:27:25 pm »

I have two OPN19008R Firewalls running the latest production version of opnsense.
One is the main firewall which allows access to the internet and DMZ, the other one is behind the first one and allows access to the LAN.
I need to allow some servers in the DMZ to communicate to some servers in the LAN on some ports, so on the first firewall I was able to define a rule allowing access to the destination server/vlan on some ports, but on the second Firewall that option is not allowed as it is in the first firewall. I could consider that since the port filtering was done on the first firewall I can assume all traffic coming from those servers is safe, but I'd rather also check on the second firewall, and I'd like to understand why that option is not available (it shows a forbidden sign on mouse over for any port or vlan coming and going anywhere). I checked both firewalls seem to be configured with the same options.

A side question I have enabled the suricata ids and rulesets, then selected "download and update rules" but they still show as not installed, any reason why ?

Having used PFSense before but being new to opnsense these are possibly basic questions, but I'd appreciate any insight