Messages

[1. The Logical Architecture (The Concept)]

Hi all,

Is it possible to set up an OPNSense environment as a high-availability HA Cluster with the following constraints:

- Having private-addressed public interfaces, whilst the public IP address is in a /30 CIDR block?

Example: 1.2.3.4/30 as the public IP address - ISP GAteawy = 1.2.3.5/20 - As it is therefore not possible to have a public IP address per interface in the HA Cluster group on WAN-type interfaces.

Here is the planned setup below, would like opinions if it is a valid setup : thanks in advance for feedback ;-)



To set up High Availability (HA) on OPNsense (via CARP), the official documentation typically states that you need 3 IP addresses in the same subnet: one for the Master, one for the Backup, and one shared Virtual IP (VIP). With a /30 subnet (which only provides 2 usable IP addresses), this standard method is mathematically impossible.

Fortunately, there is an excellent workaround: using the "IP Alias" mode combined with private addressing on the physical WAN interfaces.

Here is how to structure this configuration.

1. The Logical Architecture (The Concept)
The trick is to configure the physical WAN interfaces of your two OPNsense nodes with private IP addresses (for example, in a dedicated /24 or /29 subnet for the interconnection with your public router/gateway), and to set up a private CARP between them.

Then, you assign your single available public IP as an IP Alias attached to this private CARP VIP.

Addressing Schema
Imagine your provider's router (Gateway) has the public IP 1.2.3.5/30 and provides you with the IP 1.2.3.6/30. For the local interconnection, we will use the private network 192.168.100.0/29.

ISP Router / Gateway: 1.2.3.5/30 (Routes traffic to your block)

WAN Interconnection Network: 192.168.100.0/29

OPNsense Master (Physical WAN): 192.168.100.1

OPNsense Backup (Physical WAN): 192.168.100.2

CARP VIP (Virtual): 192.168.100.3

Virtual Public IP (IP Alias): 1.2.3.6 (carried by the CARP VIP 192.168.100.3)

2. Configuration Steps in OPNsense
Step 1: Configure Physical IPs and Private CARP
On the Master, assign the address 192.168.100.1/29 to the WAN interface.

On the Backup, assign the address 192.168.100.2/29 to the WAN interface.

Go to Interfaces -> Virtual IPs -> Settings.

Create a new Virtual IP of type CARP:

Interface: WAN

Address: 192.168.100.3 / 29

Virtual IP Password: [Your password]

VHID: [A unique identifier, e.g., 1]

Step 2: Add the Public IP as an IP Alias
This is where the magic happens to bypass the /30 limitation.

Still in Interfaces -> Virtual IPs -> Settings, create a new Virtual IP.

Choose the IP Alias type.

Interface: WAN

Address: 1.2.3.6 / 30 (Your single public IP)

Virtual IP Association: Select the CARP VIP you just created (192.168.100.3).

💡 Why does this work? The public IP Alias will "graft" itself onto the private CARP IP. When the Master is active, it takes ownership of both the private CARP IP and the associated public IP. If the Master goes down, the Backup instantly recovers the CARP VIP and the public IP Alias.

Step 3: Configure the Gateway and NAT
In System -> Gateways -> Single, ensure that your WAN's default gateway correctly points to your ISP router's IP (1.2.3.5).

In Firewall -> NAT -> Outbound, switch the mode to Hybrid or Manual.

Modify your outbound NAT rules (or create one) so that Internet-bound traffic does not use the physical interface IP (192.168.100.x), but rather the public IP Alias (1.2.3.6).

3. Important Points of Vigilance
The Upstream Router (ISP / Switch): The router providing your /30 subnet must be capable of routing traffic destined for 1.2.3.6 toward the CARP VIP 192.168.100.3. If it is a simple Layer 2 switch or bridge, ensure that the ARP protocol resolves correctly.

Synchronization (pfsync): Do not forget to configure a dedicated interface (often called DMZ or SYNC) for state table synchronization (pfsync) and configuration synchronization (XMLRPC) between your two nodes.

"Promiscuous" Mode: If your OPNsense instances are virtual machines (VMware ESXi, Proxmox, Hyper-V), you must enable Promiscuous Mode / MAC Address Spoofing on the vSwitches of both the WAN and LAN interfaces. Otherwise, the CARP protocol will be blocked."

Bonjour,

Pas moyen de bloquer sur la Box directement les n° surtaxés (ou dans la gestion du compte du FAI) ?

J'ai posé la question à un pote, il gère ça généralement sur le PABX/IPBX en général, mais il m'a indiqué que certains opérateurs n'offrent pas forcément la fonction de blocage :-(

Bonjour,

Merci pour vos pistes de réflexions !

Le problème venait dû fait qu'il n'y avait pas d'association de règle de filtrages avec les redirections de ports...

Problème réglé.

Cool ;-)

Je me suis fait avoir également (de mémoire lors du passage en V24 ou V25 je pense), la création d'un DNAT ne fabrique pas automatiquement la règle firewall associé, il faut le préciser (exemple en V26 ci dessous plus bas). Pour ma part, je laisse le réglage en manuel & je fais la règle à la main => Je préfère contrôler la création des règles par moi même ;-)

You cannot view this attachment.

Bonjour,

Il faut également une règle de firewall pour autoriser le trafic entrant sur l'interface WAN : un Destination NAT seul ne suffit pas, il faut aussi une règle de firewall ;-)

Le NAT et les règles de filtrage (Firewall) sont deux choses distinctes : Si vous créez une règle de Destination NAT (Port Forward) sans règle de pare-feu correspondante, le paquet sera correctement traduit vers l'IP de votre serveur local, mais le pare-feu le bloquera immédiatement après.

Ci dessous, exemple de trafic bloqué AVANT application d'une règle firewall à destination du port TCP:5000 sur l'interface WAN et AUTORISÉ après la mise en œuvre de la règle de firewall.

You cannot view this attachment.

A ma connaissance, non (je ne suis pas expert Fortigate), mais il existe certaines alternatives :

- Q-Feeds, qui effectue une agrégation de sources "malvaillantes", que l'on applique à des regles de filtrage Inbound / Outbound,

- Un produit (abonnement) que je prends chez mon fournisseur de firewall OpnSense (réponse par message privé), et fait plus ou moins le même chose que Q-Feeds,

- ZenArmor, qui est un plugin (Next Generation Firewall - NGFW) dispo pour OpnSense.

Couplé à du filtrage GeoIP (je suis passé sur IPINFO.IO dernièrement), tu devrais trouver déjà pas mal de bonnes choses, simples à déployer.

Hi,

I'm also wondering about the presence of "old" floating rules or not after a migration from 25.x to 26.

Below the rules shown in the legacy rules setup :

You cannot view this attachment.

Does this mean (Automatically generated rules) that rules are still alive ?

If I check under "Rules [New]", so such rules are shown (normal behaviour).

Does those "legacy" Floating rules are necessary at all ?

Hi all@Community,

I'm quite newbie with Wireguard + OPNSense, but my first setups are working fine (for the moment, only in Roadwarrior Tunnel Mode). I can reach the Endpoint, use all devices allowed from the tunnel to the destination LAN, etc...

My question is about Peer Generator : I've been able to generate some peers, copy/paste the configuration at the Roadwarrior Side to use the VPN;

But OPNSense + Wireguard has not any Export Function, like we have with OpenVPN.

I dealt a little with a IA to have opinions, recommended solution is using API + shell script : my customers are relatively small companies, my intend is not to use API.

For sure, I can copy/paste each generated Peer configuration manually in some .conf files (not so much time consuming), but an Export function will be a nice feature ;-)

Any feedback will be appreciated.

Bonjour,

Personnellement, je prends tous les matériels auprès d'une entreprise française, Provya :

https://store.provya.fr/10-firewall-pfsense-opnsense

Les matériels sont testés, éprouvés et validés par l'équipe.

Ils sont très actifs sur la partie pfSense et OpnSense.

Je recommande vivement de travailler avec eux.

PS : j'ai stoppé depuis assez longtemps de commander des matériels chez Amazon ou ailleurs : SAV catastrophique...

Une réponse en message privé.

@vicent

Quelle liste GeoIP utilisez vous dans votre config ?

J'ai l'habitude d'utiliser GeoIPLite country (CSV) de Maxmind, mais leur politique a changé, et l'URL "classique" ne charge pas les listes Maxmind.

Merci pour vore réponse ;-)

[- Issue we meet]

Dual WAN Failover : loose primary Gateway when workload is higher

Hi All !

We are facing an issue with a DualWAN OpnSense Configuration.

Details :

- OpnSense version :    OPNsense 23.7.10_1-amd64
FreeBSD 13.2-RELEASE-p7
OpenSSL 1.1.1w

We have two identicals nodes, running as HA Cluster

- Fisrt Gateway :
Static IP - Priority : 254
Behind a 5G LTE Zyxel NR7101 Routeur
WAN IP is static

- Second gateway :
Static IP - Priority 255
Behind an aDSL routeur

- Gateway Group
Tier1 : Fisrt Gateway
Tier 2 : Second gateway
Trigger Level : Member down

- Issue we meet : when a quite big workload is initiate (ie trying to download a several Gb file), the lantency / packet loss grows up for the Fisrt Gateway, the firewall toggles from Tier1 to Tier2, but the Fisrt Gateway is ssen as back about 10 to 20 seconds later :-(

I tried to setup the gateways monitoring from 8.8.8.8 / 8.8.4.4 to ISP DNS servers : same issue.

I tried to change the Default Packet Loss thresholds for the First Gateway, from Default is 10/20 to 20 /40 : same issue.

Here is one of the last dpinger log :

Code Select Expand

2024-03-08T15:47:17 Notice dpinger ALERT: WAN_5G (Addr: 185.61.146.1 Alarm: loss -> none RTT: 30.7 ms RTTd: 6.3 ms Loss: 20.0 %)
2024-03-08T15:46:23 Notice dpinger ALERT: WAN_5G (Addr: 185.61.146.1 Alarm: none -> loss RTT: 31.2 ms RTTd: 6.7 ms Loss: 21.0 %)
2024-03-08T15:38:44 Notice dpinger Reloaded gateway watcher configuration on SIGHUP
2024-03-08T15:38:44 Warning dpinger send_interval 1000ms loss_interval 4000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 0ms loss_alarm 0% alarm_hold 10000ms dest_addr 185.61.147.1 bind_addr 192.168.1.251 identifier "LIVEBOX_ORANGE_GWv4 "
2024-03-08T15:38:44 Warning dpinger send_interval 1000ms loss_interval 4000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 0ms loss_alarm 0% alarm_hold 10000ms dest_addr 185.61.146.1 bind_addr 192.168.100.251 identifier "WAN_5G "
2024-03-08T15:38:44 Warning dpinger exiting on signal 15
2024-03-08T15:38:44 Warning dpinger exiting on signal 15
2024-03-08T15:37:16 Notice dpinger ALERT: WAN_5G (Addr: 185.61.146.1 Alarm: delay -> none RTT: 85.3 ms RTTd: 116.9 ms Loss: 0.0 %)
2024-03-08T15:37:04 Notice dpinger ALERT: WAN_5G (Addr: 185.61.146.1 Alarm: none -> delay RTT: 203.7 ms RTTd: 184.8 ms Loss: 0.0 %)

Do I need to increase the Packet Loss thresholds ?

The latency for this 5G LTE routeur is slitgly higher than the aDSL one, but no so much.

Thank you so for your opinon and/or any clues / ideas, if you also met such behaviours.

Ton premier post n'est - de toute façon - pas assez clair : fais un schéma de ton infra + adressage IP de toutes tes interfaces.

Entre autres, que vient faire le réseau 172.22.8.0/24 ? Il est où ?

Et Check sur les tutos ici https://provya.net/index.php?liste, il y a pas mal de bons articles. Je bosse avec eux sur les matériels, rien à reprocher ! Les articles sont souvent basés pfSense, mais c'est kif-kif.

Bonjour,

Personnellement, je ne vois pas du tout l'intérêt de vouloir agir ainsi : un pare-feu ne sert pas à faire tourner / exécuter une VM dessus... A la rigueur OPNSense dans une VM, mais même moi je ne ferais pas ça ==> Je préfère installer un Cluster HA OPNSense.

Sur un autre forum (FR), en l'occurence pfSense, j'ai connais un qui ne mégote pas avec la sécurité : il trouvera certainement ceci étrange ;-)

Bonjour,

Je changerais la plage DHCP du serveur VPN, 10.0.29.0/24 par exemple.

Dans ta description, ton interface OPT1 possède une plage IP identique aux adresses obtenues via VPN.

Quel est le type de VPN utilisé ? Si c'est OpenVPN, indiquer les plages IP accessibles ( 192.168.0.0/24,172.29.0.0/24 dans ton cas) dans la section correspondante (IP V4 Local Network)

Back for news ;-)

The HA Cluster has been tested in a separate way, behind an ISP router different from the first post (Sagem Livebox V5).

I added a small 8 ports switch between the ISP routeur & the WAN interface of each Opnsense box.

One modification has been done : the outbound rules where set as "Source : LAN Address" instead of "LAN Net" ==> My expert told me it wasn't relevant enough for the previous CARP_VIP/MacAddress issue we met.



For now, the HA Cluster is fully responsive, the failover works great, incoming VPN or incoming NAT rules are processed ;-) The only "trouble" I have is OpenVPN connection not kept during a failover, but this will be not a real problem for end users.

I will give you some feedback when I well go back on site.