Messages

Issue: Not all interfaces are failing over to the Backup Firewall. When any interface fails on the Master Firewall, the only interface that switches over to the Backup is the interface that fails. All others stay up on the Master.
However, if the Master Firewall goes completely down then all interfaces fail over to the Backup.

I've tested this by unplugging the WAN cable and saw that it failed over to the Backup, but all other interfaces stay up on the Master.I plugged the WAN back in, it failed back to the Master firewall.
I unplugged the LAN cable and it failed over to the Backup, but all other interfaces remained up on the Master.


Setup: I have an HA setup using two OPNsense virtual machines on 20.7.2. The baremetal OS is Ubuntu 20.04.1.
Both baremetals have 4 ports with a bridge configured on all four ports.
The interfaces for both OPNsense VMs are the same:
1. WAN        vtnet0  VHID1
2. LAN          vtnet1  VHID2
3. pfsync      vtnet2 
4. DMZ         vtnet3  VHID3

The WAN ports are connected to a dumb switch.
The pfsync ports are connected directly.
The LAN and DMZ ports are connected to a managed switch ( The managed switch has no routing capabilities, only configured VLANs).

I have "Disable Preempt* unchecked for both the Master and Backup firewall.

I followed the directions for setting up the high availability using:
- https://www.thomas-krenn.com/en/wiki/OPNsense_HA_Cluster_configuration
- https://docs.opnsense.org/manual/how-tos/carp.html

After reading through the forums (reddit, opnsense, netgate, etc.); I know the HA setup is suppose to work that if one connection fails on the Master then all interfaces fail over to the Backup. However, in my own setup that is not the case. I've looked over the configuration several times to see if I've made a mistake, but nothing pops out. I followed the steps in those links above.

I'll keep researching and see what I can tell in the logs, but I thought I'd post here and ask, has anyone else had this issue?

Kind regards,
penley


EDIT:
I've tested failing over from the Master to the Backup again. I pulled the plug on the WAN and watched the logs. The Master still considers itself the Master of the WAN connection, but when I look at the Backup firewall it now thinks it's the Master of the WAN.
The log showed nothing from the Master firewall when I pulled the WAN cable out. The Backup firewall log showed:
kernel: carp: 1@vtnet0: MASTER -> BACKUP (more frequent advertisement received)
kernel: vtnet0: deletion failed: 3

Ok, I think I've figured this out.
I have a NAT rule that is port forwarding rtp 10000-20000 for the pbx. I need to change the openvpn ports because they are within that range.

Update:- changing server OpenVPN ports worked.

OPNsense version: 20.1.8_1

I'm trying to setup the OpenVPN road warrior. I've setup 3 different OpenVPN servers, two using the manual method https://docs.opnsense.org/manual/how-tos/sslvpn_client.html, and one using the OpenVPN wizard.

I have setup OpenVPN servers to use a different port than the defualt OpenVPN such as 11941 and the other two VPN servers use a different port as well. I've set this up before doing that same thing and never had an issue.
However, with this setup I am unable to VPN successfully when hitting the WAN, receive an error TLS Handshake failed. I've checked the TLS keys and they are correct. I also changed one of the VPN server ports to 1194 and when I did that I was able to VPN successfully.

I'll keep researching to try and solve this, but wanted to ask here if anyone had any ideas?

Kind regards,
penley

I'd like to close the loop on this one. I think I've figured out my mistake.
Our primary HA are VMs and their WAN and LAN IP addresses were in the same subnet. This has been changed and ever since then I've not seen the "Authenticate/Decrypt packet error: bad packet ID " message in the OpenVPN log. I still have netflow disabled, but have been trying to change one thing at a time to figure this out.

For anyone else who reads this, do not make my mistake and set the WAN and LAN addresses in the same subnet. It causes issues.

Kind regards,
penley

[Authenticate/Decrypt packet error: bad packet ID (may be a replay): -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings.]

The remote OPNsense machines went unavailable again and then connectivity came back after 12 minutes. I looked in their VPN log and see the message again- Authenticate/Decrypt packet error: bad packet ID (may be a replay):  -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings.


Is this message because I do not have the ovpn interface listed in the listening interfaces within the Netflow configuration? These firewalls are using a site to site OpenVPN connection.

Hello,

I enabled netflow for local capture on several OPNsense machines for LAN and WAN interfaces. These machines have a site-to-site VPN setup.
Once netflow was enabled the connection to the LAN for the remote OPNsense machines went down. Each for around 10 to 15 minutes.

I looked at our main firewall these remote sites connect to and the OpenVPN tunnel to each site never showed as down. So it seems only the connection to the LAN went down.

The version of OPNsense we're using is 20.1.6
I did see this message in the VPN log file on the remote OPNsense machines:
Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #1552650 / time = (1590696725) Thu May 28 16:12:05 2020 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings

I've not seen this message before, but it's only in the log of the remote OPNsense machine, not our main firewall these machines connect to. The "authenticate\decrypt packet error" message shows in the VPN log as starting when the connection was lost and ending when the connection came back up.

Has anyone else experienced this before?

Kind regards,
penley

Just an update to close the loop on this.
The OPNsense firewall is now deployed and the connection is good. All that needed to be done was set the WAN to PPPoE, enter the credentials. It connected as soon as the WAN was plugged in. The static IP we got from the ISP was assigned to the WAN interface on connection.
Thank you all for the help and advice.

Kind regards,
penley

@mimugmail and @kx001, thank you for the quick responses. I am unfamiliar with this type of setup and appreciate ya'll answering. I asked the ISP directly to make sure.
They said "It is assigned once you authenticate with the username and password. You don't need to manually assign it."
I get the impression this is not always the case, so I'm glad it's now clear.

Thank you,
penley

Our ISP says in order for us to connect to their service we need to configure the WAN port for PPPoE.
We have one static IP address the ISP has given us.

I need some help to figure out how to configure the WAN with a static IP and PPPoE?

When I change the WAN interface from Static IPv4 to PPPoE, there's no where to put a static IP address.

Kind regards,
penley

Thank you very much for your help @mimugmail.

Thank you for the quick response. So I guess that means it's not possible at the moment. Do you know or have you seen if this is a feature that will be available in a future release?

With OPNsense 20.1, is it possible to setup a High Availability with two OPNsense firewalls using PPPoE?
The PPPoE is configured on the WAN interface.

I've been searching through the forums and internet. I've only found this back in 2018:https://forum.opnsense.org/index.php?topic=9746.0.

Kind regards,
penley

Is it possible on OPNsense to use one OpenVPN server to multiple sites in a site to site VPN, if the server uses port 1194 then can all other sites connect in using that same port?

For example we'd like our setup to be:
- Site A: The main site
- Sites B, C, and D are in other regions and need to connect back to Site A.

Sites B, C and D have no need to talk to each other.

I'm struggling to find an answer to this on the internet forums and youtube. I did find the following:
https://forum.opnsense.org/index.php?topic=5675.0 and it seemed to be what I was looking for except on one comment it says "works fine with pre shared key" and then said they ended up creating a server for each site.

Edit:
Based off this conversation https://forum.netgate.com/topic/83777/openvpn-multiple-site-to-multisites-routing/13, it looks like you would have to have multiple VPN servers on the main site A firewall to connect each site.
If this is true would it be that if each site was using the same VPN server and coming in on the same port they'd be competing for the same connection?

Kind regards,
penley

We have a single FreeRadius server we want to use to consolidate user authentication with VPN, wireless, etc.
I have the wireless authenticating against AD through FreeRadius, but I cannot get it to work with the vpn.
The information I'm struggling to find is does it work differently when using VPN, for example do I have to configure the ldap module in FreeRadius?
I have OPNsense vpn pointed at FreeRadius, but each attempt to login produces the Error:
(0) pap: WARNING: No "known good" password found for the user.  Not setting Auth-Type
(0) pap: WARNING: Authentication will fail unless a "known good" password is available

(0) ERROR: No Auth-Type found: rejecting the user via Post-Auth-Type = Reject

I've tested this using the PAP module and it works, but I'm not sure how to make it authenticate to AD instead.

The OPNsense version is 17.7 and the FreeRadius version is 3.0.


Kind regards,
penley

Thank you so much @franco! I'll research some more on this as well.