OPNsense Forum

English Forums => High availability => Topic started by: u63725 on September 30, 2020, 02:15:31 pm

Title: OPNsense Firewalls Crashes in HA mode
Post by: u63725 on September 30, 2020, 02:15:31 pm

Hi, I have two OPNsense Firewalls in HA Cluster. Both Firewalls have two ssd's in Raid 0 and runs on version 20.7.3. The Problem is, that firewall 1 hangs up after a half hour. After that firewall 2 becomes master and hangs also up after a half hour. There is no error in Logfiles or on the Screen. I removed one older SSD in firewall one. I could not find the reason for this problem.

Title: Re: OPNsense Firewalls Crashes in HA mode
Post by: mimugmail on September 30, 2020, 05:39:03 pm

Raid0 in hardware Controller or Raid1 geom mirror? Maybe better Install on a clean setup with only one ssd

Title: Re: OPNsense Firewalls Crashes in HA mode
Post by: u63725 on September 30, 2020, 11:02:20 pm

Raid1 geom mirror

Title: Re: OPNsense Firewalls Crashes in HA mode
Post by: mimugmail on October 01, 2020, 10:35:58 am

Troubleshooting would look like this:

1) In cluster put a display on both units and watch for stack traces and collect logs
2) cut the cluster and let both units just run to see if they are alive for more than one day
3) User one FRESH ssd on each node, reinstall and restore config

Title: Re: OPNsense Firewalls Crashes in HA mode
Post by: u63725 on October 02, 2020, 04:42:34 pm

After Firewall one hangup Firewall two hangs also up after some minutes. On Firewall one is no error message on the screen, on Firewall two is an arp error:

https://imgur.com/a/SSmkGvk (https://imgur.com/a/SSmkGvk)

Title: Re: OPNsense Firewalls Crashes in HA mode
Post by: mimugmail on October 02, 2020, 06:20:41 pm

Next test, both units without cable attached, wait if it hangs

Title: Re: OPNsense Firewalls Crashes in HA mode
Post by: u63725 on October 03, 2020, 11:36:17 am

Should all data cables be disconnected from the firewalls, or should only the sync cable be left on? It should be mentioned that I only allowed CARP in the firewall rules for the sync interface.

Title: Re: OPNsense Firewalls Crashes in HA mode
Post by: mimugmail on October 03, 2020, 02:47:35 pm

And what about config sync? If it's a direct connection you can safely allow everything

Title: Re: OPNsense Firewalls Crashes in HA mode
Post by: u63725 on October 03, 2020, 03:00:56 pm

Yes it is a direct connection for sync. The communication between the firewalls is established. On the Ubiquity Core Switch is Rapit Spanning tree enabled. When I disconnect WAN and the Vlan Uplink on both firewalls, direct connection for Sync is still connected there are no hangups on both Firewalls. On the WAN interface is DHCP enabled and on the LTE Modem side both Firewalls have a static ip address assingment.

Title: Re: OPNsense Firewalls Crashes in HA mode
Post by: mimugmail on October 03, 2020, 07:32:27 pm

DHCP and carp ha doesnt really work well

Title: Re: OPNsense Firewalls Crashes in HA mode
Post by: u63725 on October 03, 2020, 07:53:27 pm

DHCP on WAN or on LAN?

Title: Re: OPNsense Firewalls Crashes in HA mode
Post by: mimugmail on October 03, 2020, 08:07:57 pm

DHCP in WAN. But for me it seems you have a loop somewhere and not related to hardware

Title: Re: OPNsense Firewalls Crashes in HA mode
Post by: u63725 on October 04, 2020, 09:52:19 pm

Is it recomendet to enable rstp on WAN and LAN Switch? How can I find the loop, are there some specific messages in the log file

Title: Re: OPNsense Firewalls Crashes in HA mode
Post by: mimugmail on October 04, 2020, 10:25:08 pm

Only on the Switch .. but Unify are veeeery bad at logging

Title: Re: OPNsense Firewalls Crashes in HA mode
Post by: u63725 on October 04, 2020, 11:26:58 pm

https://imgur.com/a/GIr5LTg (https://imgur.com/a/GIr5LTg) This are the Ubiquity Core Switch settings. I set the Priority to 40960. Does CARP have to be allowed over the firewall rules for the WAN and LAN interfaces?