FRR daemon combined with CARP

[rlnw]

Thanks in advance for infos and help.
We are using two OPNsense-Firewalls (Version: 19.7.3) with some VLAN Interfaces and CARP.

Examples Firewall 1:
Vlan200, VIP 10.2.200.1, Status MASTER
Vlan610, VIP 10.2.161.1, Status MASTER

Examples Firewall 2:
Vlan200, VIP 10.2.200.1, Status BACKUP
Vlan610, VIP 10.2.161.1, Status BACKUP

FRR Configuration as follows (both Firewalls, Screenshot attached):

Routing: General, Configuration:
Enable               x
Enable CARP Failover x
Create a logfile     x

Therefore, FRR daemon is running on Firewall1 (MASTER) and not running on Firewall2 (BACKUP).

OSPF Configuration,
both Firewalls:

General:
OSPF enabled,
Passive Interfaces (the Vlan Interfaces),
no Route Redistribution

Networks:
VLAN-Ranges (e.g. 10.2.161.0/24)

Interfaces:
Transport-VLAN-Interface (only Interface without CARP Config)

Failover-Test:
Shutdown of Firewall1 -> Firewall2 becomes MASTER.
But:
FRR daemon does not start automatically on the new MASTER, manual start necessary.

Is there any config issue or workaround?

[mimugmail]

Something in the logs on unit2? Normally it should start right away

[rlnw]

Good Morning,

Sorry 4the delay. I have repeated the test with logging level DEBUG for the Routing log, both Firewalls.
Nothing in the Routing Log of Firewall2 during Shutdown of MASTER Firewall1.
But in the General Log, I have found the following:

OUTPUT OMITTED
Sep 25 07:29:28    kernel: carp: 202@lagg0_vlan202: BACKUP -> MASTER (master timed out)
Sep 25 07:15:05    kernel: pflog0: promiscuous mode enabled
Sep 25 07:15:05    kernel: pflog0: promiscuous mode disabled
Sep 25 07:15:04    root: /usr/local/etc/rc.d/frr: WARNING: failed precmd routine for ospfd
Sep 25 07:15:04    root: /usr/local/etc/rc.d/frr: WARNING: failed precmd routine for zebra
OUTPUT OMITTED

I have searched for the 2 frr warnings, but not very successful yet.
Other logs to activate/check for this issue?

[rainerle]

Hi,

just had the same issue and there is a patch existing:
You are using lagg interfaces with vlans on top. devd does not pick up on these interfaces.
https://github.com/opnsense/core/pull/3721

Upgrade to 19.7.4_1 and apply above patch on the CLI using
opnsense-patch 201ed68
then
/etc/rc.d/devd restart


That should fix it.

[rlnw]

Hi,

Thanx a lot for the info.

On applying the patch, I receive the following - does this mean, the interfaces have to be renamed?

Fetched 201ed68 via https://github.com/opnsense/core
Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|From 201ed6894f36a41d9b608454cc29f9f6478866ae Mon Sep 17 00:00:00 2001
|From: stumbaumr <stumbaumr@yahoo.de>
|Date: Tue, 24 Sep 2019 08:23:36 +0200
|Subject: [PATCH] devd does not pick up on laggX_vlanXXXX devices... (#3721)
|
|...since the underscore devices are filtered out.
|---
| src/etc/devd/carp.conf | 2 +-
| 1 file changed, 1 insertion(+), 1 deletion(-)
|
|diff --git a/src/etc/devd/carp.conf b/src/etc/devd/carp.conf
|index 48ddb616c3..5a99579106 100644
|--- a/src/etc/devd/carp.conf
|+++ b/src/etc/devd/carp.conf
--------------------------
Patching file etc/devd/carp.conf using Plan A...
Hunk #1 succeeded at 5.
done
All patches have been applied successfully.  Have a nice day.

[rlnw]

Add:

I have tried to rename the VLAN Interfaces via GUI -
e.g. LAGG0_Printer to LAGG0-Printer.
But the patch has not been applied.

On the console, there are lagg0_xxx interface namings, e.g.:
VLAN210_Printer (lagg0_vlan210) -> v4: 10.2.210.3/24
 VLAN211_Infra (lagg0_vlan211) -> v4: 10.2.211.3/24

current version:
OPNsense 19.7.4_1 (amd64/OpenSSL)

I did not find a way to rename the lagg0_xx interfaces.

[rainerle]

No need to rename the interfaces.

But you have to restart the devd daemon (/etc/rc.d/devd restart ) to use the adjusted configuration.

And then your failover tests should work.

[rlnw]

devd daemon was restartet -
/etc/rc.d/devd restart
Stopping devd.
Waiting for PIDS: 27199.
Starting devd.

But after the patch and devd restart on both nodes, when shutting down Firewall1, Firewall2 gets the master. But FRRouting Daemon is not started there. You either have to start it manually on Firewall2 - or bring Firewall1 back online again.

[rainerle]

Have you applied the patch to both firewalls and restarted devd on both firewalls?

[rlnw]

Yes, I did so.
Update, patch, devd restart on both devices.

[rlnw]

Hello all,

good news.
carp.conf was upgraded after the patch automatically on Firewall2, but not on Firewall1.
We have edited the file manually on Firewall1, restart and now the failover is working.
Thanx a lot and have a nice weekend.

[franco]

opnsense-patch is a local tool and you should be aware that updates will remove the patch unless it is shipped by default.


Cheers,
Franco

[rlnw]

Thanx a lot for the hint.
We will check this during the next upgrade.