Messages

The concern is over splitting resources. How many places can one person look for assistance with an issue. Suppose something big get uncovered in Discord and fixed, but that info never gets back to official channels, so they never officially fix the issue?

I agree.

The other challenge I have with discord or other "chat-like" communities, is you can't search them via web search engines. History on Discord tends to dissolve into the ether, whereas a forum - it's preserved and accessible to all with nothing more than a web browser. Using kagi, duckduckgo, google, or whatever flavor - I can aggregate information from reddit, forum posts, github, personal websites, and even the pfsense communities (wherein a similar problem/solution could exit), and the rest of the entire internet.

With discord, you can't do that. You must search in the app and hope someone had an un-threaded real-time discussion that was relevant to your topic of interest.

Threading is terrible in discord - it's a chat and voice app first and foremost. It's is fine for Realtime discussion of an issue where low latency and rapid exchange is required, and not a whole lot else.

I work at a formula 1 team. We used it in specific simple circumstances that didn't warrant a more enterprise-y firewall (such as a palo alto networks device)

For reference, it appears the format suggested by franco simply follows the directory structure that can be seen in the mirror, since apparently you are instructing opnsense to pull down those packages directly. For example, I'm trying to go to 24.7.1 even though 24.7.6 is available.

The mirror that automatically showed up when I browsed to /ui/core/firmware#status on my device is: https://pkg.opnsense.org/FreeBSD:14:amd64/

That mirror has the folder inside it: 24.7/MINT/24.7.1/latest/
so I will put that into the flavour field and install the version I am after.

I am a Google Fiber subscriber. My environment is simple with an active/passive firewall -  a KVM VM with hardware passthrough of a quad port NIC, and physical hardware firewall with some intel NICs. I have a single WAN, and a single LAN interface running CARP.  The VPNs I use continue to function after failover. Stateful protocols such as ipsec or openvpn will drop and need to re-negotiate, but can reconnect immediately. Wireguard has no such issue.

Spali's github post is very useful: https://gist.github.com/spali/2da4f23e488219504b2ada12ac59a7dc

I have updated my personal script to the following, which is a mash of theirs and mine, which I posted in Reddit some time ago: https://www.reddit.com/r/opnsense/comments/runb4r/diy_ha_activepassive_for_home_internet/

Code Select Expand


#!/usr/local/bin/php
<?php
require_once("config.inc");
require_once("interfaces.inc");
require_once("util.inc");
$subsystem = !empty($argv[1]) ? $argv[1] : '';
$type = !empty($argv[2]) ? $argv[2] : '';
if ($type != 'MASTER' && $type != 'BACKUP') {
    log_error("Carp '$type' event unknown from source '{$subsystem}'");
    exit(1);
}
if (!strstr($subsystem, '@')) {
    log_error("Carp '$type' event triggered from wrong source '{$subsystem}'");
    exit(1);
}
foreach($config['interfaces'] as $ifkey => $interface) {
    if ($ifkey=='opt3') {
        if ($type == 'MASTER') {
            log_msg("Carp Status is now Master!");
            log_msg("Enabling interface: $ifkey - {$interface['if']}");
            shell_exec("/sbin/ifconfig {$interface['if']} up");
            $config['interfaces'][$ifkey]['enable'] = '1';
            write_config("enable interface '$ifkey' due CARP event '$type'", false);
            interface_configure(false, $ifkey, false, false);
            sleep(1);
            log_msg("Restarting DHCPD");
            shell_exec('pluginctl -s dhcpd restart');
            sleep(1);
            log_msg("Issueing dhclient command to request a DHCP lease");
            shell_exec("dhclient {$interface['if']}");
        } else if ($type == 'BACKUP') {
            log_msg("Carp Status is now Backup!");
            log_msg("Disabling interface: $ifkey - {$interface['if']}");
            shell_exec("/sbin/ifconfig {$interface['if']} down");
            unset($config['interfaces'][$ifkey]['enable']);
            write_config("disable interface '$ifkey' due CARP event '$type'", false);
            interface_configure(false, $ifkey, false, false);
            log_msg("Stopping DHCPD");
            shell_exec('pluginctl -s dhcpd stop');
        }
    }
}
?>


(the forum is breaking the greater than and less than in the PHP brackets at the start and end, correct them yourself)

• This version will also manually "down" interfaces, as disabling them does not appear to fully "shut" the interface in my environment. This can cause mac flapping, and all of the issues related to that condition.
• My version also stops the DHCP Daemon, which ensures that I only have one DHCP server running on my LAN. I need the backup device to actually become "passive". Calling dhclient may not be necessary with the interface_configure call, but it's a holdover from when I previously only used shell_exec("/sbin/ifconfig {$interface['if']} down");   to up/down the interfaces, instead of enabling/disabling the interfaces.
• I use log_msg instead of log_error so that these events show up in the general system log as a "notice".

I do recommend creating a gateway with "Upstream Gateway" checked and a higher metric than the normal WAN gateway, as per spali's github comments to allow the backup to reach the internet via the LAN.

I also recommend disabling the "Backup" router's WAN interface - so that your secondary device will boot up with the WAN in disabled state, and the CARP script will re-enable the interface if CARP goes master. This prevents the devices from both booting up and each having active WAN interfaces.

Oh how funny! I've been working on this independently over the past few days and didn't check this thread to see you've solved it a couple of days ago!
We've effectively arrived on the same method to achieve this. Except your calls, Spali, are probably much better since you are using the config system's normal calls (which I'm not familiar with. I'm instead smashing in console commands via exec, equivalent to using a hammer. (unsanitized code execution risks here!)

Anyway, I also disable DHCPD on the passive/backup device (so I don't have two DHCP servers on my LAN) and make a call to the dhcp client to request a new lease on the WAN interface. I think we could also enumerate "wan*" interfaces to facilitate environments with multi-wan.


For reference, I create the following as this file on both devices: usr/local/etc/rc.syshook.d/carp/50-DHCP
and then "chmod +x 50-DHCP"

Code Select Expand

#!/usr/local/bin/php

<?phprequire_once("config.inc");require_once("interfaces.inc");require_once("util.inc");$subsystem = !empty($argv[1]) ? $argv[1] : '';$type = !empty($argv[2]) ? $argv[2] : '';if ($type != 'MASTER' && $type != 'BACKUP') {    log_error("Carp '$type' event unknown from source '{$subsystem}'");    exit(1);}if (!strstr($subsystem, '@')) {    log_error("Carp '$type' event triggered from wrong source '{$subsystem}'");    exit(1);}foreach($config['interfaces'] as $ifkey => $interface) {	if ($ifkey=='wan') { // could change this to match on wan* interfaces for multi-wan setups, maybe?		if ($type == 'BACKUP') {			log_error("Carp Status is now Backup!");			log_error("Shutting interface: {$interface['if']}");			shell_exec("/sbin/ifconfig {$interface['if']} down");			log_error("Stopping DHCPD");			shell_exec('pluginctl -s dhcpd stop');		} else if ($type == 'MASTER') {			log_error("Carp Status is now Master!");			log_error("Starting interface: {$interface['if']}");			shell_exec("/sbin/ifconfig {$interface['if']} up");			log_error("Restarting DHCPD");			shell_exec('pluginctl -s dhcpd restart');			shell_exec("dhclient {$interface['if']}");		}	}}?>

For future reference in case Spali's github post ever disappears, they are doing the following instead of my foreach statement:

Code Select Expand

$ifkey = 'wan';
if ($type === "MASTER") {
    log_error("enable interface '$ifkey' due CARP event '$type'");
    $config['interfaces'][$ifkey]['enable'] = '1';
    write_config("enable interface '$ifkey' due CARP event '$type'", false);
    interface_configure(false, $ifkey, false, false);
} else {
    log_error("disable interface '$ifkey' due CARP event '$type'");
    unset($config['interfaces'][$ifkey]['enable']);
    write_config("disable interface '$ifkey' due CARP event '$type'", false);
    interface_configure(false, $ifkey, false, false);
}


Edit: For those who may be looking for a DIY on how to enable this, I have a small write-up on the opnsense subreddit, here:  https://old.reddit.com/r/opnsense/comments/runb4r/diy_ha_activepassive_for_home_internet/