Messages

In theory ZFS should never use more than 50% of that IIRC so you should have nothing to worry about despite these values.

That used to be a hard limit set in Linux because Linux memory management had difficulties flushing ARC in case of memory pressure. In FreeBSD such a limit does not exist and ARC will happily use all available memory. This is not a problem. It will be freed if needed.

I don't know the current state on Linux. At least TrueNAS removed the hard 50% limit AFAIK.

Yes, but why even consider buying a new Checkpoint unit? There are many more affordable options for running OPNsense as @sopex already pointed out.

Why buy Checkpoint gear to run OPNsense in the first place?

Did you configure firewall rules permitting the necessary access? If yes, please show them - there is probably something not quite right about them.

https://forum.opnsense.org/index.php?topic=52069.msg268125#msg268125

[ee]

Whos going to explain to absolute beginners that quitting and saving your changes requires pressing Esc key and then : and typing wq. Im trying to make these instructions as simple as possible for everyone. But the easiest way is what Cedric recommended.

As I was saying ee is available as a reasonably easy alternative to vi. Introduced in August 1995, so available since FreeBSD 2.1 release.

Also you do not need to install nano to edit a file. Both vi and ee (the "easy editor") are available out of the box.

FreeBSD as a "traditional" Unix does not have an explicit OOM killer. But in situations of memory pressure, whenever a process demands swap space while swap space is exhausted, it's killed.

Meaning if both sides are not actively talking proper LACP the LAGG will not establish....

Check the source please - possibly I am reading it wrong. I have a fair knowledge of C but no experience with these parts of the kernel code. All in all I am stuck in the 70s i.e. Lions' Commentary, and of course Minix ;-)

[net/if_lagg.c]

This home lab would of course work just as well with just one link per system. I only use LAGG for all core infrastructure because I can - and to gain experience with such setups.

In the production DC we have MLAG to catch the complete loss of a single switch. I *think* we use 30s - which is good enough for hosted web applications, IMHO. STP convergence is in the same time range. Flapping of course is a different beast altogether and somehow even worse than a complete loss of connectivity.

I could not find any documentation on the "strict" option, though. So I turned to the tried and true method of "use the source, Luke".

In net/if_lagg.c, around line 1538 we find:

Code Select Expand

struct lacp_softc *lsc;
struct lacp_port *lp;

lsc = (struct lacp_softc *)sc->sc_psc;

switch (ro->ro_opts) {
[...]
case LAGG_OPT_LACP_STRICT:
lsc->lsc_strict_mode = 1;
break;
case -LAGG_OPT_LACP_STRICT:
lsc->lsc_strict_mode = 0;
break;

In net//ieee8023ad_lacp.c we have a per LCAP partner bit mask that more or less defines which variables we accept from the partner on reception or not:

Code Select Expand

/*
 * partner administration variables.
 * XXX should be configurable.
 */

static const struct lacp_peerinfo lacp_partner_admin_optimistic = {
.lip_systemid = { .lsi_prio = 0xffff },
.lip_portid = { .lpi_prio = 0xffff },
.lip_state = LACP_STATE_SYNC | LACP_STATE_AGGREGATION |
    LACP_STATE_COLLECTING | LACP_STATE_DISTRIBUTING,
};

static const struct lacp_peerinfo lacp_partner_admin_strict = {
.lip_systemid = { .lsi_prio = 0xffff },
.lip_portid = { .lpi_prio = 0xffff },
.lip_state = 0,
};
[...]
if (lp->lp_lsc->lsc_strict_mode)
lp->lp_partner = lacp_partner_admin_strict;
else
lp->lp_partner = lacp_partner_admin_optimistic;

The only actual code path using that mechanism is in lines 1732 ff. and 1812 ff.

Code Select Expand

/*
* XXX Maintain legacy behavior of leaving the
* LACP_STATE_SYNC bit unchanged from the partner's
* advertisement if lsc_strict_mode is false.
* TODO: We should re-examine the concept of the "strict mode"
* to ensure it makes sense to maintain a non-strict mode.
*/
if (lp->lp_lsc->lsc_strict_mode)
lp->lp_partner.lip_state |= LACP_STATE_SYNC;
[...]
static void
lacp_sm_rx_update_default_selected(struct lacp_port *lp)
{

LACP_TRACE(lp);

if (lp->lp_lsc->lsc_strict_mode)
lacp_sm_rx_update_selected_from_peerinfo(lp,
    &lacp_partner_admin_strict);
else
lacp_sm_rx_update_selected_from_peerinfo(lp,
    &lacp_partner_admin_optimistic);
}

So essentially strict mode clears some of the information received by the partner because these flags are (supposedly) not part of the 802.3ad standard. Looks like more or less a no-op to me. See the "XXX" comment above.

I'll re-enable it and whatch what happens.

No more flapping during the night and half a day. So it seems the conservative approach is to use slow timeouts.

I vaguely remember reconfiguring all my LAGG ports to use the same settings across all devices a couple of months ago. Probably I changed OPNsense-switch to fast at that time.

Hi all,

update - I can confirm there might be an interoperability problem with Mikrotik devices and FreeBSD/OPNsense concerning LACP.

Back in '24 I did not notice anything strange as I reported above but for a couple of months now we had intermittent network outages that I could track down to the OPNsense to main switch LACP connection. Possibly introduced by some Mikrotik update, which I track regularly (sticking to the "stable" branch).

Since I changed the LACP timeout from 1s/fast to 30s/slow on both sides and disabled "strict" mode on the OPNsense side the connection now seems to be stable. On the Mikrotik side there is nothing to adjust but the timeout. I also disabled flowid explicitly on OPNsense but if I am not mistaken that was the default all the time, anyway.

This setting sets sysServices to 76 while the setting without Layer 3 visibility checked is 72. The sysServices parameter in snmpd.conf populates the matching OID telling a network management system which services the managed host runs.

https://oidref.com/1.3.6.1.2.1.1.7

72 means applications and end-to-end
76 means applications, end-to-end and IP

HTH,
Patrick

Configure the OPNfirewall with interface WAN port
to operate with a public ip My question is is it safe ?

Well, it's a firewall, which means being safe/secure is the purpose. And if configuration of the WAN parameters to get your uplink working is the only thing you change from the installation defaults, then, yes, you are safe.

As soon as you start adjusting the firewall policy or anything else, that question can only be answered by you.

In AdGuard Home you could do something like this:

Code Select Expand

'rewrites':
  - 'domain': *.logs.roku.com
    'answer': 127.0.0.1

And in Unbound: