Messages

1

22.1 Legacy Series / Re: CARP MASTER during reboot despite maintenance mode

« on: February 24, 2022, 04:10:49 pm »

Well these are the very two buttons I always check.
Temporarily disable CARP won't survive the reboot (which is just what I need), and persistent CARP maintenance state doesn't prevent CARP becoming MASTER unconditionally either, which is the very reason for this post.

2

22.1 Legacy Series / Re: py37-markupsafe still exist after 22.1 update

« on: February 24, 2022, 12:55:38 pm »

Just checked on a box that never had manual fixes, but that py37-markupsafe leftover nevertheless.
pkg remove py37-markupsafe is your friend ;-)

3

22.1 Legacy Series / Re: CARP MASTER during reboot despite maintenance mode

« on: February 24, 2022, 12:48:50 pm »

in this case you might try to disable carp completely.

This is what I'd expect from "maintenance mode". I'm not aware of any other means in opnsense to disable carp. Digging FreeBSD docs, I found sysctl net.inet.carp.allow. So am I supposed to use that tuning or am I missing something? (some "enable/disable CARP" opnSense setting)?

4

22.1 Legacy Series / Re: CARP MASTER during reboot despite maintenance mode

« on: February 23, 2022, 11:14:13 am »

Ok did a while to understand, because I was looking for the problem in the router...

To summarize, if STP is configured on the switch, it will not forward traffic for a while directly after the port is physically up; in consequence the freshly rebooted firewall won't receive CARP packets from the master and assume it's dead.

However, this doesn't explain the initial question: why does the firewall do carp at all? I want CARP disabled in the first place to prevent such very glitches.

5

22.1 Legacy Series / Re: Interface errors after Upgrade

« on: February 23, 2022, 10:10:22 am »

Some days before the errors started, the switch was replaced (cabling and switch are in the provider's realm), but the switch was still silent afterwards. Errors began right after upgrade, so I wonder if the driver now can detect some error situation that it couldn't before.

Actually, I have TWO firewalls in CARP configuration. The regular error rate is on the carp master, but there are also some error on the backup.

6

22.1 Legacy Series / Re: CARP MASTER during reboot despite maintenance mode

« on: February 22, 2022, 05:56:28 pm »

I think I have seen STP packets, I think I can have them disabled.
But what has spanning tree to do with carp? I'd expect only proto-112 packets to have any impact (and least of all in maintenance mode).

7

22.1 Legacy Series / Re: Assigning physical interfaces ...

« on: February 22, 2022, 04:37:30 pm »

I played with this, and apparently (as Franco stated) they're 'somehow' applied. But this means, if you change settings (tso or so), the lagg members will take the settings only on reboot from global settings, no possibility to tweak the hardware online or individually.

8

22.1 Legacy Series / CARP MASTER during reboot despite maintenance mode

« on: February 22, 2022, 04:33:59 pm »

When doing regular maintenance on our CARP cluster, I regularly disable CARP on the machine and enter persistent maintenance mode. I'd expect it to never get MASTER until I enable CARP again.

Now, I rebooted the machine (22.1.1), and while it came up I glanced "Timeout on ix2, becoming MASTER" on the console for a second or so until it stepped back to BACKUP.
While I also have layered interfaces (vlan over lagg over 10GBit), this very ix2 interface is just a plain 1GBit onboard Intel NIC, connected to a switch, no VLAN no whistles or bells (upstream internet).

Having double master even for fractions of a second will screw up network traffic more or less badly, so this really isn't good and shouldn't happen, maintenance mode or not.

So how to safely reboot a router without triggering major trouble?

9

22.1 Legacy Series / Re: Interface errors after Upgrade

« on: February 22, 2022, 03:32:57 pm »

The internet provider guy says there's no errors on the switch side.

sysctl -A shows
  dev.ix.2.mac_stats.checksum_errs: 15381
  dev.ix.2.mac_stats.rx_errs: 15381
which correspond with 15381 IErrs from netstat.
Since there seems a constant error rate, I captured some 5 minute and filtered with wireshark for errors:

eth.fcs.status=="Bad" || ip.checksum.status=="Bad" || tcp.checksum.status=="Bad" || udp.checksum.status=="Bad"

But the result was zero.

10

22.1 Legacy Series / Re: Interface errors after Upgrade

« on: February 19, 2022, 12:39:48 pm »

netstat -i output, redacted after 42h uptime:

Name    Mtu Network   Address  Ipkts        Ierrs  Idrop  Opkts        Oerrs  Coll
ix2    1500 <Link#5>  xx:xx:xx 40644829 6335  0       59986652  0        0

The firewall is monitored by checkmk, which calculates the error rate. Attached the current graph (was flat zero for the last 400 days until recently)

I agree this isn't a horribly high error rate, but the firewall was quiet for years until the update. So something must have changed. Maybe some driver issue?

11

22.1 Legacy Series / Interface errors after Upgrade

« on: February 18, 2022, 06:30:19 pm »

After upgrading from 21.7 to 22.1.1, my firewall shows an error rate of 0.05/s avg on the WAN interface (SuperMicro A2SDI on-board NIC connected to a Juniper switch afaik), which used to be zero before (netstat -i Ierrs). The machine has 4 more connections, all are still at zero errors.

What may be the reason?

12

German - Deutsch / Re: OPNsense 21.7 und entfallene "Custom options" bei Unbound DNS

« on: August 26, 2021, 11:04:38 am »

Mehr Feature Requests auf GitHub, mehr Diskussionen im Forum.. mittelfristig bis langfristig eine bessere Unbound Integration dadurch.


Grüsse
Franco

Ich verstehe den Ansatz, allerdings sind in Unbound allerhand recht exotische Optionen möglich die nicht nur ausgesprochen mühsam komplett umzusetzen sind, sondern auch für den Allerweltsuser die Angelegenheit sehr unübersichtlich machen würde. Die Extraoptionen sind zudem so speziell, dass ein Einzeilen-Hilfetext nicht ausreicht, da muss man schon die richtige Unbound-Doku lesen.
Der häufigste Fallstrick beim custom-Feld war sicher die Notwendigkeit, die Section "server:" mitschreiben zu müssen, das wäre doch im Hilfetext klärbar.

13

20.7 Legacy Series / Re: IPSEC traffic stalling after 20.7.1 upgrade

« on: September 04, 2020, 07:10:44 pm »

I did an after-hour test, after upgrading the fw to 20.7.2.

The tunnel traffic still stalls after a while (it did so after about 100MB inbound traffic).

When pinging a remote host, I see ICMP on enc0 entering the tunnel, a corresponding outgoing ESP packet on wan, but no returning packet; there's still communication on port 500, with no anomalies (afaics) in the log.

Switching back to the downgraded fw, same config (synced from the 20.7 machine) works flawlessly.

14

20.7 Legacy Series / Re: IPSEC traffic stalling after 20.7.1 upgrade

« on: September 02, 2020, 11:47:12 am »

I did a packet capture on WAN, no more ESP packets visible.
Didn't capture on other interfaces.

15

20.7 Legacy Series / Re: IPSEC traffic stalling after 20.7.1 upgrade

« on: September 02, 2020, 09:45:05 am »

No hint in the logs on either side of the tunnel (already elevated some log levels)