Messages

I'm running two CARP-mirrored firewalls, and want to configure an IPSec VPN. Naturally, the virtual WAN address should be used for all traffic, so I configured only the virtual IP as "Local addresses".

When initiating a connection via the IPSec status page, the logfile shows "sending packet: from <virtual IP>[500] to <remote IP>[500]", but this is a blatant lie: checking with "tcpdump -ni <wanif> port 500" on the WAN interface, actual traffic is using the primary WAN IP address, not the VIF.

As a workaround, I added all WAN IPs on the remote FW as well, which seems to work:
- When initiating from the remote FW, phase 1 and phase 2 will use the virtual IP as expected
- When initiating on the CARP master FW(*), phase1 traffic uses the primary IP, while phase2 uses the Virtual IP

(*)note: Starting the connection by pressing the connect button in "Status Overview" doesn't work (nothing logged on any loglevel); to initiate the connection, the setting needs to be disabled and re-enabled.

Anything that I might miss?

That's helpful.

Thanks!

I have several legacy IPSEC Tunnel Settings, and I wonder if I need to migrate them all at once to the new Connections settings, or if I can migrate them one by one.

I'd like to add one connection at a time, disable the old tunnel setting, and test the new one, so both configuration flavors would be active at the same time.

I just saw that there actually IS traffic to the upstream ftp server in PASV mode, and the setup works again after I corrected ftp-proxy source address to use the outbound NAT interface address as well.

Case closed.

After upgrading a firewall from 24.1.x to 24.7.7, ftp-proxy doesn't work any more. Analyzing traffic with tcpdump on both LAN and WAN interface:

- in PASV mode, the client sends SYN packets to the port as returned from ftp-proxy, but there's not traffic to the upstream ftp server.
- in active mode, the server sends SYN packets to the port as presented by the PORT command that ftp-proxy issued upstream, but won't forward any traffic to the client.

I have logging enable on both the client-to-ftp-proxy redirect on Port21, as well as client-to-server traffic for the data connection, both log as "pass" when issuing the client data command.

I checked against that FTP server with another client/firewall (different site, a lot simpler firewall setup), also on 24.7.7, which works correctly there.

From the ftp-proxy man pages, I'd expect to see something with

Code Select Expand

pfctl -a ftp-proxy -s rules, but there's nothing while the data connection is stuck.

I'm out of clues now, anybody with an idea?

Regards,
Andreas

Same thing with me. Not only the gateway assignment in the WAN interface was removed, the gateway definition itself was gone.
All other gateways were still present.

After upgrading to 24.1_1, firewall groups cannot be edited any more:
The overview doesn't show the members any more, when editing all fields are empty (no members available in the dropdown list)

Well these are the very two buttons I always check.
Temporarily disable CARP won't survive the reboot (which is just what I need), and persistent CARP maintenance state doesn't prevent CARP becoming MASTER unconditionally either, which is the very reason for this post.

Just checked on a box that never had manual fixes, but that py37-markupsafe leftover nevertheless.
pkg remove py37-markupsafe is your friend ;-)

in this case you might try to disable carp completely.

This is what I'd expect from "maintenance mode". I'm not aware of any other means in opnsense to disable carp. Digging FreeBSD docs, I found sysctl net.inet.carp.allow. So am I supposed to use that tuning or am I missing something? (some "enable/disable CARP" opnSense setting)?

Ok did a while to understand, because I was looking for the problem in the router...

To summarize, if STP is configured on the switch, it will not forward traffic for a while directly after the port is physically up; in consequence the freshly rebooted firewall won't receive CARP packets from the master and assume it's dead.

However, this doesn't explain the initial question: why does the firewall do carp at all? I want CARP disabled in the first place to prevent such very glitches.

Some days before the errors started, the switch was replaced (cabling and switch are in the provider's realm), but the switch was still silent afterwards. Errors began right after upgrade, so I wonder if the driver now can detect some error situation that it couldn't before.

Actually, I have TWO firewalls in CARP configuration. The regular error rate is on the carp master, but there are also some error on the backup.

I think I have seen STP packets, I think I can have them disabled.
But what has spanning tree to do with carp? I'd expect only proto-112 packets to have any impact (and least of all in maintenance mode).

I played with this, and apparently (as Franco stated) they're 'somehow' applied. But this means, if you change settings (tso or so), the lagg members will take the settings only on reboot from global settings, no possibility to tweak the hardware online or individually.

When doing regular maintenance on our CARP cluster, I regularly disable CARP on the machine and enter persistent maintenance mode. I'd expect it to never get MASTER until I enable CARP again.

Now, I rebooted the machine (22.1.1), and while it came up I glanced "Timeout on ix2, becoming MASTER" on the console for a second or so until it stepped back to BACKUP.
While I also have layered interfaces (vlan over lagg over 10GBit), this very ix2 interface is just a plain 1GBit onboard Intel NIC, connected to a switch, no VLAN no whistles or bells (upstream internet).

Having double master even for fractions of a second will screw up network traffic more or less badly, so this really isn't good and shouldn't happen, maintenance mode or not.

So how to safely reboot a router without triggering major trouble?