Messages

Like I said above, Gateway switching is enabled.

Ok so it didn't quite work.

I just had an outage where the uplink on the primary circuit (priority 1) was effectively dead: the CableModem still had an IP (and as such so did the firewall), but the probe-IP configured (a known point beyond the CM, within the provider's network) was unreachable. The Gateway was even recognized as down.

But the default gateway did not switch to the secondary (priority 2).

Both the primary and secondary individual gateways are the only gateways marked as "upstream".  All other gateways are VPNs, and are priority 255 (for obvious reasons - if there's no base circuit, there can be no VPN over them). Gateway switching is indeed enabled in System settings.

I'll run a more controlled test tomorrow/weekend. For now, I'm sad to report that the feature does not appear to be working.

I'm fully updated, btw (OPNsense 19.7.2-amd64, FreeBSD 11.2-RELEASE-p12-HBSD, OpenSSL 1.0.2s 28 May 2019).

Cheers!

I'm interested in failover and preference: i.e. use circuit A if it's available, if not then use B, but if A comes back, go back to A.

That doesn't work with "upstream" because opnSense will just choose one, without consideration for preference. At least, that's how it did for me. Perhaps there's another setting that I'm ignoring that helps resolve this?

Cheers!

Hi!  I've posted about this before (https://forum.opnsense.org/index.php?topic=11497.msg52045#msg52045).  The issue is still there: on a prolonged outage for the primary circuit (Cable), every so often the firewall's default gateway will simply get nulled out (i.e. set to "nothing") even though the secondary circuit (ADSL) is up and running.

The "workaround" is to log into the UI, open the ADSL gateway's configuration, save it (no changes!!), and then click on "Apply Changes". This causes the ADSL link to be selected as the default gateway.  But then again, a few minutes later, the same thing happens again (default gateway gets de-configured), and off we go again to the workaround...

Here are some configuration tidbits:

• There are 4 gateways in the system: Cable, ADSL (these are physical interfaces), VPN1 and VPN2 (these are "soft" interfaces - OpenVPN the both)
• I added all gateways to the same group, with Cable as tier 1, ADSL as tier 2, and the VPN gateways as tier 5
• The VPN interfaces are configured with "Mark Gateway as Down", precisely so they won't be promoted to primary (not that it matters if both Cable & ADSL are down)
• Both Cable and ADSL have explicit monitoring IPs set, in order to validate if the link is really up, vs just the interface is up (frequent case when Cable goes out is that the interface remains in the UP state, even though the actual link is down)
• All gateways are set for DHCP on IPv4
• NONE of the gateways is configured with "Disable Gateway Monitoring" as this will (erroneously, if you ask me) override "Mark Gateway as Down" and cause the gateway to be marked as UP even if you don't want it to

Basically, I have everything configured like the "textbooks" say I should have it, and yet I can't get it to work the way (I think) it should.  The problem seems to be with dpinger (or related processes), since if I change the VPN gateways to "Disable Gateway Monitoring" (i.e. assume they're always UP), then for some inexplicable reason they will be preferred ahead of the ADSL link as gateway, even though the ADSL link is in a higher tier within the same gateway group...!!!

Can someone please help me figure this out?

Thanks!

That github issue appears to be the same one, so I think I'm good for now with the workaround (don't clear states when IPs change), and will be fine with 19.1.2.

Thanks!

Hi!

I'm experiencing many sudden and unexplained connection interruptions.  Suddenly, and without explanation, the firewall will even reset an SSH connection I have running on it displaying traffic capture over a LAN link!!!

I'm not sure where to begin to offer up log information.  It feels as though the firewall is resetting states periodically. Is there any logging feature that enters a message in the logs when firewall states are reset (and how do I enable it)? Maybe that can be the first step to figuring this out...

Thanks!

[unless I explicitly mark the Cable gateway as the default gateway]

Hi, all!

I'm experiencing some issues on the failover.  I had a power outage last night and one of my ISPs (the Cable provider) seems to be down due to some line damage, and they'll be offline for a few hours. This happened in the wee hours of the morning, while I was asleep.

When I woke up, I found that I had no internet service. This means that failover had once again been unsuccessful.

After some poking around, I've discovered that even though I have a failover gateway group set up for both my ISPs (Cable and ADSL), and the group is (apparently) configured properly (Cable is Tier 1, ADSL is Tier 2, Trigger Level is "Member Down"), the failover algorithm will not work as expected.

This is the behavior that I would expect:

* When the Cable link is up, the Cable link is promoted to default gateway
* When the Cable link is down, the ADSL link is promoted to default geteway
* When the Cable link comes back up, the Cable link is promoted to default gateway irrespective of the ADLS link's status

This is the behavior I'm seeing:

* When the Cable link is up, the Cable link is promoted to default gateway
* When the Cable link is down, the ADSL link is promoted to default geteway
* When the Cable link comes back up, the ADSL link remains as default gateway unless I explicitly mark the Cable gateway as the default gateway
* However, when I mark the Cable gateway as the default gateway, and a prolonged outage occurs, it seems that fail-back simply won't happen at all and it will remain as the default gateway regardless of up/down status

I've configured each gateway (Cable + ADSL) to have a Monitor IP setting for an address on the far side of the link, so it can be used to determine if the link really is up vs. appearing to be up. I've noticed that even though there's an outage, the Cable link's status shows as "pending" vs. "down". Perhaps this is the issue? Perhaps the algorithm is assuming that the link is up because it's not explicitly marked as "down"?

Thoughts?

If that's the case, then definitely the algorithm should compare the link's current status (UP, UP+Latency, UP+Packet Loss, UP+Latency+Packet Loss, Pending, Down) vs. the "Trigger Level" condition set up in the gateway group, taking into account that until a link is in UP or UP+* state, it should be considered to be down? (i.e. Down + Pending should be equivalent, I think)...

Thanks!

[destination]

Hi!

I'm trying to do something that on the surface appears simple enough, but I'm having a hell of a time configuring. I have a VPN through which I want to route all traffic that the firewall detects going to a specific port number (say...8888).  Simple enough, and done easily enough.

The trick is that I also want that outgoing traffic's destination port number to change from "8888" to "7777", without touching the destination address... and this is where I'm having a hell of a time.

The Outbound rules UI allows me to change the source port, the source address, the destination address... but not the destination port.

I also can't use a port forward since I don't know the request's possible destination beforehand and the port forwarding UI (at least) requires a destination IP to forward the traffic to...

I refuse to believe this "simple" (right?) functionality is impossible in OPNSense, and instead I prefer to think that the issue is that I'm uneducated/incompetent in OPNSense management and configuration, since that seems to me the more likely scenario. :D

So.... any ideas?

Thanks!

I think I've found the issue.  If I set any destination address as part of the selector for the NAT rule, the NAT rule won't be generated. If I leave the destination address as "any", the rule is generated just fine.

This seems like a bug to me: if destinations aren't supported as part of the rule selector, then one shouldn't be able to set them via the GUI.  If one is able to set them via the GUI, then the rule generator should generate the NAT rules properly.

So - it's either a bug in the rule generator (not applying the destination specification to the rule's "to ..." selector), or a bug in the GUI permitting rule configurations that aren't allowed.

This is on 18.7.8, fully updated.

Thoughts?

Cheers!

More details I left out about the manual rules I added (I posted in a hurry, sorry :D):

• The interface the packets will be outbound on is an OpenVPN client interface (already assigned a static name, and marked as "non-removable")
• The OpenVPN connection is coming up fine, and appears to be working fine

Regardless of what I do, I can't get the rule generator to create those rules. Or, at least, they're not being listed when using

Code Select Expand

pfctl -sa.

Cheers!

Hi!

The description for what "Hybrid outbound NAT rule generation" does is as follows: Automatically generated rules are applied after manual rules

However, I added some manual rules that I've confirmed aren't being added accordingly.  Adding and removing the rules has no effect: using pfctl -sa produces the same NAT rule output each time.

I don't want to switch to fully manual rule generation if I can avoid it, so I can leverage the system's automatic rules.

Is this a known issue? Perhaps there's a misconfiguration somewhere else tripping me up?

Thoughts?

Thanks!

I had to manually add the push rules for dhcp-option DNS and dhcp-option DOMAIN to get it working.

You'll have to look into the OpenVPN manual to see what those mean ;)

But this can be worked around.

Read the original post fully. A workaround is described.

Cheers!

Hi, all!

For my setup, I have several OpenVPN links going to-and-fro.  I managed to get everything working in a pretty clean manner, but have found one inconsistency that I thought could bear some discussion.

Out of preference, and b/c the tunnels I have are pretty much constantly up, I decided to assign a static interface to each of the tunnels so their access rules would be easier to manage.  That worked as expected, minus a speedbump when I realized that the OpenVPN tunnels had to be bounced after all the configuration was done. Minor setback, but easily resolved. Moving on...

What I discovered is that the rules for "OpenVPN" are evaluated *before* the rules for each of the individual tunnels. Intuitively, I would have thought that the most specific rules groups are always evaluated first, but this doesn't appear to be the case here.

Is this by design? Is this a defect that needs correcting?

I discovered this when I added what I hoped to be a "catch-all" REJECT rule, for easier debugging, and found that it was the cause of the traffic not flowing. As soon as the rule was disabled (later removed), everything worked as it should.

So... thoughts?

[entire]

Right. I understand why you wouldn't auto-install stuff, and that's definitely NOT what I'm suggesting.

However, you could give the user the option to accelerate restoring that bit, i.e. "Do you also want to update the installation as described in this configuration?" (and describe the updates), as well as "Please select which plugins you wish to restore" (and present a list of the plugins selected from which the user might untick the ones he doesn't want to restore)...

Optionally, once restored and on reboot (with all the above done), also compare the list of packages installed to the list of packages included in the restored configuration and tell the user "The following packages which you installed manually are still missing, do you wish for them to be installed now?"

Does that sound more sensible? I guess what I'm looking for is a means to restore the entire firewall to its backed-up state, to make it as painless as possible if I'm starting from scratch with fresh hardware (for example). I realize there are modifications users can do which would make this more difficult, but we shouldn't anticipate/account for those just yet. For now, something like the above would do very nicely.

Later on we can decide if we add a mechanism that actually tracks filesystem-level deviations from the base install and backs those up (or at least documents them for the user).