Topics

1

21.1 Production Series / Dual-WAN - one interface misbehaving causes problems for entire router

« on: April 01, 2021, 09:25:58 am »

I am running dual WAN with two different cable ISPs. Both modems are in bridge mode, both WAN interfaces are DHCP. The two ISPs are Spectrum and WOW Cable.

I do not (currently) attempt any load balancing; I just have two different gateway groups that are setup for failover, with one connection preferred over the other. However, I also have a few rules in place to policy-route traffic from specific hosts out each of the connections - these hosts are "monitor-io" monitoring devices, so I can get "user friendly" visual status indication of problems with one provider or the other. (The monitor-io is a really cool device to give easy at-a-glance status of your uplink, I use them at home and one at my parents' house... but it's offtopic here.) Important to this story: I currently have WOW set as the primary connection, and Spectrum is the secondary.

Yesterday, Spectrum was having significant problems; packet loss was through the roof, and latency was very high as well. dpinger was constantly toggling the connection between "alarm" and "clear" states every minute or two. (I have all of the interval / timer settings for gateway monitoring at the defaults.)

It seemed like every time the Spectrum gateway went from bad to good status, something was being reloaded on opnsense, and it was causing traffic interruptions for management sessions to the firewall, as well as causing a fair amount of delay on traffic going out to the (still functional) Whiskey ISP connection.  I also noticed (via SSH log output, before I'd get kicked off) that the "nut" service (UPS monitoring) seems to get restarted each time this "something" gets reloaded, too.

In order to get everything to a usable state again, I had to disable the gateway for Sierra (clicked the enable/disable toggle to the left of the entry, by the checkbox, then applied).

After disabling the gateway for Sierra, policy routing was not behaving like it should. One of my policy routes is supposed to send one of the two monitor-io devices out the Sierra connection, and only that connection... but it was not working. The device that should've been monitoring Sierra was showing "all good," even though I had manually disabled that gateway.

So, my two main questions:

• What exactly is supposed to happen when a gateway goes from online to warning / bad due to packetloss or latency? Should it be interrupting nut? Should it be affecting SSH sessions to the firewall/router itself? Any idea why it was impacting other traffic flow through the firewall via the remaining good WAN link?
• When a gateway is manually disabled (not "marked as always down," but fully disabled), should policy routing still be working against that gateway? My intent is for certain traffic to just be dropped on the floor in this state, but it wasn't happening.

The hardware in play here is not exactly new - quad-core Atom C2558 on a Supermicro A1SAi, quad Intel igb interfaces, and plenty of RAM (32GB ECC). Beyond nut and UPnP, I don't have anything special on the system; it's a pretty basic firewall (other than dual WAN and a few policy routing rules). I did not notice significant load spikes (even when dpinger was changing the gateway state)... CPU would blip a little, but it wasn't sitting at full load or anything. Load average was no higher than 1 or 1.5 on the shorter end, the longer timeframes were obviously lower.

I'm happy to share my rules or config if someone can give me a suggestion on the best way to do that while anonymizing passwords / etc.

2

20.7 Legacy Series / Dynamic DNS can't register - OpenSSL error with curl?

« on: October 04, 2020, 10:14:41 pm »

I have a dual WAN setup at home, using WOW cable and Spectrum cable.

Right now, WOW is setup with a lower priority (under Gateways > Single), so OPNsense itself uses WOW as the "default route."

I have Dynamic DNS configured for both interfaces, with two different hostnames (wan-wow and wan-spectrum), but both are on the same domain at Namecheap (so they're both using the same update password, too).

The hostname for the WOW connection updates correctly.

The hostname for the Spectrum connection is throwing errors when it tries to update, like so (the ".example.com" is not my actual domain name of course):

Code: [Select]

2020-10-04T15:47:13 opnsense[59877] /services_dyndns_edit.php: Curl error occurred: OpenSSL SSL_connect: Connection reset by peer in connection to dynamicdns.park-your-domain.com:443
2020-10-04T15:47:13 opnsense[59877] /services_dyndns_edit.php: Dynamic DNS (wan-spectrum.example.com): Current Service: namecheap
2020-10-04T15:47:13 opnsense[59877] /services_dyndns_edit.php: Dynamic DNS (wan-spectrum.example.com): _checkStatus() starting.
2020-10-04T15:47:03 opnsense[59877] /services_dyndns_edit.php: Dynamic DNS (wan-spectrum.example.com via Namecheap): _update() starting.
2020-10-04T15:47:03 opnsense[59877] /services_dyndns_edit.php: Dynamic DNS (wan-spectrum.example.com): running dyndns_failover_interface for wan. found igb3
2020-10-04T15:47:03 opnsense[59877] /services_dyndns_edit.php: Dynamic DNS (wan-spectrum.example.com): 173.91.x.y extracted
2020-10-04T15:47:03 opnsense[59877] /services_dyndns_edit.php: Dynamic DNS: updatedns() starting

Any suggestions? I already enabled the "Verbose logging" option for this hostname, and that's the output above. When it's not Verbose, there are fewer lines, but the same error.

This identical setup worked correctly on pfSense, both hostnames (using Namecheap and the same base domain) were updated as expected.

I'm running 20.7.3 OpenSSL version.