Messages

1

21.1 Production Series / Re: Having some UPnP issues.

« on: April 08, 2021, 06:58:21 am »

Hi all,

I have a LOT of experience with this.  Are both the OPNSense box & the other device on the same switch?  Are they on the same vlan?  Can they ping each other directly without going through a router?

Well, yes, sort-of.

all of my devices are on a single VLAN. I have a Ubiquiti EdgeSwitch 24-PoE as my "core", and in two locations I have Ubiquiti Unifi switches (an 8 port and a 16 port PoE). I have IGMP snooping disabled on the EdgeSwitch, and have not enabled it in the Unifi controller in the "Network" section, so everything should be getting multicast fllood. All systems that would normally be using UPnP are hardwired.

I did just realize that I had some level of IGMP filtering happening on my Unifi APs ("Multicast Enhancement" was enabled on the wireless network), which would partially explain why some of my testing was failing (laptop is on wifi)... but it doesn't explain why the Xbox wasn't working (which is hardwired).

However, there is one other piece in the game I often forget about... I have a "SamKnows" bandwidth testing box on my LAN at home. It's basically a small linux box that sits behind my opnsense box and it just bridges all traffic through it, and then runs internet speed tests when it detects enough of an "idle window" in the traffic flow. It's supposed to be a transparent bridge, although I've discovered that it filters 802.1q (i.e. it chokes on vlan tagging). Multicast had been working fine in the past though (UPnP has not been a problem before.)

I wouldn't be entirely surprised if something changed and it started filtering multicast though.  That said, after fixing the filtering on the wifi, I now can use the Windows UPnP test app that thecodemonk mentioned, and I'm getting responses from opnsense. So it doesn't seem like multicast is entirely blocked, at least. More testing needed to see if it will allow me to actually map a port though...

Win10 (in a VMware VM, using bridged networking on my Mac) with this UPnPTest app can map a port. 

The Mac itself cannot map a port or even get a response from opnsense via UPnP (although NAT-PMP works). I even pulled down the "upnpc" client  (which is part of the miniupnpd project, same thing used on opnsense) with homebrew, and it isn't even getting an answer from opnsense... which makes me wonder if Apple is doing something with multicast on recent versions of MacOS. I don't really care much about the Mac though, the xboxes are what need to work.

I need to give an xbox another try and see if it magically starts working now... I definitely have IGMP snooping disabled on the Unifi switch it is uplinked to, so it should be getting flooded, but it's probably worth verifying that with wireshark because Unifi gear is known to often do things you don't expect.

2

21.1 Production Series / Re: 1-to-1 NAT confusion OPNSense 21.1.4

« on: April 06, 2021, 02:03:47 am »

This is pfSense documentation, but AFAIK since OPNsense is still based on the same guts, the order of operations are the same...

https://docs.netgate.com/pfsense/en/latest/nat/process-order.html

3

21.1 Production Series / Re: OPNsense 21.1 hanging at /boot/defaults/loader.conf

« on: April 06, 2021, 01:54:08 am »

I think you'd be able to feed the various loader.conf variables to the "loader" prompt, but you wouldn't be able to edit /etc/ttys from there (AFAIK).

So I think it would boot and give you all of the kernel boot output at that point, but the second it passed to userland and where you'd normally get a screen or login prompt, you'd have nothing (if /etc/ttys isn't setup correctly).

I could be wrong though...

4

21.1 Production Series / Re: Having some UPnP issues.

« on: April 06, 2021, 01:47:49 am »

So I think I've come to the conclusion for me that it's Call of Duty that isn't working right, plus I think there might be a bug in the gui UPNP status.

The app I am using to test with will also display current port forwards and I have not been using it to check for them.. I've just been using the status page.

Curious, what app are you using to create the forwards (presumably on Windows)? I have a Win VM and a work laptop running Windows that I can break out to try a different app.

If I create a forward without a description, it does not display in the list. According to the specs, a description isn't required. But without a description it doesn't show in the GUI status list. I've been creating them without this whole time and seeing them not show up. Also, when testing these forwards, I haven't been closing the browser window or clearing states. I think FreshTomato must be doing that behind the scenes without telling you it is..

How is FreshTomato part of the scenario here, I thought you were running OPNsense?
You shouldn't need to clear states in the router to make a new port forward work.

Anyway. My test is by running nginx on my local PC (same one that runs Warzone/Cold War) with it's default web page. I create a forward using upnp for external port 8080 and internal port 80 to my local PC. Then on my phone, I turn off Wifi, and go to http://myexternalip:8080 and see if the page comes up. Without the forward, it doesn't come up. I then close that tab, clear all the states, then create the forward. I then go to that address on my phone and the page shows up immediately. Close the tab, remove the forward using the utility, clear all states, and open a tab and go to that address and it times out.

Yeah, so this proves that you are able to get a port to forward... if the opnsense GUI isn't showing that port mapping, that definitely seems like a bug.

I've tried that both with a description and without, and it works. Now, this was before reading about nat-pmp. So mine is on right now. I will test again with that turned off, but for now, it does look like this is working on my config (without changing any settings like xpendable has. I will most likely try his settings as well.

I don't know what utility you are using to create the mappings from your PC, but NAT-PMP is originally an Apple thing, so unless your utility explicitly has NAT-PMP support in it, it probably is just using UPnP.

I'm having problems with an XBox One X that is failing to get a port mapping via UPnP (and it has worked in the past with OPNsense), so there's definitely something wrong.  I have the same problem as you - wife will be upset if I break the internet to downgrade the router unless she's asleep, so I have to work around that. I'm also not exactly sure how to rollback OPNsense to an older release... Do I need to make a config backup and then fully reinstall from an older ISO?

5

21.1 Production Series / Re: Having some UPnP issues.

« on: April 05, 2021, 03:16:37 am »

I had some time to play around with this some more... from a Mac, using the free/open source "Port Map" app, and the Mac creates a port mapping successfully.

but, by default the Mac app is using NAT-PMP and not UPnP.

When I disable NAT-PMP in OPNsense's daemon, the mac app wants to fall back to UPnP, but it's not working. It sits at "searching" forever and things are not happy. I'm not seeing anything revealing popping up in the log on opnsense though...

6

21.1 Production Series / Re: Having some UPnP issues.

« on: April 04, 2021, 06:05:33 am »

I can confirm you’re not crazy, something is wrong with UPnP for me too (on 21.1.4).

I’d be happy to provide more info if I knew what would help.

7

21.1 Production Series / Re: Having some UPnP issues.

« on: April 03, 2021, 09:56:59 am »

And in outbound nat, I have it in hybrid mode, and a rule for source lan net with the wan address at the nat address and static port enabled.

ZPrime he already has his outbound NAT using hybride mode with a single rule with static-ports for his entire LAN network.

Sorry about that... I shouldn't be replying at 4am local time. :p

Make sure you reboot your consoles or PC. I've noticed my Xbox or PS5 won't send AddPortMapping requests after they are up and running so no port mappings will show up in OPNsense until you reboot. But this isn't always the case.

I can confirm that with Xbox at least, it sometimes doesn't want to map. What sometimes helps (and avoids a lengthy reboot) is to get into the networking section and check/test the NAT status. If it still doesn't map after that, there's a secret code (IIRC it's hold both triggers and then hit both bumpers simultaneously) that gives you a "more details" screen, and after bringing that up and then checking NAT status again, it usually seems to force the mapping.

UPnP is so great in theory, but the practice/execution is very janky at times...

8

21.1 Production Series / Re: Having some UPnP issues.

« on: April 03, 2021, 04:32:55 am »

Turn on UPnP

Then, Firewall -> NAT -> Outbound

Switch the radio buttons at the top to "Hybrid outbound NAT"
Now, either setup a rule for the entire LAN subnet, or configure your game systems/consoles to be in a sub-subnet (e.g. a /29 or /28 out of your LAN /24), and setup a rule for just that chunk... you want "Static Port = Yes" for the problem systems.

Again, this is just for outbound traffic, it insures that the firewall doesn't shuffle around the source port on the WAN side.

As long as UPnP is enabled and setup correctly, and you have static port outbound NAT for the game systems, this should take care of it.

9

21.1 Production Series / Dual-WAN - one interface misbehaving causes problems for entire router

« on: April 01, 2021, 09:25:58 am »

I am running dual WAN with two different cable ISPs. Both modems are in bridge mode, both WAN interfaces are DHCP. The two ISPs are Spectrum and WOW Cable.

I do not (currently) attempt any load balancing; I just have two different gateway groups that are setup for failover, with one connection preferred over the other. However, I also have a few rules in place to policy-route traffic from specific hosts out each of the connections - these hosts are "monitor-io" monitoring devices, so I can get "user friendly" visual status indication of problems with one provider or the other. (The monitor-io is a really cool device to give easy at-a-glance status of your uplink, I use them at home and one at my parents' house... but it's offtopic here.) Important to this story: I currently have WOW set as the primary connection, and Spectrum is the secondary.

Yesterday, Spectrum was having significant problems; packet loss was through the roof, and latency was very high as well. dpinger was constantly toggling the connection between "alarm" and "clear" states every minute or two. (I have all of the interval / timer settings for gateway monitoring at the defaults.)

It seemed like every time the Spectrum gateway went from bad to good status, something was being reloaded on opnsense, and it was causing traffic interruptions for management sessions to the firewall, as well as causing a fair amount of delay on traffic going out to the (still functional) Whiskey ISP connection.  I also noticed (via SSH log output, before I'd get kicked off) that the "nut" service (UPS monitoring) seems to get restarted each time this "something" gets reloaded, too.

In order to get everything to a usable state again, I had to disable the gateway for Sierra (clicked the enable/disable toggle to the left of the entry, by the checkbox, then applied).

After disabling the gateway for Sierra, policy routing was not behaving like it should. One of my policy routes is supposed to send one of the two monitor-io devices out the Sierra connection, and only that connection... but it was not working. The device that should've been monitoring Sierra was showing "all good," even though I had manually disabled that gateway.

So, my two main questions:

• What exactly is supposed to happen when a gateway goes from online to warning / bad due to packetloss or latency? Should it be interrupting nut? Should it be affecting SSH sessions to the firewall/router itself? Any idea why it was impacting other traffic flow through the firewall via the remaining good WAN link?
• When a gateway is manually disabled (not "marked as always down," but fully disabled), should policy routing still be working against that gateway? My intent is for certain traffic to just be dropped on the floor in this state, but it wasn't happening.

The hardware in play here is not exactly new - quad-core Atom C2558 on a Supermicro A1SAi, quad Intel igb interfaces, and plenty of RAM (32GB ECC). Beyond nut and UPnP, I don't have anything special on the system; it's a pretty basic firewall (other than dual WAN and a few policy routing rules). I did not notice significant load spikes (even when dpinger was changing the gateway state)... CPU would blip a little, but it wasn't sitting at full load or anything. Load average was no higher than 1 or 1.5 on the shorter end, the longer timeframes were obviously lower.

I'm happy to share my rules or config if someone can give me a suggestion on the best way to do that while anonymizing passwords / etc.

10

20.7 Legacy Series / Re: Dynamic DNS can't register - OpenSSL error with curl?

« on: October 20, 2020, 07:52:07 am »

Anybody?
What do I need to file a bug against this problem?

11

20.7 Legacy Series / Re: Gateway in Subnet

« on: October 06, 2020, 03:50:52 am »

look up ICMP redirects. your OPNsense box is probably telling the "Server" hosts "hey, you don't need to talk to me, you can talk to this other router directly."

https://www.cisco.com/c/en/us/support/docs/ios-nx-os-software/nx-os-software/213841-understanding-icmp-redirect-messages.html

12

20.7 Legacy Series / Re: Firewall initially allows, then denies, connections between two internal subnets

« on: October 06, 2020, 03:00:24 am »

Could use clarification here -
is the OPNsense box the one that is in both 192.168.35.0/24 and .36.0/24?

Or is there some other system involved that's on both .35 and .36, and the OPNsense box is only on .35? If the OPNsense box is not in both subnets, how is the other system ("a single host") configured to be a gateway? Is it doing routing, or bridging?

And are these different subnets in two separate L2 segments (i.e. different VLANs / separate switches), or on one switch?

13

20.7 Legacy Series / Re: Dynamic DNS can't register - OpenSSL error with curl?

« on: October 06, 2020, 02:49:56 am »

Talking to myself... I was able to work-around this using OpenDNS's "DNS-O-Matic" service and having it update this hostname at Namecheap.

But there's definitely something wrong, since I could do two separate hostnames using pfSense and OPNsense is failing the same task. Is there a way to get more debug information so I can file a bug? The repro steps are pretty easy, other than requiring a domain at Namecheap...

14

20.7 Legacy Series / Dynamic DNS can't register - OpenSSL error with curl?

« on: October 04, 2020, 10:14:41 pm »

I have a dual WAN setup at home, using WOW cable and Spectrum cable.

Right now, WOW is setup with a lower priority (under Gateways > Single), so OPNsense itself uses WOW as the "default route."

I have Dynamic DNS configured for both interfaces, with two different hostnames (wan-wow and wan-spectrum), but both are on the same domain at Namecheap (so they're both using the same update password, too).

The hostname for the WOW connection updates correctly.

The hostname for the Spectrum connection is throwing errors when it tries to update, like so (the ".example.com" is not my actual domain name of course):

Code: [Select]

2020-10-04T15:47:13 opnsense[59877] /services_dyndns_edit.php: Curl error occurred: OpenSSL SSL_connect: Connection reset by peer in connection to dynamicdns.park-your-domain.com:443
2020-10-04T15:47:13 opnsense[59877] /services_dyndns_edit.php: Dynamic DNS (wan-spectrum.example.com): Current Service: namecheap
2020-10-04T15:47:13 opnsense[59877] /services_dyndns_edit.php: Dynamic DNS (wan-spectrum.example.com): _checkStatus() starting.
2020-10-04T15:47:03 opnsense[59877] /services_dyndns_edit.php: Dynamic DNS (wan-spectrum.example.com via Namecheap): _update() starting.
2020-10-04T15:47:03 opnsense[59877] /services_dyndns_edit.php: Dynamic DNS (wan-spectrum.example.com): running dyndns_failover_interface for wan. found igb3
2020-10-04T15:47:03 opnsense[59877] /services_dyndns_edit.php: Dynamic DNS (wan-spectrum.example.com): 173.91.x.y extracted
2020-10-04T15:47:03 opnsense[59877] /services_dyndns_edit.php: Dynamic DNS: updatedns() starting

Any suggestions? I already enabled the "Verbose logging" option for this hostname, and that's the output above. When it's not Verbose, there are fewer lines, but the same error.

This identical setup worked correctly on pfSense, both hostnames (using Namecheap and the same base domain) were updated as expected.

I'm running 20.7.3 OpenSSL version.