miniupnpd is working fine, but I noticed the increasing number of duplicate entries in the status (Services: Universal Plug and Play: Status).
Example:
Port Protocol Internal IP Int. Port Description
62852 tcp 10.0.0.87 6690 upnpclient:6690
62852 tcp 10.0.0.87 6690 upnpclient:6690
62852 tcp 10.0.0.87 6690 upnpclient:6690
62852 tcp 10.0.0.87 6690 upnpclient:6690
62852 tcp 10.0.0.87 6690 upnpclient:6690
62852 tcp 10.0.0.87 6690 upnpclient:6690
9308 udp 10.0.0.149 9308 10.0.0.149:9308 to 9308 (UDP)
9308 udp 10.0.0.149 9308 10.0.0.149:9308 to 9308 (UDP)
9308 udp 10.0.0.149 9308 10.0.0.149:9308 to 9308 (UDP)
9308 udp 10.0.0.149 9308 10.0.0.149:9308 to 9308 (UDP)
9308 udp 10.0.0.149 9308 10.0.0.149:9308 to 9308 (UDP)
9308 udp 10.0.0.149 9308 10.0.0.149:9308 to 9308 (UDP)
9308 udp 10.0.0.149 9308 10.0.0.149:9308 to 9308 (UDP)
9308 udp 10.0.0.149 9308 10.0.0.149:9308 to 9308 (UDP)
9308 udp 10.0.0.149 9308 10.0.0.149:9308 to 9308 (UDP)
9308 udp 10.0.0.149 9308 10.0.0.149:9308 to 9308 (UDP)
9308 udp 10.0.0.149 9308 10.0.0.149:9308 to 9308 (UDP)
9308 udp 10.0.0.149 9308 10.0.0.149:9308 to 9308 (UDP)
They stick there forever, never expire, and the list grows over time.
For example, 10.0.0.149 is my PS5. 10.0.0.87 is Synology NAS, and there is already a port forward rule for 6690 (Cloud Station).
Is it expected? If not, why it happens and should I do something about it? I know that explicit port forwards are recommended, but in case I want to keep using miniupnpd, should I be worried about these duplicates and possible resource leaks, or maybe it's just a visual representation bug?
EDIT:
I found some more details which may explain why it happens.
Using command line miniupnpc 2.2.6 and other upnp clients (like https://github.com/kaklakariada/portmapper), listing the existing mappings shows corrupted results:
i protocol exPort->inAddr:inPort description remoteHost leaseTime
0 UDP 0->10.0.0.87:0 '' '34.26.0.0' 0
1 UDP 0->10.0.0.87:0 '' '34.26.0.0' 0
2 UDP 0->10.0.0.87:0 '' '34.26.0.0' 0
(https://i.imgur.com/yfbV80J.png)
Instead of these:
62852 tcp 10.0.0.87 6690 upnpclient:6690
62852 tcp 10.0.0.87 6690 upnpclient:6690
62852 tcp 10.0.0.87 6690 upnpclient:6690
It could be that PS5 and other clients get the same corrupted results, don't see the mapping they already created and try to add a new one which produces duplicates (and probably also cannot delete a mapping).
The question is why miniupnpd 2.3.3_2,1 server on opnsense returns these corrupted mappings.
After reading this I check and I'm seeing the same growing list happening. Not sure it's causing any issues, but it's concerning for sure.
I'm seeing this too. Do you happen to have NAT-PMP checked as well in your UPnP settings?
Quote from: AhnHEL on February 19, 2024, 05:32:02 AM
I'm seeing this too. Do you happen to have NAT-PMP checked as well in your UPnP settings?
I did have that turned on. I've disabled it for now, not sure it would matter, but here's hoping!
Disabling Allow PCP/NAT-PMP Port Mapping setting doesn't help. I can still see duplicate entries and the client reads them incorrectly.
Quote from: JetSerge on February 21, 2024, 05:34:44 PM
Disabling Allow PCP/NAT-PMP Port Mapping setting doesn't help. I can still see duplicate entries and the client reads them incorrectly.
Confirmed. I thought at first it was working, but nah, back to a bunch of duplicate entries.
Found a related bug report: https://github.com/opnsense/plugins/issues/3831.
I nudged the upstream bug report but to be frank a lot of the pfSense/libpfctl work has been a train wreck hit and run on both the src and ports infrastructure of FreeBSD.
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277226
Since the author ignores reports that originate from OPNsense just because he can don't expect this to be fixed any time soon. I've talked through all the FreeBSD ranks and nobody can/wants to do anything about it.
Cheers,
Franco
Quote from: franco on February 23, 2024, 09:35:09 AM
Since the author ignores reports that originate from OPNsense just because he can don't expect this to be fixed any time soon. I've talked through all the FreeBSD ranks and nobody can/wants to do anything about it.
Thanks!
Maybe it's possible to use miniupnpd 2.3.3_1 instead of 2.3.3_2 somehow?
The pkg you suggested on GitHub from 23.7 version didn't work due to openssl dependency change.
Quote from: franco on February 23, 2024, 09:35:09 AM
I nudged the upstream bug report but to be frank a lot of the pfSense/libpfctl work has been a train wreck hit and run on both the src and ports infrastructure of FreeBSD.
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277226
Since the author ignores reports that originate from OPNsense just because he can don't expect this to be fixed any time soon. I've talked through all the FreeBSD ranks and nobody can/wants to do anything about it.
Cheers,
Franco
This is just sad.... Still holding a grudge because somebody took his milkyway in kindergarden.
Regards,
S.
Doesn't look like grudge to me but more like maintainer has gone missing, disinterested, whatever ...
If you look at the patch that introduced the bug, that one was created by Kristof Provost and not my the miniupnpd maintainer. And this patch already was "approved" by maintainer timeout.
So more an orphaned port than anything else. Possibly best to nudge Kristof about it - if he can find the time.
Kind regards
Patrick
> Maybe it's possible to use miniupnpd 2.3.3_1 instead of 2.3.3_2 somehow?
Yes, but after a very nasty bug I had to fix myself on stable/13 after him saying "we should fix our own stuff" I'm pretty confident he should fix his stuff, too.
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273207#c18
> Supporting opnsense is your job, not mine. You don't get to just throw bugs over the wall without doing any actual testing on freebsd.
And this to a bug he himself added... And then we had this which went into production causing quite a stir and a whole night overtime for me working it out...
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=275006
I'm done enabling others to continue do sub-par work while complaining about others. :)
Cheers,
Franco
I see your point :)
Actually I mentioned this elsewhere and asked why we can't have an option like LIBPFCTL which can be on by default but easily disabled to unbreak stuff it that happens. Because this also happened on pftop if anyone remembers.
Cheers,
Franco
This issue appears to have been resolved upstream by the miniupnp devs with version 2.3.5
https://github.com/miniupnp/miniupnp/issues/719
I'm not familiar with how freebsd ports work... how do you get the maintainer of the freebsd port to update from source?
I tried to update to 2.3.6 but of course the custom patching in FreeBSD ports for pf breaks the build of the new version. I guess this is as good as it gets. ;)
Cheers,
Franco
Well I created a freebsd bugzilla account and commented on bug 277226. Not sure if it will make a difference, but I figure the more noise that is made... the more likely it will be fixed :)
Appreciate your help!
Hey guys,
I'm quite new to OPNsense and my setup with three PC:s and two Xbox consoles depends on UPNP to work, especially Call of Duty: Warzone.
Why can't the new version 2.3.6 that resolves the problem introduced with OPNsense 24.1 be included in easily so we can update the plugin and get a working UPNP-function again? I believe I don't understand the dependencies between FreeBSD versions, miniupnd versions and how they are implemented within OPNsense?
Please try to oversee my lack of knowledge on the internals and try to give a easy to understand explanation.
FreeBSD introduced custom code patches into their port that now requires rewriting.
Cheers,
Franco
Is it in OPNsense or miniupnp the rewriting needs to be done? Is there any preliminary time plan when this can be implemented?
OPNsense uses the FreeBSD port of miniupnpd, it is the FreeBSD port maintainers that need to update the package.
Link to the FreeBSD port of miniupnpd:
https://www.freshports.org/net/miniupnpd/
Link to the FreeBSD bugzilla for this issue:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277226
I have also bumped this issue at the FreeBSD bugzilla. Is there anything else we can do to push to get this issue resolved as soon as possible?
Does this not affect anyone using vanilla FreeBSD? Seems to be no incentive in updating the port even though miniupnpd has been updated 3 times since the last port was updated a year ago.
I assume the only prominent consumers are pfSense and OPNsense. Everybody running a simple FreeBSD setup can work around it ;)
Is this the same problem in pfSense? Seems Mr. Provost is pretty active over there, hmmmmm.
https://redmine.pfsense.org/issues/15470
Seems like the same problem to me but I could be wrong? If it's fixed in pfSense why can't it be fixed in OPNsense?
You mean fixed in pfSense only in a "FreeBSD-ports" repository by a FreeBSD committer?
"I've updated miniupnpd to the latest version and adjusted the libpfctl patch in https://gitlab.netgate.com/pfSense/FreeBSD-ports/-/commit/6e7d96166c051915155356546474a1c6e68cf2aa
That fixes the lack of expiring entries."
As of right now I don't see it in the actual FreeBSD ports and the FreeBSD ports upstream report is unattended. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277226
Apparently you can find the patch here ;)
https://github.com/pfsense/FreeBSD-ports/commit/6e7d96166c051
What makes me wonder is why pfSense only saw this bug 20 days ago and fixed it in 2 days, but the upstream report has been open since February. You'd think a number of pfSense users would have been affected by that more than 20 days ago. That there doesn't seem to be incentive to hurry the fix into FreeBSD ports is just icing on the cake. Yummy. :)
Cheers,
Franco
Ya seems to be some favoritism here, I left a new comment on the FreeBSD bugzilla questioning this exact thing. Probably end up getting deleted if they really are playing favorites ???
I can throw the patch into bugzilla and call maintainer timeout later this evening, but I have the feeling we're still missing a maintainer who will commit (and/or not ignore) this.
Think of all the reports of users suffering from this problem so far. If it weren't for the libpfctl complications that were introduced by the party who finally fixed it I'd have submitted an update for 2.3.6 already. You can see this was a non-trivial binary change from the patch and has no relation to upstream in the first place.
I'm even struggling to explain the magnitude of the situation bordering to the unbelievable, which would still have been solvable if the author had cared a month or two ago.
Cheers,
Franco
It does seem that this miniupnpd package is modified a fair bit from the upstream version from the original developer. I think maybe calling it a port is not really accurate, probably closer to say that it's a fork from the main developer.
Which blurs the lines a bit, but either way I do think that the FreeBSD maintainers seem to be lacking on maintaining their packages, or at least this one in particular.
I'm sure things will get sorted out eventually, it's just a shame on how this bug was handle by the FreeBSD maintainers. Notablely the individual that caused the issue in the past commit and then ultimately fixed it... although in a completely different repository :(
We will get this sorted. I published the verbatim patch there (it applies cleanly) and asked for assistance.
Cheers,
Franco
Please keep us updated on all the upcoming details. I got my popcorn ready. ;)
@franco, is it not possible to have the new version of miniupnp as a Hotfix as a temporary solution?
No, I'm not going to touch this mess. libctl was introduced to the FreeBSD port in October last year. If the maintainer of libpfctl wants to maintain it he can, but so far he doesn't seem to be interested. I'm not going to spend more time on it than I already have.
Cheers,
Franco
@franco, to be honest it's really hard to understand who is responsible for what?? :o
Who should we try to put some pressure on to get this mess sorted?
Ok so it was actually November 2023 when this was introduced:
https://github.com/freebsd/freebsd-ports/commit/81e8bb983432251
libpfctl use was added via custom patches not found in upstream.
So now when we try to update to upstream 2.3.6 we have to deal with breakage in that custom scripting and I don't see why anyone else than the author should deal with this.
You can also see nobody cared to add 2.3.4 and 2.3.5 in the meantime.
Now the FreeBSD bug report exists since February 2024. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277226
The maintainer of libpfctl has not cared to work on it most likely because the report comes from OPNsense and as per his view we should fix our own problems.
Now in May 2024 this was also reported to pfSense proving that is not an OPNsense issue. It's a FreeBSD issue. https://redmine.pfsense.org/issues/15470
It was fixed within two days with the patch here https://github.com/pfsense/FreeBSD-ports/commit/6e7d96166c051
Apparently that is the same author from November who said we should work on our own issues... updating miniupnpd and adjusting the libpfctl code accordingly.
But that code never went to FreeBSD ports where it belongs. Incidentally the author is on CC in the February issue raised in FreeBSD.
I have appended his patch to the FreeBSD ticket. The standard procedure is that someone with a commit bit will bring it into the tree. I can't do more than this.
I also believe that I shouldn't have to chase an issue for a couple of months only to find out the author doesn't even want to bring his patch to FreeBSD ports and/or isn't interested in maintaining miniupnpd after making it unmaintainable for 99% of the contributors by adding libpfctl custom patches instead of working with upstream to integrate them.
I've fixed several libpfctl issues in FreeBSD base in the past that have negatively impacted our production releases because they were added to FreeBSD 13 stable branch hastily.
I know some people don't see it this way but intentionally harming OPNsense and effectively also FreeBSD with a power play over who can do what commit in FreeBSD is not a good idea.
In closing: the ball is in the court of FreeBSD now. A bug report was raised and a seemingly appropriate patch was posted. Somebody with a commit bit needs to commit it. It's the standard procedure, but it should (and could) have been taking care of weeks ago in my opinion.
Cheers,
Franco
FYI, It looks like the FreeBSD maintainer that caused all this is trying to work with the upstream dev.
https://github.com/miniupnp/miniupnp/pull/671
However progress only just picked back up again by the looks of it.
Fair enough. Doesn't help to rectify the situation of the past 3 months though.
It has been committed to FreeBSD ports now.
Cheers,
Franco
# opnsense-revert -z miniupnpd
Final confirmation would be good. Obviously thanks to all involved in making this happen.
Quote from: franco on May 29, 2024, 05:59:26 PM
# opnsense-revert -z miniupnpd
Final confirmation would be good. Obviously thanks to all involved in making this happen.
Does this mean we have a working version for OPNsense now??? ;D ;D ;D
"# opnsense-revert -z miniupnpd", is this the command to update miniupnpd without waiting for next OPNsense patch?
Yes, opnsense-revert will give you the latest snapshot in this case. One caveat applies: your firmware mirror needs to sync the latest snapshots or use the main mirror as a workaround.
Cheers,
Franco
Quote from: franco on May 29, 2024, 07:25:35 PM
Yes, opnsense-revert will give you the latest snapshot in this case. One caveat applies: your firmware mirror needs to sync the latest snapshots or use the main mirror as a workaround.
Cheers,
Franco
The results of my testing so far seems to be good :)
A big thank you to all involved sorting this out, I wish I had more knowledge and comeptence to contribute.
Will the updated code be included in 24.1.9? Also, thank you!
Yes, you can wait for 24.1.9 (1-2 weeks) or install the snapshot package with the opnsense-revert command now. It's unlikely to receive more fixes given the circumstances. ;)
Cheers,
Franco
For the sake of having another data point, I figured I'd report back. I ran the opnsense-revert command, restarted the service, and everything works perfectly.
Thanks much for everything! It's much appreciated.
Likewise, thanks for the feedback!