OPNsense Forum

Archive => 20.7 Legacy Series => Topic started by: uglymotha on August 07, 2020, 12:39:34 pm

Title: Multicast / igmpproxy broken in 20.7 (FreeBSD 12)
Post by: uglymotha on August 07, 2020, 12:39:34 pm
I discovered a pretty serious issue with multicast in version 20.7 / FreeBSD 12. Multicast groups are joined, but never left. This causes streams to keep running indefinitely and can quickly saturate upstream links.

I am currently in the process of rewriting / overhauling the igmpproxy source code and discovered this issue after upgrading my opnsense installation from 20.1 to 20.7. After testing using stock installatins, I established the issue is present since FreeBSD 12, up to the most recent stable p8 release.

I have opened up a bug report with FreeBSD:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=248512

For now I would advice anybody using multicast routing / igmpproxy to stay away from 20.7.
Title: Re: Multicast / igmpproxy broken in 20.7 (FreeBSD 12)
Post by: uglymotha on August 07, 2020, 03:01:38 pm
Issue was fixed a few weeks ago, and no longer exists in FreeBSD latest snapshot kernel.
https://svnweb.freebsd.org/base?view=revision&revision=362472

The hardened BSD kernel in use by opnsense however still suffers from this rather nasty bug, rendering multicast pretty much unusable. I took the liberty of opening a PR for their stable branch, but don't know whether they'll accept it.

Is there any documentation for compiling a custom kernel to use with opnsense? I would rather go that route than downgrading to 20.1. Hardned BSD dev branch is up to date with FreeBSD, last merge there was a few days ago.
Title: Re: Multicast / igmpproxy broken in 20.7 (FreeBSD 12)
Post by: franco on August 07, 2020, 03:26:58 pm
FreeBSD 12.1 suffers still and may never receive a fix. Honestly, I doubt their release engineering approach with regards to a reliable release as it seems to mean reliable in "features as well as bugs". ;)

I followed the discussion, added:

https://github.com/opnsense/src/commit/d6dc01f471b4

You can try the kernel to help get this into 20.7.1:

# opnsense-update -kr 20.7-mcast
# opnsense-shell reboot


Cheers,
Franco
Title: Re: Multicast / igmpproxy broken in 20.7 (FreeBSD 12)
Post by: staticznld on August 07, 2020, 03:40:25 pm
Thanks Franco!

It looks like the same problem i have.

https://forum.opnsense.org/index.php?topic=18347.0

Going to try the new kernel tonight or tomorrow.
Title: Re: Multicast / igmpproxy broken in 20.7 (FreeBSD 12)
Post by: franco on August 07, 2020, 03:46:16 pm
Thanks for the link. I was looking for the topic briefly to post a reference but happy you found this directly. :)


Cheers,
Franco
Title: Re: Multicast / igmpproxy broken in 20.7 (FreeBSD 12)
Post by: uglymotha on August 07, 2020, 07:12:59 pm
Unfortunately the issue persists  with the supplied test kernel. It must have been a different bug and commit that fixes this. Most likely:
https://github.com/freebsd/freebsd/commit/63bc20993b4f570ff1a7c45b5dead0109768d494#diff-c9065ed6e74837c7cb1ded9eb39e7fb9
Title: Re: Multicast / igmpproxy broken in 20.7 (FreeBSD 12)
Post by: staticznld on August 07, 2020, 07:27:22 pm
Installed kernel but no succes. :-[

Multicast streams keeps stacking up.
Also after 4 minutes the tvstream looses connectivity. All multicast stream are gone.

Also a lot of error in the log of igmpproxy

igmpproxy[77319]: MRT_DEL_MFC; Errno(49): Can't assign requested address
Title: Re: Multicast / igmpproxy broken in 20.7 (FreeBSD 12)
Post by: franco on August 10, 2020, 10:15:33 am
Patchfest... yay, new kernel coming soon. Hold tight.


Cheers,
Franco
Title: Re: Multicast / igmpproxy broken in 20.7 (FreeBSD 12)
Post by: franco on August 10, 2020, 11:12:38 am
# opnsense-update -kr 20.7-re-mcast
# opnsense-shell reboot


Cheers,
Franco
Title: Re: Multicast / igmpproxy broken in 20.7 (FreeBSD 12)
Post by: staticznld on August 10, 2020, 11:32:56 am
Thanks for the new test kernel!

Tried the the new kernel but still no succes.
Errors are gone in the system log, when stopping igmp proxy multicast streams stop comming in.
But when zapping through TV channels multicast traffic is still stacking up.

Edit:

Still seeing error mesages in the log
igmpproxy[6669]: MRT_DEL_MFC; Errno(49): Can't assign requested addres
Title: Re: Multicast / igmpproxy broken in 20.7 (FreeBSD 12)
Post by: franco on August 10, 2020, 11:39:14 am
I'm open for suggestions. But just for documentation purposes... I did not expect this particular adventure down the rabbit hole.


Cheers,
Franco
Title: Re: Multicast / igmpproxy broken in 20.7 (FreeBSD 12)
Post by: franco on August 10, 2020, 11:47:47 am
Last try for the day:

https://github.com/opnsense/src/commit/6a106324130

Kernel to install:

# opnsense-update -kr 20.7-re-mcast2
# opnsense-shell reboot

(Will be available in a few minutes)


Cheers,
Franco
Title: Re: Multicast / igmpproxy broken in 20.7 (FreeBSD 12)
Post by: staticznld on August 10, 2020, 11:57:25 am
Woohoo!!

It looks like the leave requests are leaving with this kernel!
Still some testing to do, but the first results are fine!
Everything looks like 20.1 now!

Thanks for the awsome support gonna make a donation today!
Title: Re: Multicast / igmpproxy broken in 20.7 (FreeBSD 12)
Post by: heresjody on August 10, 2020, 03:25:50 pm
Franco I can confirm the mcast2 kernel solves the leaving multicast problems. Thanks! Do I understand correctly this fix will be tweaked into the kernel for the upcoming 20.7.1 release?
Title: Re: Multicast / igmpproxy broken in 20.7 (FreeBSD 12)
Post by: franco on August 10, 2020, 03:40:10 pm
Thanks all! Yes, we will include this in 20.7.1. ETA is Thursday this week.


Cheers,
Franco
Title: Re: Multicast / igmpproxy broken in 20.7 (FreeBSD 12)
Post by: mattjes04 on October 05, 2020, 10:09:08 am
I have the feeling this problem still persists. I'm on:

OPNsense 20.7.3-amd64
FreeBSD 12.1-RELEASE-p7-HBSD
OpenSSL 1.1.1g 21 Apr 2020

After switching through three or four IPTV channels, ifmcstat and igmpproxy verbose showing this:

Code: [Select]
root@utm:~ # ifmcstat
pppoe1:
inet 79.249.156.66
igmpv3 rv 2 qi 60 qri 100 uri 3
group 232.0.20.85 mode exclude
inet6 fe80::20e:c4ff:fed4:4615%pppoe1 scopeid 0xe
mldv2 flags=2<USEALLOW> rv 2 qi 125 qri 10 uri 3
group ff01::1%pppoe1 scopeid 0xe mode exclude
group ff02::2:9546:cde2%pppoe1 scopeid 0xe mode exclude
group ff02::2:ff95:46cd%pppoe1 scopeid 0xe mode exclude
group ff02::1%pppoe1 scopeid 0xe mode exclude
group ff02::1:ffd4:4615%pppoe1 scopeid 0xe mode exclude
inet 79.249.156.66
igmpv3 rv 2 qi 60 qri 100 uri 3
group 224.0.0.1 mode exclude
group 232.0.20.234 mode exclude
group 232.0.20.35 mode exclude


Code: [Select]
root@utm:~ # /usr/local/sbin/igmpproxy -d -vv /usr/local/etc/igmpproxy.conf
Current routing table (Activate Route):
-----------------------------------------------------
#0: Src0: 87.141.215.251, Dst: 232.0.20.35, Age:2, St: A, OutVifs: 0x00000000
#1: Src0: 87.141.215.251, Dst: 232.0.20.85, Age:2, St: A, OutVifs: 0x00000000
#2: Src0: 87.141.215.251, Dst: 232.0.20.234, Age:2, St: A, OutVifs: 0x00000000
-----------------------------------------------------
Route activate request from 87.141.215.251 to 232.0.20.234 on VIF[1]

Current routing table (Activate Route):
-----------------------------------------------------
#0: Src0: 87.141.215.251, Dst: 232.0.20.35, Age:2, St: A, OutVifs: 0x00000000
#1: Src0: 87.141.215.251, Dst: 232.0.20.85, Age:2, St: A, OutVifs: 0x00000000
#2: Src0: 87.141.215.251, Dst: 232.0.20.234, Age:2, St: A, OutVifs: 0x00000000
-----------------------------------------------------
Route activate request from 87.141.215.251 to 232.0.20.85 on VIF[1]

Current routing table (Activate Route):
-----------------------------------------------------
#0: Src0: 87.141.215.251, Dst: 232.0.20.35, Age:2, St: A, OutVifs: 0x00000000
#1: Src0: 87.141.215.251, Dst: 232.0.20.85, Age:2, St: A, OutVifs: 0x00000000
#2: Src0: 87.141.215.251, Dst: 232.0.20.234, Age:2, St: A, OutVifs: 0x00000000
-----------------------------------------------------
Route activate request from 87.141.215.251 to 232.0.20.35 on VIF[1]
[/s]

Sorry, I had a problem with the upgrade process and have been still on 20.7. After upgrading to 20.7.3 it's working flawlessly!