Messages

Great, The plugin 0.1.6 with netflector 0.16.0 is installed and working perfectly here.

Thanks again for all the work and support!

Happy to help, thanks for the idea and all the testing.

netflector 0.16.0 and plugin 0.1.6 are released. The main change is addition of generic UDP relay (udp_ports, udp_groups, udp_broadcast) to support other discovery protocols (like Roon's SOOD) and peers (source_peers, target_peers) to support interfaces without multicast/broadcast domains.

Thanks for running the baseline. If it's just as flaky on plain Wi-Fi with no tunnel and no netflector, then it's Roon itself after a force-close, and there's nothing left to fix on the network side.

Good to know where things stand though. Peers fix discovery over WireGuard, but the phone needs a /24 on its tunnel address, otherwise it sends its endpoint announcement to itself. And the router has to drop 10.10.10.255 on the WireGuard interface, or it forwards the packet back into the tunnel and replies to the phone with ICMP. I'll put both into the docs, we'd never have found them without your captures.

Thanks again for all the testing. I'll prepare a new official release soon.

Great, thanks for nailing down the rule direction. That closes the netflector side: discovery now crosses correctly both ways. The Core's queries reach the phone, the phone's announcements reach the LAN broadcast once each with the phone's source kept, nothing answers the phone with ICMP anymore, and the Core replies to them.

What is left happens after discovery. From the announcement the Core learns the endpoint's RAAT port (tcp_port 9200 in your capture) and connects to the phone over plain routed TCP, which netflector is not part of. Three things would tell us where it fails:

• A baseline: the same four force-close/open attempts with the phone on your LAN Wi-Fi, WireGuard off. If the endpoint is intermittent there too, it is Roon's own behaviour after a force-close, for example the Core still holding the previous session for that device.
• During a run over WireGuard with both good and failed attempts, capture on both interfaces at once:
  tcpdump -ni wg0 -w /tmp/wg0.pcap host 10.10.10.2
  tcpdump -ni ax1 -w /tmp/ax1.pcap host 10.10.10.2
  Note the time of each attempt and whether the endpoint appeared, then send me both files. I want to see whether the Core opens a TCP connection to 10.10.10.2:9200 on the failed attempts, whether that connection shows up on both interfaces, and what the phone answers.
• In a failed attempt, how long did you wait? The phone re-announces every few seconds, so if the endpoint never shows up while announcements keep arriving, the Core is ignoring them or its TCP connect keeps failing.

@RamSense, thanks, this run settles the relay question: wg0 resolves with its /24, both directions are registered with broadcasts, and the announcements reach ax1 on every attempt. Sorry about trace, by the way: release builds compile it out, so debug was all that binary could log.

Your log shows something else though: every datagram from the phone is relayed twice. I reproduced that locally once I enabled IP forwarding on my test router. WireGuard interfaces have no broadcast flag, so the router treats 10.10.10.255 as an ordinary host and forwards the packet back out wg0. netflector picks up that forwarded copy too, and the WireGuard driver, finding no peer for 10.10.10.255, answers the sender with an ICMP "host 10.10.10.255 unreachable". So since the /24 change OPNsense has most likely been answering every Roon announcement with an ICMP error to the phone, which is exactly what udp-proxy-2020 found breaks Roon on iOS. Your earlier capture without ICMP was taken with the /32, where this does not happen.

Could you try this:

• First confirm the ICMP: run tcpdump -ni wg0 icmp while opening Roon on the phone. I expect "host 10.10.10.255 unreachable" from 10.10.10.1 to 10.10.10.2.
• Add a firewall rule on the WireGuard interface: action Block (not Reject), direction in, quick, destination single host 10.10.10.255, placed above your pass rules.
• Repeat the ICMP capture, it should now stay empty, and repeat the four open/close attempts.

netflector captures ahead of the firewall, so it still relays the announcement. In my test setup the rule leaves exactly one relayed copy per announcement and no ICMP toward the phone.

@RamSense, thanks, that narrows it down. I rebuilt your setup locally: FreeBSD, a real WireGuard tunnel, the router's wg0 at 10.10.10.1/24, a client at 10.10.10.2/24 sending 348 bytes to 10.10.10.255:9003. There netflector relays the packet to 192.168.1.255:9003 with the client's source kept, so something differs on your box.

Could you set log_level = "trace", restart netflector, open Roon on the phone so it announces a couple of times, then stop it, and send me:

• The output of netflector-0.16.0rc2-freebsd15-amd64 --version and your config file.
• Command line you use to start it, and any NETFLECTOR_* environment variables set for it.
• The full log of that run.
• Whether your LAN is ax1 itself or a VLAN, bridge or lagg on top of it.

Also, please make sure the netflector service from the plugin is not running at the same time. pgrep -lf netflector should list only the netflector-0.16.0rc2-freebsd15-amd64 process you started by hand. If it also shows /usr/local/sbin/netflector or a daemon: line for it, disable the service on the plugin's page, or stop it with service netflector onestop.

@RamSense, that is the smoking gun, thank you. The 348-byte packet is the phone announcing itself as an endpoint, sent to the broadcast address of its tunnel interface. Roon derives that from the address and mask, and for a /32 the broadcast address is the host's own address, which is why you see 10.10.10.2 -> 10.10.10.2. netflector relays broadcasts from wg0, but that packet is not one, so the Core never hears the announcement and only learns the phone when it happens to answer one of the Core's own queries.

Could you change the iPhone's interface address in the WireGuard app to 10.10.10.2/24 and test again? Roon should then announce to 10.10.10.255, which netflector relays to the LAN broadcast with the phone's source address kept. A capture on the LAN should show the 348-byte packets arriving at 192.168.1.255:9003.

@RamSense Thanks, that confirms the peers change does what it should: the copy carries the server's own source, and the multicast and broadcast errors are gone.

On the endpoint problem. The ICMP you captured goes from the phone to the server, so it tells us the Roon app had no socket open on 9003 at that moment: the phone cannot answer the server's query while its port is closed, which is exactly "not an endpoint". The open question is why the port was closed. Two candidates: the app was in the background (iOS suspends it, and Roon holds its listener only in the foreground), or something made the app drop the socket while it was up. udp-proxy-2020's finding was the second kind, but that needs an ICMP port unreachable arriving at the phone, and there is none in what you sent. netflector's host does not hold port 9003 open, so if the phone ever sends unicast 9003 traffic to one of the OPNsense addresses, OPNsense would answer with exactly that ICMP. Whether that happens depends on where the 350-byte packets go.

Could you check:

• The exact destination address of the 350-byte packets on wg0, and whether their payload is a SOOD query or a reply.
• Any ICMP toward the phone on wg0: tcpdump -ni wg0 icmp and dst host 10.10.10.2.
• The wg0 address and prefix on OPNsense, and the phone's tunnel address and prefix (/24 or /32).
• Whether the Roon app was in the foreground when you captured the port-unreachable from the phone.
• Which OS the Roon Core runs on.

@RamSense

Next iteration is ready. Binary:
https://drive.google.com/uc?export=download&id=1FPDtVM_e-lk2ygqvrxRcMZ9HWt5kVZ_P

SHA256:
e0b0c32a9f062c36c29ddc7cf193ae984e59fc5215ee05167391f3e14cc2f3b4

I've added support for source_peers and target_peers (will be used for source_if and target_if correspondingly). The idea is not to open a fake socket, but to actually fake multicast over WireGuard by delivering unicast messages to peers. Which means discovery in the other direction should properly work. We'll see how it goes :)

If we assume that your WireGuard interface is target_if, then you need to add target_peers with the IP of your phone (or multiple IPs if you have more than one road warrior). Something like this:

Code Select Expand

log_level = "info"

# Roon for road warriors: the server on the LAN, the phones behind WireGuard.
# wg0 has no broadcast domain, so the server's discovery (multicast and
# broadcast on 9003) goes to each listed peer as a unicast copy instead.
[reflectors.roon-remote]
source_if = "igc0"                # the LAN interface, as ifconfig names it
target_if = "wg0"                 # the WireGuard interface
target_peers = ["10.10.10.2"]     # every road-warrior address that should see the server; add more as needed
udp_ports = [9003]
udp_groups = ["239.255.90.90"]
udp_broadcast = true
bidirectional = true

NOTE: peers should work with almost all protocols, not only UDP relay. The only exception is mDNS. It will not accept source_peers, and no peers at all if bidirectional=true. mDNS via WireGuard is a different topic though. RFC requires multicast responses to multicast queries, which we cannot do. I have some ideas, but this I can try myself, after we figure Roon out.

@RamSense I have identified the problem with the ICMP unreachable responses. It has to do with the fact that WG tunnel has no broadcast and no multicast domains. I'm working on the solution.

Thanks, great to hear that the first implementation is ready. I'm happy to test it.

I'm running:

OPNsense 26.7.3_11-amd64
FreeBSD 15.1-RELEASE-p3
OpenSSL 3.5.8
Architecture: amd64

Binary:
https://drive.google.com/uc?export=download&id=1VcsmIz9SZDM5Sxqav6a19TTZbojL_gCR

SHA256:
f485f068b0529f36851e8dfeeae06e1bd1f118b5405dadcd67b9eb78fe34a633  netflector-0.16.0-freebsd15-amd64

Here is a config (I assume you will call it roon.toml) you can use for Roon reflection. Feel free to enable more protocols if needed. It assumes that unicast replies are allowed in the firewall. Add more entries if you need reflection between more VLANs.

Code Select Expand

# "debug" shows every relayed datagram; drop to "info" once it works.
log_level = "debug"

[reflectors.roon]
# Interface names as ifconfig lists them, e.g. igb0 / igb1 or vtnet0 / vtnet1.
source_if = "lan"
target_if = "iot"
udp_ports = [9003]
udp_groups = ["239.255.90.90"]
udp_broadcast = true
bidirectional = true

First smoke test:

Code Select Expand

./netflector-0.16.0-freebsd15-amd64 --version
./netflector-0.16.0-freebsd15-amd64 --check-config roon.toml

Then run it (for the purity of this experiment make sure that netflector plugin is stopped):

Code Select Expand

./netflector-0.16.0-freebsd15-amd64 roon.toml

At start the log should show something like this:

Code Select Expand

reflector roon: lan <-> iot [udp(9003 on 239.255.90.90,broadcast)]

And then each reflected query will log something like this:

Code Select Expand

reflected UDP relay datagram from <ip>:<port> to ...


That sounds like a sensible approach.

First implementation is ready. Which version of OPNsense are you running? Is it amd64 or arm64? Basically I need to know FreeBSD version and architecture to prepare a binary for you.

That was added because ICMP Port Unreachable responses could otherwise cause problems for Roon clients

netflector does this trick for some protocols, so this machinery is already built. However, I don't think ICMP Port Unreachable can happen for SOOD. This problem manifests itself when we are capturing a unicast packet addressed to our IP using BPF device. In this case kernel still sees the packet addressed to itself and, since there is no socket waiting for it, replies with the ICMP Port Unreachable. We are going to spoof source IP, which means all unicast responses will be directed (and routed) to the device with that IP (and not to netflector's IP). I'm not sure why udp-proxy-2020 would need to solve ICMP Port Unreachable problem... Maybe I'm missing something.

In any case, thanks for your insights. I think I might implement this as a generic case of payload-unaware reflection.

Would you consider adding dedicated support for Roon discovery (SOOD/RAAT discovery) as an additional reflector type?

I have read a bit about SOOD discovery protocol. There are a few differences from existing protocols, but I still feel like it's a good match. Differences:

• We need to preserve source IPs. This is easy because netflector already uses raw BPF devices for all UDP traffic.
• There are 3 types of devices involved: remotes, server and outputs. Discovery would need to happen for remotes -> server and for server -> outputs. Potentially, all three could be in separate VLANs. For now, I think it would require two reflector entries.

Regarding your specific setup with blackjack and VPNs: I don't think we need to implement anything specific for that to work. If remotes are connected via VPN, that VPN interface would just be a source_if in the entry responsible for remote -> server discovery. I don't think that netflector needs to be aware of anything VPN-specific.

There is also the question of unicast responses. For SSDP and WSD netflector proxies these responses. For SOOD, I'm not yet sure what would be the best approach. The normal discovery flow is not a problem: remote sends "Q", server responds with "R". We can proxy the unicast response fine here, just use netflector's source IP/port when relaying "Q" to the server. However, it looks like "Q" does not necessarily mean Query. Output devices also send "Q" messages to server, which are not discovery queries. For these there is no unicast response, server connects directly to output device instead. So this "Q" must be relayed with the preserved IP/port. Naive solution here would be to not relay unicast responses and always preserve source IP/port for all relayed SOOD packets. This will require allowing UDP unicast packets from server back to remote (and from output back to server) in firewall. In your case you should probably already have that configured.

Can you create a PR for https://github.com/opnsense/ports preferably under opnsense/netreflector to avoid clashing with an eventual FreeBSD inclusion? The files can stay the same, but so we know it's out of tree.

Done: https://github.com/opnsense/ports/pull/286.