Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - coreyo

#1
I picked up a 4GB Nano-Pi R4S a little while back (RK3399 SoC based ARM v8-A 64-bit board) and I'm using custom 21.7 OPNSense image and repository from PersonalBSD.org here:

https://personalbsd.org/
https://personalbsd.org/?page_id=2 (downloads)

The device itself is great, and much appreciation for the maintainer of this image and repository. Unfortunately, I have found no identifying information and no way to contact them. Most everything seems to be working just fine and all of the necessary packages are present except for one. The reason that I purchased the Nano-Pi was because it is one of the few little devices capable of handling wireguard encryption above and beyond my internet speeds. Unfortunately, neither the default image nor the default wireguard package includes the kernel module. This means that the encryption/decryption over the VPN is all running in userspace land which is significantly slower. On other setups, installing the wireguard-kmod package improved my throughput speeds by as much as a factor of 10. My current throughput averages around 110 Mbps (that's pretty impressive for a little ARM cortex device running in userspace), but I've seen kernel space benchmarks posted that have it topping out at around a gigabit. In the meantime, my internet speeds are suffering from throttling.

Here's a link to the wireguard-kmod project:
https://fingerlessgloves.me/2021/03/31/hello-wireguard-kmod-on-opnsense/

Would it be possible to compile my own wireguard kernel module for this architecture? If so, what would I need and can anyone kindly point me in the right direction? Can it be done directly on the device itself, or would I need to set up cross-compiling on one of my Linux boxes? Thanks in advance for any help.
#2
I have 2 subnets: "LAN" and "VLAN_NoVPN". The latter is a VLAN. All internet traffic for LAN is routed through a wireguard VPN, and all traffic for VLAN_NoVPN is not. However, this likely has nothing to do with the issue. Each is on it's own subnet with a 16 bit prefix. I have my Rokus on the VLAN_NoVPN (because services like Netflix, Hulu, etc. have a nasty habit of blocking access from commercial IP addresses). Most of my computers, phones, etc. are connected to the main LAN so that their traffic is masked by the VPN. Unfortunately, I cannot seem to get casting to work across subnets. Devices on one network do not seem to be able to discover devices on the other subnet. Here's what I've done and the debugging that I've done so far:

1. There is currently no reason for privacy across the two subnets, so my firewall rules are quite liberal. Both the LAN an VLAN interfaces have a general "allow all" rule. To test, this clients from one subnet can ssh, ping, and generally connect to clients on the other subnet no problem. This is all working as expected.

2. I can confirm that udpbroadcastrelay is running as expected by inspecting the current processes:
root@opnsense:~ $ ps aux |grep udp
root     5126   0.0  0.0  12556   1924  -  S    18:06       0:00.09 /usr/local/sbin/udpbroadcastrelay --id 2 --dev re0 --dev re0_vlan100 --port 1900 --multicast 239.255.255.250 -f
root    31808   0.0  0.0  12556   1932  -  I    22:39       0:00.05 /usr/local/sbin/udpbroadcastrelay --id 3 --dev re0 --dev re0_vlan100 --port 137 -f
root    51337   0.0  0.0  12556   1924  -  I    22:39       0:00.02 /usr/local/sbin/udpbroadcastrelay --id 4 --dev re0 --dev re0_vlan100 --port 138 -f
root    69254   0.0  0.0  12556   1924  -  S    22:39       0:00.39 /usr/local/sbin/udpbroadcastrelay --id 1 --dev re0 --dev re0_vlan100 --port 5353 --multicast 224.0.0.251 -s 1.1.1.1 -f
root  35513   0.0  0.1  12636   2140  0  S+   22:02       0:00.01 grep udp


3. I can confirm all of the broadcast packets are seen across subnets. In this particular case, the client running tcpdump is on the LAN subnet, and the "KitchenRokuUltra" device is on the NoVPN_VLAN subnet.
:~$ sudo tcpdump -i eno2 | grep 239.255.255.250
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eno2, link-type EN10MB (Ethernet), capture size 262144 bytes
22:09:15.182499 IP 10.3.1.16.46502 > 239.255.255.250.1900: UDP, length 146
22:09:15.395361 IP KitchenRokuUltra.Homeslice.43948 > 239.255.255.250.1900: UDP, length 214
22:09:15.395362 IP KitchenRokuUltra.Homeslice.43948 > 239.255.255.250.1900: UDP, length 220
22:09:15.395362 IP KitchenRokuUltra.Homeslice.43948 > 239.255.255.250.1900: UDP, length 273
22:09:15.395363 IP KitchenRokuUltra.Homeslice.43948 > 239.255.255.250.1900: UDP, length 214
22:09:15.395363 IP KitchenRokuUltra.Homeslice.43948 > 239.255.255.250.1900: UDP, length 220


Unfortunately, when I launch Chrome or a mobile YouTube app from within one subnet, none of the Roku devices from the other subnet are discoverable. Do I need a separate rule to make inter-subnet Roku discovery work?

It should also be noted that many of the devices are connected to WiFi via my ubiquiti UniFi routers. I'm not sure if that makes any difference. Thanks in advance for any insights.

--- As A Side Note ---

The mDNS rule seems to be working just fine. Not only can I see google and chromecast devices from other subnets, but they are discoverable as intended by various devices on either subnet. Looking at tcpdump, I have noticed that some of the packets actually seem to be originating from the OPNSense router. Are those the packets that it's forwarding?

:~$ sudo tcpdump -i eno2 | grep 224.0.0.251
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eno2, link-type EN10MB (Ethernet), capture size 262144 bytes
21:57:35.009888 IP opnsense.Homeslice.mdns > 224.0.0.251.mdns: 0 PTR (QM)? _googlecast._tcp.local. (40)
21:57:35.221076 IP Google-Nest-Mini.Homeslice.mdns > 224.0.0.251.mdns: 0*- [0q] 1/0/3 PTR Google-Nest-Mini-4e96b3f5decfc1f3a9367e8c59d56b88._googlecast._tcp.local. (392)
21:57:35.221077 IP opnsense.Homeslice.mdns > 224.0.0.251.mdns: 0*- [0q] 1/0/3 PTR Google-Nest-