"Recent posts
#1
Virtual private networks / Hub-Spoke Wireguard VPN works,...
Last post by ala_nathaniel - Today at 08:34:14 PMHave 10 OPNsense firewalls configured in a Hub - Spoke Site to Site Wireguard VPN. 99% is functioning like it should. All LAN clients can talk to any other client on any other network, speed is great, adding new sites auto adds routes. The only issue I am having is that anything from the Firewall itself will not route over the WG tunnel. Things like LDAP, DNS, Ping, Traceroute, etc, will just fail with no route available. This only affects the firewalls. Anything behind the firewall works just fine, and the firewall can route over any other interface.
Configuration from hub to 1 spoke (other spokes are identical, just increment the second octet by 1)
Hub Firewall
WireGuard Instance WG0
Listen Port = 51820
Tunnel Address = 172.19.0.1/24
Peers = TAC, BEL, SEA, TUK, LYN, SIL, POU, KNT, BAL
Disable Routes = Unchecked
Peer - TAC
Allowed IP = 172.19.0.2/24, 10.2.0.0/16
Endpoint Port = 51820
Keep Alive = 25
TAC Firewall
WireGuard Instance WG0
Listen Port = 51820
Tunnel Address = 172.19.0.2/24
Peers = HUB
Disable Routes = Unchecked
Peer - HUB
Allowed IP = 172.19.0.1/24, 10.1.0.0/16, 10.2.0.0/16, etc
Endpoint Port = 51820
Keep Alive = 25
Firewall Rules are fairly open right now as I am trying to troubleshoot. On the LAN and WG interfaces I have currently set to allow all.
What I see -
Pinging from TAC firewall to (Via Shell):
HUB Firewall LAN IP = 100% Packet Loss
HUB Firewall WG IP = 100% Packet Loss
Server on 10.1.0.0/16 Subnet = 100% Packet Loss
8.8.8.8 = 0 Packet Loss
Server on 10.2.0.0/16 Subnet = 0% Packet Loss
SEA Firewall LAN IP = 100% Packet Loss
SEA WG IP = 100% Packet Loss
Pinging from client on TAC LAN Subnet to any of the above gets 0 packet loss and low latency. Works as expected.
Traceroute from TAC Firewall to (via Shell):
HUB Firewall LAN IP = Timeout
HUB Firewall WG IP = Timeout
Server on 10.1.0.0/16 Subnet = Timeout
8.8.8.8 = Successfully traces
Server on 10.2.0.0/16 Subnet = Successfully traces
SEA Firewall LAN IP = Timeout
SEA WG IP = Timeout
Netstat -rn on either firewall shows correct routes over WG0
Firewall logs on both firewalls do not show blocking any traffic
Packet Capture on both firewalls show the packets going out and even over the tunnel, but never a response.
Using Web GUI to configure LDAP Server, I get a host can not be reached.
Using Web GUI to forward specific domains on Unbound DNS to HUB DNS server, I can add it, but does not get packets.
Things I have tried -
Rebooting :D
Changing from Disable Routes unchecked to checked and manually adding routes = No change
Making the firewall rules more open = No Change
Rebuilding WG = No Change
Not sure if this applies to me:
https://forum.opnsense.org/index.php?topic=41506.msg232809#msg232809
But willing to try if people think it will help.
I am kinda stumped. I have setup WireGuard before on OPNsense and have not had this issue. We were previously on ZeroTier, so I thought that might be causing issues, but we have added two new firewalls since then and they have the same problem.
Thanks in advance for any thoughts!
Configuration from hub to 1 spoke (other spokes are identical, just increment the second octet by 1)
Hub Firewall
WireGuard Instance WG0
Listen Port = 51820
Tunnel Address = 172.19.0.1/24
Peers = TAC, BEL, SEA, TUK, LYN, SIL, POU, KNT, BAL
Disable Routes = Unchecked
Peer - TAC
Allowed IP = 172.19.0.2/24, 10.2.0.0/16
Endpoint Port = 51820
Keep Alive = 25
TAC Firewall
WireGuard Instance WG0
Listen Port = 51820
Tunnel Address = 172.19.0.2/24
Peers = HUB
Disable Routes = Unchecked
Peer - HUB
Allowed IP = 172.19.0.1/24, 10.1.0.0/16, 10.2.0.0/16, etc
Endpoint Port = 51820
Keep Alive = 25
Firewall Rules are fairly open right now as I am trying to troubleshoot. On the LAN and WG interfaces I have currently set to allow all.
What I see -
Pinging from TAC firewall to (Via Shell):
HUB Firewall LAN IP = 100% Packet Loss
HUB Firewall WG IP = 100% Packet Loss
Server on 10.1.0.0/16 Subnet = 100% Packet Loss
8.8.8.8 = 0 Packet Loss
Server on 10.2.0.0/16 Subnet = 0% Packet Loss
SEA Firewall LAN IP = 100% Packet Loss
SEA WG IP = 100% Packet Loss
Pinging from client on TAC LAN Subnet to any of the above gets 0 packet loss and low latency. Works as expected.
Traceroute from TAC Firewall to (via Shell):
HUB Firewall LAN IP = Timeout
HUB Firewall WG IP = Timeout
Server on 10.1.0.0/16 Subnet = Timeout
8.8.8.8 = Successfully traces
Server on 10.2.0.0/16 Subnet = Successfully traces
SEA Firewall LAN IP = Timeout
SEA WG IP = Timeout
Netstat -rn on either firewall shows correct routes over WG0
Firewall logs on both firewalls do not show blocking any traffic
Packet Capture on both firewalls show the packets going out and even over the tunnel, but never a response.
Using Web GUI to configure LDAP Server, I get a host can not be reached.
Using Web GUI to forward specific domains on Unbound DNS to HUB DNS server, I can add it, but does not get packets.
Things I have tried -
Rebooting :D
Changing from Disable Routes unchecked to checked and manually adding routes = No change
Making the firewall rules more open = No Change
Rebuilding WG = No Change
Not sure if this applies to me:
https://forum.opnsense.org/index.php?topic=41506.msg232809#msg232809
But willing to try if people think it will help.
I am kinda stumped. I have setup WireGuard before on OPNsense and have not had this issue. We were previously on ZeroTier, so I thought that might be causing issues, but we have added two new firewalls since then and they have the same problem.
Thanks in advance for any thoughts!
#2
26.1 Series / Re: MiniUPNPD
Last post by Javier® - Today at 08:23:14 PMHi, static NAT ports for UDP are a godsend for real-time protocols. Anyone who has troubleshooted WebRTC knows this: they're worth their weight in gold. They cost nothing, except to acknowledge that port "randomization" in UDP is not a security feature.
pass out quick on igc0 inet proto udp from igc1:network nat-to (igc0) static-port
pass out on igc0 inet from igc1:network nat-to (igc0)
pass out quick on igc0 inet proto udp from igc1:network nat-to (igc0) static-port
pass out on igc0 inet from igc1:network nat-to (igc0)
#3
26.1 Series / Re: Upgrade to RC1 successful
Last post by franco - Today at 08:05:08 PMYep, looked good to me: only two explicit entries and both were disabled before and after migration. If you have more tracking interface the radvd was maybe starting and feeding the others, but not these two.
Cheers,
Franco
Cheers,
Franco
#4
Tutorials and FAQs / Re: OPNsense aarch64 firmware ...
Last post by jasonczerak - Today at 07:59:10 PMI have a couple of NanoPI R6S' on order. I'm traditionally using OpenWRT (WRT3200ACM, without wifi), however since these boxes will be a fresh start, I figured I'd look into OPNsense on ARM.
Please correct me if I'm wrong here. I'm going to have to do a FreeBSD install, then boot strap per the first post?
Will all the plugins/modules work? Specifically I'm looking for Multiwan, vlan routing, firewall, S2S VPN (stringswan or something simular) and the ability to failover (virtual mac) for WAN connections between device with conntractd to maintain state.
I found https://yrzr.github.io/running-opnsense-on-r6s/, but it's dated, doesn't look to useful anymore.
Please correct me if I'm wrong here. I'm going to have to do a FreeBSD install, then boot strap per the first post?
Will all the plugins/modules work? Specifically I'm looking for Multiwan, vlan routing, firewall, S2S VPN (stringswan or something simular) and the ability to failover (virtual mac) for WAN connections between device with conntractd to maintain state.
I found https://yrzr.github.io/running-opnsense-on-r6s/, but it's dated, doesn't look to useful anymore.
#5
26.1 Series / Re: Upgrade to RC1 successful
Last post by OPNenthu - Today at 07:41:37 PMIt was further down the list but I found it. Sent.
I'm not impeded so no concern on my end if this doesn't seem actionable. Mainly just reporting on the offchance mine isn't an isolated incident.
Thanks franco!
I'm not impeded so no concern on my end if this doesn't seem actionable. Mainly just reporting on the offchance mine isn't an isolated incident.
Thanks franco!
#6
26.1 Series / Re: Upgrade to RC1 successful
Last post by franco - Today at 07:21:13 PMI'd like to see the System: Configuration: History diff for the migration of the radvd settings when it went to RC1.
It says "run_migrations.php made changes" in the left dropdown. The top one is probably it. Just click and it shows the diff vs. the previous one which is the interesting one. You can send it to franco AT opnsense DOT org
Thanks,
Franco
It says "run_migrations.php made changes" in the left dropdown. The top one is probably it. Just click and it shows the diff vs. the previous one which is the interesting one. You can send it to franco AT opnsense DOT org
Thanks,
Franco
#7
General Discussion / Re: [Noob question] - DNS Cach...
Last post by Taxickdk - Today at 07:18:43 PMHi,
Thanks for the reply. I rebooted my servers and PC, and now it works fine.
Thanks for the reply. I rebooted my servers and PC, and now it works fine.
#8
26.1 Series / Re: MiniUPNPD
Last post by fotring - Today at 07:13:02 PMQuote from: Kewin on Today at 09:48:45 AMJust to chime in since I guess not that many are using miniupnpd.. I'm still running 25.7.11_2 and I use UPnP for consoles and kids' gaming, and I'm not seeing those errors in my log..
(A lot of other errors, but I'm guessing it's because the clients didn't clear their active mappings before shutting off)..Code Select2026-01-27T09:23:31 Error miniupnpd upnpevents_processfds: 0x1239f410080, remove subscriber uuid:4a4dccd4-fb59-11f0-af55-00d0b4023658 after an ERROR cb: http://10.0.10.134:2869/upnp/eventing/dkgqwukrhw
2026-01-27T09:23:31 Warning miniupnpd upnp_event_process_notify: connect(10.0.10.134:2869): Operation timed out
2026-01-27T09:23:31 Error miniupnpd upnpevents_processfds: 0x1239f410100, remove subscriber uuid:4a487762-fb59-11f0-af55-00d0b4023658 after an ERROR cb: http://10.0.10.134:2869/upnp/eventing/bejxzoycej
2026-01-27T09:23:31 Warning miniupnpd upnp_event_process_notify: connect(10.0.10.134:2869): Operation timed out
2026-01-27T09:23:31 Warning miniupnpd upnp_event_process_notify: connect(10.0.10.134:2869): Operation timed out
2026-01-27T09:23:15 Warning miniupnpd upnp_event_process_notify: connect(10.0.10.134:2869): Operation timed out
2026-01-27T09:23:15 Warning miniupnpd upnp_event_process_notify: connect(10.0.10.134:2869): Operation timed out
2026-01-27T09:23:15 Warning miniupnpd upnp_event_process_notify: connect(10.0.10.134:2869): Operation timed out
2026-01-27T09:21:25 Warning miniupnpd upnp_event_process_notify: connect(10.0.1.153:2869): Operation timed out
2026-01-27T09:07:13 Error miniupnpd upnpevents_processfds: 0x1239f410000, remove subscriber uuid:0319bd80-fb57-11f0-af55-00d0b4023658 after an ERROR cb: http://10.0.10.127:2869/upnp/eventing/ujhzqdwdtn
2026-01-27T09:07:13 Warning miniupnpd upnp_event_process_notify: connect(10.0.10.127:2869): Operation timed out
2026-01-27T09:07:13 Error miniupnpd upnpevents_processfds: 0x1239f410280, remove subscriber uuid:0314bbca-fb57-11f0-af55-00d0b4023658 after an ERROR cb: http://10.0.10.127:2869/upnp/eventing/ocsmvlvmza
/Kewin
Good note! Then it's not just my install. Can something have changed upstream in miniupnpd? Im on 2.3.9_2,1.
#9
26.1 Series / Re: Upgrade to RC1 successful
Last post by OPNenthu - Today at 06:00:19 PM(Sorry, saw your message late. I didn't reboot after RC2.)
---
Ok, I'm getting a different set of results this time but nothing too concerning like from earlier.
This time I got a php error after the last upgrade toward 26.1.r2_2 and I submitted the crash report from within the GUI.
I saw that on the intermediate upgrade to 26.1.r_9, the list of interfaces under Services->Router Advertisements had already shrunk down to two interfaces from the full set of ten internal interfaces that I have, and it's the same two as before (LAN & CLEAR). The other 8 interfaces are missing from the radvd list, even though they all have the same settings (Track Interface w/ Allow manual adjustments checked). The difference this time is that the radvd service was not enabled on them after the upgrade.
Next I manually changed all the internal interfaces to "Identity Association," and as before I clicked "Save" between each interface change and applied them all in one shot at the end. It took a bit to process and my WAN IPv6 gateway went down for some reason (I didn't edit that interface) but came up after some time. The UI is still responding at this point.
Finally I removed the ISC plugin from the GUI. This time I did not get a crash. The GUI is still responding.
So it seems as though the upgrade is not deterministic when starting from the "same" state. This surprised me. There's some variability somewhere, but it's likely not in the packages (?) and it's likely not in the config file (preserved by snapshot).
Anyhow, I have the before/intermediate/after configs and I also captured the terminal buffer during the major upgrade to 26.1.r_9. Let me know if you need me to send them to you.
---
Ok, I'm getting a different set of results this time but nothing too concerning like from earlier.
This time I got a php error after the last upgrade toward 26.1.r2_2 and I submitted the crash report from within the GUI.
I saw that on the intermediate upgrade to 26.1.r_9, the list of interfaces under Services->Router Advertisements had already shrunk down to two interfaces from the full set of ten internal interfaces that I have, and it's the same two as before (LAN & CLEAR). The other 8 interfaces are missing from the radvd list, even though they all have the same settings (Track Interface w/ Allow manual adjustments checked). The difference this time is that the radvd service was not enabled on them after the upgrade.
Next I manually changed all the internal interfaces to "Identity Association," and as before I clicked "Save" between each interface change and applied them all in one shot at the end. It took a bit to process and my WAN IPv6 gateway went down for some reason (I didn't edit that interface) but came up after some time. The UI is still responding at this point.
Finally I removed the ISC plugin from the GUI. This time I did not get a crash. The GUI is still responding.
So it seems as though the upgrade is not deterministic when starting from the "same" state. This surprised me. There's some variability somewhere, but it's likely not in the packages (?) and it's likely not in the config file (preserved by snapshot).
Anyhow, I have the before/intermediate/after configs and I also captured the terminal buffer during the major upgrade to 26.1.r_9. Let me know if you need me to send them to you.
#10
26.1 Series / Re: 26.1.rc1 -> 26.1 rc2 ........
Last post by franco - Today at 05:55:14 PMWhat does your System: Firmware: Status say?
Cheers,
Franco
Cheers,
Franco
