Messages

How would you set it up to have only a few user going through TOR and not others ?
Like device3 on LAN1, and device 2,3,4 on LAN2 and device5 on LAN3
All other devices goes straight to "normal" NAT ?

My LAN uses a broader set of IPs (192.168.0.0/15), but I don't want to have all client on that LAN to go through Tor. So I configured it to only have clients with IPs on 192.168.15.0/24 subnet to go through the Tor tunnel.

In my case there is only one single LAN (and no VLANs). To make it work, I just set up DNAT (port forward) for hosts  with source in the subnet 192.168.15.0/24 to redirect to target ip 127.0.0.1 and port 9040 ( plus another rule similar to this one but for the DNS to port 9053).

Hope that helps.

Do you folks have a pointer for me on how to resolve the question on Tor browser?

Question: Does anyone know how to configure Tor Browser to use the OPNSense Tor transparent proxy? I'd like to avoid to have Tor Browser establish a Tor connection within the Tor connection already provided by OPNSense.

[franco]

Thank you franco I realized from your comment I had something misconfigured. I fixed the apparent misconfiguration, but still I could not solve the issue.

In order to get back operational with the network, I decided to rebuild my firewall. As I rebuilt it, I learned about snapshots (yep...I did not know about the super convenient snapshots feature ::)), and I used them to mark any major successful step I made in my conf.

I am now at the point where I have all my tunnels (1 s2s, 3 protonVPNs, 1 road-warrior for remote maintenance ) up and running and working solid (as intended).  :)

My learnings were not necessarily on how to fix the initial situation, that still remains a mystery, but my best guess is that I either messed up in my conf. elsewhere (although I did check it for what it feels like a million times), or my configuration got corrupted somehow with one of the recent firmware updates (just a speculation to save my ego, I have no proof of that). I did learn tho how to have easier way in reverting to a working config.

This thread is not therefore SOLVED, as I rebuilt from scratch, but since my system is working now, I will go ahead and mark it as OBSOLETE.

[Question]

Thank you, this worked for me. I can now protect a subset of my LAN with this approach 192.168.15.0/24, and with TommyTran732 suggestion, I confirm I do not see DNS leaks.

Question: Does anyone know how to configure Tor Browser to use the OPNSense Tor transparent proxy? I'd like to avoid to have Tor Browser establish a Tor connection within the Tor connection already provided by OPNSense.

1 restart only the modem:
1a the connections returned?
1b only the connections that already were UP before the restart, returned?

My router gets the ethernet cable directly from the FIOS ONT, so I do not really have a modem. When I reboot the router typically (but not always) the wg vpn tunnels that were previously working (i.e. with proper handshake) come back properly after reboot and handshake correctly.

2 handshake / gateways
2a the handshake is performed for all the connections or only the ones UP
2b status of the gateways before and after the restart.

Not sure if I understand what "UP" means, I tend to use "UP" to represent a functioning wireguard tunnel. For that the handshake needs to be successfully shown in the status page. If by "UP" you mean instances and peers that are *enabled* in the GUI, then the answer is "no" all my instances (5) are *enabled*, but only a subset completes the handshake successfully.

Gateways status before and after restart are green.

3 connection
3a do you have a DSL/cabel connection or through a 4g/5g modem?
3b the modem is also a router? or it is only a modem/ configured in bridge mode?

I have fiber, which goes into the ONT and connects via ethernet to my OPNSense router.
No modem, only the OPNSense router, no bridge mode.

[configctl wireguard stop XXX]

Using FQDNs could be a factor. Tunnels in tunnels another. I'm not sure. It's a complex world.

There is another patch we tried to triage this with, but it shouldn't change things for you, although it's harmless to try as well:

https://github.com/opnsense/core/commit/dd1c2e19e548

# opnsense-patch dd1c2e19e548

I do use FQDN in the HomeWireguard and s2s tunnel configs to point at my router's ip. This worked well for several weeks, before this behavior started showing up, so I am not sure what changed... I do not know enough to speculate, but it almost feels like somehow my wireguard configuration got corrupted. Is that even possible, and if so, would this explain this behaviors?

Anyway, many thanks you for the patch, I tried it, doing the following tests.

• First, I tried it and after the first reboot only one of the three protonvpn tunnels came up with proper handshake, all other 4 tunnels did not (despite instances and peers were all marked as "enabled" in the GUI.
  
• 
  Second test, I disabled them all, and disabled wireguard, rebooted and I had the same situation.
  
• 
  Third test, I stopped every instance with configctl wireguard stop XXX, then rebooted, and once the router came back I re-enabled all the VPN tunnels, which brought me back to having the 3 protonvpn tunnels working (i.e. proper handshake), and remaining 2 HomeWireguard tunnel (for accessing from the road) and s2s tunnel (to connect across geographically separated lans), not working.

Not sure if this is helpful, but in the logs, I see these 2 "errors" messages related to the s2s tunnel:

• /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: The command '/sbin/ifconfig 'wg4' 'inet' '10.2.2.1/24' alias' returned exit code '1', the output was 'ifconfig: ioctl (SIOCAIFADDR): File exists'


• /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: The command '/sbin/route -q -n add -'inet' '10.2.2.2/32' -interface 'wg4'' returned exit code '1', the output was ''

and I see this "notice" message for every interface I have associated with my 5 tunnels:

• wireguard instance HomeWireguard (wg2) can not reconfigure without stopping it first.
  

Thank you, I did try the patch in the past couple of days and it did not work. I attempted to reboot, disable and re-enable all wireguard instances from both the GUI and the terminal, individually or in group..

Only 3 out of 5 tunnels now work  (with or without the patch), which almost always happen to be the 3 protonvpn ones. The HomeWireguard and the s2s tunnels seem to never come up anymore.

I reverted the patch, and I experience the same behavior.

There seem to be a reproducible race condition when multiple wireguard tunnels are started, which lead to blocking startup after 2-3 tunnels come up. 

What should I do next to investigate this further and provide details to help the resolution?

@franco, thank you for sharing the github thread, I am following it. The problem manifests differently, but it seems indeed related.

What is the most efficient way to help with debugging this? My config is long but not necessarily complex, I am happy to provide details on any area you would like to dive in.

[Background]

Hey folks,

I think I may have found a malfunction in OPNSense (I am running 24.7.3_1), but wanted to validate with you first.

Background: I have successfully configured my system to have 3 vpn tunnels pointing at three different ProtonVPN locations for policy based routing on three of my subnets. I also configured a 'Home' wireguard tunnel to allow me to connect to my firewall when I am on the road. Lastly, I have configured a s2s wireguard tunnel with another router I manage in a different location (the two sites have non overlapping LANs).

The fun part: I tested all these 5 tunnels and they all worked as intended, once configured and for many days since then...until I rebooted the firewall (clean reboot from the GUI).

Observations:

• After reboot, only 2 out of the 5 tunnels came back alive, and interestingly not always the same 2 across several reboots.
• I tried to disable and re-enable wireguard in the GUI and from the terminal, but only the 2 tunnels that came alive at boot restarted.
• I disabled all the instances and peers (in the GUI), I rebooted the firewall, and I manually re-enabled each one of the 5 tunnels (instances and peers), but again only 2 of them (not always the same ones) came back alive.
• the only messages I can find in the vpn logs concerning the  tunnels that do not come up look similar to the following: " wireguard instance HomeWireguard (wg2) can not reconfigure without stopping it first."

Are you aware of what could be causing this? Has someone else experienced something similar, or have good leads for me to follow?

Thank you in advance.

@umbramalison, I agree this sounds simpler. In case you wanted to give a try to a setup with NAT rules and virtual ips, you can check my working configuration here: https://forum.opnsense.org/index.php?topic=41534.msg203864#msg203864.

More complicated, but highly educational IMO :)

For future reference, I found out the issue with my configuration.  ;D

I had to disable ipv6 on the WAN interface (I set Interfaces>[WAN]->IPv6 Configuration Type to "None"). It seems that my system's ipv6 configuration (or lack of) was causing the DNS leak. As soon as I disabled it, everything started working as expected.

I wish I had a better understanding of why exactly this caused the leak.. In other words how does OPNSense prioritize firewall rules between ipv4 and ipv6 stacks, when your ISP assigns to your router one address per stack? :o

your second screen shot under DNS is empty..

That is empty by design. I only want to use ProtonDNS for clients on the 13 and 14 subnets. For all other subnets I want to be able to use Unbound as resolver. Do you see a reason why this setup cannot work?

I appreciate the reply, but I am not sure I completely understand. This is certainly due to my ignorance, so please bear with me.

3) Allow TCP Alias_IPS_For_GW1 to ANY DPort Alias-Ports(80,443) GW1
4) Allow TCP Alias_IPS_For_GW2 to ANY DPort Alias-Ports(80,443) GW2

I don't understand this rule. For my setup is the configuration below what you meant?

• action: Allow
• interface: ? ? ?
• protocol: TCP
• source: 10.2.1.1
• destination: ANY
• destination ports: 80,443
• gateway: 10.2.1.1

Which interface should I use from my example? WAN_ProtonVPNDenmark and a separate similar rule for WAN_ProtonVPNItaky? Or LAN?
Note: I am using subnets on the LAN, I do not use VLANs

Why only ports 80 and 443?

1-2)  Port FW rule, TCP/UDP, source (v)LANs, destination ANY destination port 53 - redirect to 127.0.0.1 port 53 (Unbound rule)

This seems a NAT Port forward rule that does two things

• it replaces my two port forward rules specifically doing the same for each of the subnets independently, forwarding DNS requests towards their respective protonVPN DNS (which for Proton happens to be the same for all tunnels).
• it explicitly forwards other DNS requests from the rest of my LAN (not 192.169.(13|14).0) to the local Unbound DNS resolver.

Why do I need to do 2. explicitly? (Sorry again if this is trivial, I am trying to learn.)

More general follow ups to your reply:

• Can you explain what you mean by my configuration having revolving door set of policies?
• Also, how avoiding the use of the ProtonVPN DNS, helps me understand why Unbound seems to take precedence in resolving names for clients in the vpn protected subnets, despite having rules that redirect DNS traffic to the ProtonVPN DNS for those cases. And why things work (or seem to) work as expected when I disable Unbound?
• To your comment about DNS over TLS, I didn't venture into using DNS over TLS yet (it is on the todo list). Given that, I wonder if the issue I am having could be resolved withoutt using DNS over TLS.

Again thank you in advance for all the help, and for going through my config.
I am trying to learn by reading and doing, so I apologize if my questions are too trivial, and if that is the case, feel free to redirect me to existing tutorials that I may have missed, covering a similar setup.

Cheers!
[/list][/list]

[Services-Unbound DNS-General]

Hey folks,

I am a recent user of OPNSense, who needs help with Unbound DNS and its interaction with my two ProtonVPN tunnels.

I have been trying to setup my new router to achieve the following goals.

• protect clients on 192.169.14.0 with one wireguard tunnel to the ProtonVPN endpoint in Denmark.
• protect clients on 192.169.13.0 with another wireguard tunnel to the ProtonVPN endpoint in Italy.
• ensure that all other clients in different subnets can access the internet through WAN *and* their DNS needs are served exclusively by UnboundDNS configured as resolver. I.e. I do not want to use ISP DNSes nor ones coming from Google, or others.
• It is important that clients on both subnets protected by ProtonVPN do not leak the ISP DNS, while accessing the internet.

For 1. and 2. effectively I followed https://docs.opnsense.org/manual/how-tos/wireguard-selective-routing.html#dealing-with-dns-leaks as general setup, and what truly made it work for two wireguard tunnels to ProtonVPN was following both https://www.reddit.com/r/ProtonVPN/s/NrZUVYqARH and what jlficken@ recommended here: https://forum.opnsense.org/index.php?topic=38911.msg195192#msg195192

For 3. I followed https://homenetworkguy.com/how-to/confused-about-dns-configuration-in-opnsense/ for general understanding of how OPNSense offers knobs to configure DNS, and I then proceeded configuring the system based on that understanding.

I seem to be failing in achieving 4. and here are my brief observations:

• when a client connects from an address on 192.169.14.xxx, or 192.169.13.xxx, they are correctly routed through the ProtonVPN endpoint and have access to the internet... but they leak my ISP DNS.
• If I do the same test, but *after* I have manually stopped Unbound in the OPNSense UI (pressing the "Stop" button), then I do not have leaks anymore and dnsleakstest.com correctly lists the ProtonVPN dns.

This seems to suggest that I misconfigured my system (Unbound? Firewall? NAT?), but I do not know exactly how to find nor resolve the issue.
Any suggestions?

My configuration is below, I tried to redact either private information, or experiments that I currently have disabled, to avoid confusion.

Thank you in advance for the help!

Services-Unbound DNS-General


Services-ISC DHCPv4-[LAN]


System-Gateways-Configuration


VPN-Wireguard-ProtonVPN_Denmark_Peer1


VPN-Wireguard-ProtonVPN-Denmark


Firewall-Settings-Advanced


Firewall-Rules-LAN


Firewall-Rules-Floating-WAN_ProtonVPN_Denmark-Routing


Firewall-Rules-Floating


Firewall-NAT-Outbound


Firewall-NAT-One-to-One


Firewall-NAT-Port Forward


Firewall-Aliases


Interfaces-Virtual IPs


Interfaces-Overview


System-Settings-General