Messages

Hi,

I am just going to think out loud some ideas to find out more about the issue... Apologies if you have already done all this testing.

Does the issue happen almost consistently when pinging different servers?

Yes, it happens regardless of server.

Have you tried connecting the PC directly to OPNsense to see if the issue persist? (since you said there is a switch in the middle).

Do you experience the same from the router itself? Try to do a ping (with a large count) from OPNsense "Interfaces - Diagnostics - Ping" and see what you get.

Great idea.  Hadn't tried this, and appreciate you providing the instructions on how to do it.  I ran 10 consecutive tests each with 10 pings.  No degradation at all.

Your thoughts then prompted me to perform some other tests.
I hardwired a PC direct to the OPNsense box.  Ping was just fine, for 3 minutes.
Then I ran Speedtest.net at the same time as ping, and the problem started appearing again.
Seems the issue is just when under load!

So, hooked the PC direct to modem.  Ran the same speedtest & simultaneous ping test.  Got the same result.  A few 34ms pings, then BAM, 1427ms.  When I had previously hooked the PC direct to modem, there was no download activity.

That suggests to me it is either the modem, or the the ISP and not OPNsense at all.  Which makes sense since I hadn't changed the config in OPNsense prior to the issue commencing.

Thanks for your pointers verasense!

Hi folks.

Running the latest OPNsense (all updated) on an i3 2nd gen.  CPU & RAM use always low on this device as it just runs a couple of VPN connections for routing traffic.

Over the last 3 weeks or so, I've been getting problems with glitches which I can see when I run a ping test.

If you check out the ping from Google below, you'll notice average times in the 36ms range.  This would be expected.  But you'll also see multiple high latency returns, right up to over 1000ms.

Code Select Expand

64 bytes from sea09s29-in-f14.1e100.net (142.250.217.78): icmp_seq=20 ttl=117 time=35.7 ms
64 bytes from sea09s29-in-f14.1e100.net (142.250.217.78): icmp_seq=21 ttl=117 time=33.8 ms
64 bytes from sea09s29-in-f14.1e100.net (142.250.217.78): icmp_seq=22 ttl=117 time=36.1 ms
64 bytes from sea09s29-in-f14.1e100.net (142.250.217.78): icmp_seq=23 ttl=117 time=36.0 ms
64 bytes from sea09s29-in-f14.1e100.net (142.250.217.78): icmp_seq=24 ttl=117 time=1141 ms
64 bytes from sea09s29-in-f14.1e100.net (142.250.217.78): icmp_seq=25 ttl=117 time=87.0 ms
64 bytes from sea09s29-in-f14.1e100.net (142.250.217.78): icmp_seq=26 ttl=117 time=41.6 ms
64 bytes from sea09s29-in-f14.1e100.net (142.250.217.78): icmp_seq=27 ttl=117 time=34.5 ms
64 bytes from sea09s29-in-f14.1e100.net (142.250.217.78): icmp_seq=28 ttl=117 time=523 ms


This is an issue that only recently started, and is affecting all traffic, even pure direct-to-WAN (VPN bypass).

This recurring and persistent problem is affecting VoIP calls and my kid's internet gaming is apparently suffering considerably.

I hooked a PC direct to modem, and the issue disappeared(45 minute test), so I know it is happening through OPNsense (or possibly my D-Link switch).  This is with devices that are hardwired.

Any thoughts on how I could begin to troubleshoot?
Thanks!

It's a little hard to make suggestions on this without understanding how your network DNS is set up (local resolver in the network?) and how IPs of the clients you are sending down the tunnel relate to the DNS servers (same subnet?).

Opnsense is handling all the DNS and DHCP for the network.

One alternative that might be easier though is to create static leases in OPNsense for the relevant clients (if you haven't already), and specify the VPN provider's DNS servers in the static lease configuration (see under Services/DHCPv4/[Interface Name]/DHCP Static Mappings for this interface at the bottom). This should mean that those clients are provided the VPN provider's DNS servers by DHCP, rather than the system's general DNS servers, which hopefully overcomes the issue (because the traffic to those public DNS servers should be going down the tunnel if the rest of the configuration has been set up correctly).

That is basically the reverse of what I'd like to do.
I would like to have all DHCP clients use the VPN DNS servers by default, and the static leases (of which there are just a few) use a public DNS server.

Way less configuration if it can be done that way...

I am trying to do URL-based routing.  Perhaps what I am attempting is not quite possible.

For any given client, if you enter CNN.com you will be routed via the US VPN gateway.
On the same client if you enter bbc.com you will be routed via the UK VPN gateway.
For all other destinations, routing is down on the default VON gateway unless the client device is in my exclude list.
The exclude list is a direct WAN connection to the ISP.

The goal is to enable automatic routing behind the scenes.  It is working for basic websites, but not the streaming platforms.
I feel like I'm so close...

Streaming can be a tricky one, you may need to use PIA DNS servers, as PIA do some DNS tricky to get streaming services working. But if you look in the regions list you'll see uk_2 which is described as a streaming optimised region, so you may need to use that region id in your setup, failing that PIA DNS maybe required.

Cron Jobs should be pretty easy.

Make sure your action file contains actions for all 3 setups, then reload the configd. Then the 3 actions will appear in the cron section of the webui.

Yes, I just confirmed that the actions file has all 3 setups contained within it.
There are 3 distinct entries in Cron now as well.  So I guess that's good.

I have selected the uk_2 server for British streaming.

I think you might be onto something with the DNS holding me back.
Ideally I would like to capture the DNS and route appropriately for each tunnel, although it looks as though all tunnels utilise the same internal IP addresses for DNS.

I am attempting to make sense of https://docs.opnsense.org/manual/how-tos/wireguard-selective-routing.html#dealing-with-dns-leaks in order to route the DNS appropriately.  In fact I had started a separate thread about it a few days ago at https://forum.opnsense.org/index.php?topic=24416.0, but as yet don't have it working.

You're really on the ball - thanks for monitoring this thread so closely!

A little more playing around and I currently have all gateways up and running again.

One of the issues had been that I was editing the PIAWireguard.py file on my desktop, and had forgotten to copy it back across to Opnsense.  Schoolboy error I'm afraid  :-[

I'm now showing 3 different public IP addresses.

Before going any further I'll see if I can get the Cron jobs set up, then I'll look a bit more closely at my firewall rules.  Still trying to get the hang of those, but so far the UK streaming sites are not working for me.

Now I seem to have messed it all up.
Had a brief server crash, and it obviously did some kind of reset (asked me to send a crash report).

I was still doing setup and hadn't done the cron thing.
I went back and added the original PIAWireguard.py file, with the relevant changes.

However when I run PIAWireguard.py debug, I keep getting
wireguardserver addKey request failed non 200 status code - Trying to add instance public key to server in exchnage for connection information

Not sure what that means, and I can't find other errors.  The other gateways (US & UK) came up fine when I ran the scripts and readded the gateways etc.

Any suggestions?

I highly recommend you make sure you have the actions setup for both the new and old tunnels, and have the cron job setup.

For some reason I had not been able to set up a Cron job originally.  I had attempted, but I think something was missing in one of the dropdown boxes.

I'm currently adding the 3rd WG connection.  Once I have it running, I'll get back to the Cron job.

Although I'll have 3 WG gateways, the .py file will only be aware of 2 of them, since I had obviously set the first up a different way.  I guess I might have to add the initial wg0 info manually to that file.  Will experiment as the day goes on!

[*EDIT*]

Thanks for the fast and informative reply.  This is super helpful!

When I started working through these instructions, I noticed the actions file in action.d was missing.
I guess I had originally got WG up and running with another method found elsewhere.  I don't even remember which set of instructions I followed it seems!

Anyway, I copied the file over into actions.d and removed the first entry since I must not have used it.  So it starts with the PIAUS entry.

However when I typed in configd restart, I got Command not found in the shell.

*EDIT* - I did service configd restart as per the original instructions! 
I copied and amended the previous Interface entry and updated the name.
Added the Gateway.
ran the script with debug changeserver
Enabled the Gateway, saved, and it was up and running.

I added an Alias to route certain URLs via this gateway (including 1 IP address checking site), and BINGO!  All working  :)

Many thanks for the added help & support!

https://github.com/FingerlessGlov3s/OPNsensePIAWireguard

Any question just ask and any issues make an issue on Github.

**Looks around slowly and raises hand sheepishly after several months of inactivity in thread...**

I used this script to set up a PIA WG tunnel . Worked  great  ;D

Now I am hoping to set up multiple WG tunnels.
How would I go about this?  The idea is a tunnel for US & UK in addition to my existing tunnel.

If I run the primary script again, will it break the existing connection?  Do I need to go right back and create a new API for example, or can I start further along in the process?

Have you set up a port forward? Are you VPN provider's DNS public or private IPs?

I attempted, without success, to set up a port forward.  Well, I set one up, it just didn't seem to work correctly.  When I ran a leak test I got weird IP ranges back, and not my VPN provider's public IP address.

The VPN provider uses public IP addresses.

I'm still having no luck with routing the DNS queries correctly.

If there is anyone who knows how to implement #2 from the following link and could provide a brief tutorial, I would be deeply appreciative!

https://docs.opnsense.org/manual/how-tos/wireguard-selective-routing.html#dealing-with-dns-leaks

I added VPN_Required URLs list alias to outbound NAT rule, and still having same issue.

Resurrecting an old thread to say THIS was the key to having URLs bypass the VPN for me.
I had created an alias list of URLs, and placed it in the Firewall Rules section, but had not generated a NAT Outbound rule.

Having searched the forum and found this post, it is now working for me (on simple URLs at least).
Thanks for the tip  :)

[Edit :]

See these suggested solutions: https://docs.opnsense.org/manual/how-tos/wireguard-selective-routing.html#dealing-with-dns-leaks

Thanks.  I had been studying that document, but was unsure if how to apply it.
Probably #2 is the option I want.  That way I could ultimately route non-VPN queries via public secure DNS, and VPN connected clients to the VPN's DNS servers.  However I'm not sure whether that can all be done on a single rule.

Right now I have a DNS rule set up (or name queries don't work at all).  It is a LAN rule that passes any TCP/UDP on interface LAN to Destination "This firewall" on Destination port DNS.

So would I need to create a quick rule placed above this to capture DNS queries from source "MyVPN Alias list" with Destination "WG_VPN_Tunnel"?  If so, where would I set the VPN provider DNS server addresses, or would that not be necessary?

Edit : tried this and it didn't work.

Edit 2 :  Created an Alias for the VPN DNS server IPs.  Set up a rule on LAN to match my VPN Alias and set destination as VPN_DNS alias using the VPN gateway.  DNS queries still going to the public DNS server :-(

[Problem :]

Good day everyone.  I've been searching for some answers and guides, but haven't stumbled on the correct solution yet it seems :-(

I set up OPNSense a few days ago, and I'm loving it. 
I set up a WG tunnel with my commercial VPN provider, and managed to find enough forum posts to help me figure out most of the firewall settings.  Hooray.

I've now got an Alias set up that routes specific IPs through the WG tunnel.  Other hosts remain on the standard WAN connection.

Problem :
The devices routing over the tunnel are not passing DNS leak tests.  They are showing the DNS address of Cloudflare (the public DSN provider I selected).

Question :
Is there a simple way to route DNS requests to my VPN provider for those devices that are part of the WG group and using the VPN IP address?