Messages

1

20.1 Legacy Series / Controlling outbound routing of DNS forwards from opnsense box

« on: July 08, 2020, 03:56:53 am »

Hi,

I have a VPN that, when up, I want all traffic to flow over.  When the VPN is down, traffic should flow across the WAN like usual.

I achieve this normally with a Gateway Group.  It works fine for the NAT traffic from the LAN.

However, with unbound, it's a challenge.  It wants to send traffic out the WAN interface.  I can specify multiple nameservers, but it will always send queries to each one, creating a data leak when the VPN is up.

How can I force the DNS queries out the VPN - but only when it's up?  I've tried various rules and none of them have done the right thing.

Thanks!

2

20.1 Legacy Series / Re: Improper handling of packet fragmentation

« on: May 07, 2020, 08:06:00 pm »

Yes.  Interface assigned, gateway assigned, and a standard NAT and route.

I wonder if part of the challenge here is that the VPN provider here assigns a /32, so both the from and to address on the ICMP packet are the same, and to the firewall it might look as if it originated on the firewall itself?

3

20.1 Legacy Series / Re: Repeated crash/reboot when Wireguard port number is wrong

« on: May 07, 2020, 05:52:21 am »

For the benefit of anyone else looking here, the workaround for now is to disable shared forwarding.

References:

Github bug https://github.com/opnsense/src/issues/46 "Bootloop and kernel exception with trace while configuring wireguard"

Wireguard issues https://forum.opnsense.org/index.php?topic=14403

pf+route-to: panic when shared forwarding is enabled https://github.com/opnsense/src/issues/52

Possible kernel bug https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=233955

4

20.1 Legacy Series / Improper handling of packet fragmentation

« on: May 07, 2020, 02:42:40 am »

Hello,

While troubleshooting an issue with slow TCP startup, I did some tests from a Linux box using tracepath and ping -Mdo.  None of these tests made it past the openvpn box.  The route out is over a Wireguard VPN with a MTU of 1420. 

Regular pings work.

tcpdump from opnsense on the wireguard interface showed:

19:39:01.863080 IP (tos 0x0, ttl 64, id 7009, offset 0, flags [none], proto ICMP (1), length 56)
    x.x.x.x > x.x.x.x: ICMP 8.8.8.8 unreachable - need to frag (mtu 1420), length 36
   IP (tos 0x0, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 1500)
    x.x.x.x.56499 > 8.8.8.8.44444: UDP, length 1472

However, this packet never made it back out the LAN interface.

If I add a normalization rule for "do not fragment" on the Wireguard interface, then the tracepath proceeds normally; however, it also fails to detect that the path MTU is less than 1500, which no doubt will result in a mess of fragmented packets.

Is there any solution here?

5

20.1 Legacy Series / Re: Repeated crash/reboot when Wireguard port number is wrong

« on: May 06, 2020, 09:51:50 pm »

Some more details - any issue with the Wireguard link appears to cause this.

I have the setup documented here - Wireguard peer/local defined, an unused RFC1918 as the gateway IP, locked interface for it, gateway for it, and NAT/routing rules about it.

After several crashes, I tried disabling the NAT/routing rules and the interface but this did not resolve it.

Outright deleting the gateway and interface did resolve the issue.

This is OPNsense 20.1.6.  RAM and disk do not appear to be culprits here.

6

20.1 Legacy Series / A few questions from a PFSense convert

« on: May 06, 2020, 02:55:24 pm »

Hi everyone,

First, thanks for OPNsense!  I finally made the switch from pfsense, for reasons you all are probably aware of, in addition to the Wireguard support in opnsense.  There are a few things that are still puzzling me:

1) What are the "periodic backups" in settings -> miscellaneous?  Does this imply that the RRD is normally only in RAM and not on disk?  (Same for Netflow)  If I set the RRD backup to, say, 6 hours, will it also be backed up on power off?  Where is it being backed up to?  Was it on disk to begin with?  (I'm puzzled because RRD normally would be on disk anyhow)

2) Is there a way to restore a config.xml file at install time?  Eg, by copying it to a spot on the USB drive?

3) It appears that restoring a config.xml file -- although it does record the list of plugins used -- doesn't reinstall the necessary plugins.  Is there a way to do that?

4) I want to maintain a ssh connection from opnsense to a remote box, which uses -R to permit a port forward from another secure machine.  On pfsense, I used the Filer plugin, which lets me define a file to be installed on the system, which is saved as part of config.xml.  I used it to define a shell script that would make sure that the SSH keys were set, and acted as a watchdog for the ssh process.  Then I used its cron plugin, which would run the shell script periodically to keep the tunnel up.

opnsense has cron built in, but Filer isn't, and it doesn't support an arbitrary command.  Sites like https://forum.opnsense.org/index.php?topic=2263.0 and http://kb.unixservertech.com/other/networking/opnsense/cron-jobs advocate adding a file to /usr/local/opnsense/service/conf/actions.d/, restarting configd, and then using this in the cron GUI.  I would presumably drop my script under /usr/local or something.

Questions about this:

A) Does this stuff get backed up in config.xml?  (the file under actions.d)  If no, then would the missing service imply a problem for restoring a config.xml backup?

B) Would this file, or my script, get blown away on reboot?  On upgrade?  Is there any way to persist it (and my script)

7

20.1 Legacy Series / Repeated crash/reboot when Wireguard port number is wrong

« on: May 06, 2020, 02:43:14 pm »

Hi folks,

This morning, I wanted to test my opnsense firewall's failback procedures when my Wireguard tunnel goes down.  To do this, I edited the peer configuration for the Wireguard tunnel and added 1 to the remote port number, breaking the configuration.

My test was all good, and when I went to set the port number back, the firewall crashed.

It rebooted, and from then on, its uptime would be measured in minutes, followed by another crash.

I eventually went over to it and plugged in a VGA monitor to see what was going on, since there was nothing in the logs.  Immediately before the crash, a ton of bold text flowed by really fast.  Some sort of kernel crash messages perhaps?  I don't know where to find these on FreeBSD.

Eventually I noticed that the initial crash had happened before the fix to the Wireguard port number was committed to disk.  I fixed that port number again, and then the crashes stopped.

As Wireguard isn't even in-kernel on FreeBSD, I have no idea how this could possibly have led to this sort of crash, but yet it has.

How can I help debug this?

(I used FreeBSD *way* back but most of my more recent experience is on Linux)