Topics

1

22.1 Legacy Series / [SOLVED] CARP switches to BACKUP frequently since 22.1.10 Upgrade

« on: July 27, 2022, 12:13:01 pm »

Since upgrading to 22.1.10 last friday, our 2-Node OPNSense CARP Cluster switches the Master node to BACKUP for no apparent reason.
The slave node takes over on all six configured interfaces. The Master shows an error as in the attached screenshot. By klicking on "Enter persistent CARP Maintenance Mode" twice I can switch back as expected.

The cluster has been running stable with different OPNSense releases for months now, there are no signs for hardware errors.

• has anyone else noticed CARP errors with 22.1.10, that where not there before upgrading ?
• Is there a CARP logfile I can have a look at to find out, what's happening ?

2

21.7 Legacy Series / QEMU-Guest-Agent causes lots of zombie-processes, OPNSense stalls

« on: November 21, 2021, 11:30:57 pm »

Im running half a dozen virtual OPNSense Gateways (21.7.4) under oVirt using QEMU-Guest-Agent.
The plugin qemu-ga became available within the 21.1 release.

After roughly 60 days without reboot these gateways stall, the Web-GUI as well as the console show "cannot fork" errors and it is impossible to login via console or ssh.

To find the culprit I had the smart idea, to reboot an affected OPNSense gateway from oVirt using qemu-ga and did a

Code: [Select]

ssh root@mygateway ps uxawj
very quickly during reboot to show the process list.

There were several thousand zombie-processes starting at various dates with PPID 8995

Code: [Select]

...
root 182 0.0 0.0 0 0 - Z Fri05 0:00.01 <defunct> 8995 8995 8995 0
root 192 0.0 0.0 0 0 - Z 4Nov21 0:00.00 <defunct> 8995 8995 8995 0
root 215 0.0 0.0 0 0 - Z Fri04 0:00.00 <defunct> 8995 8995 8995 0
root 249 0.0 0.0 0 0 - Z 13Nov21 0:00.00 <defunct> 8995 8995 8995 0
root 253 0.0 0.0 0 0 - Z 3Nov21 0:00.00 <defunct> 8995 8995 8995 0
...

Luckily the process with PID 8995 was still alive

Code: [Select]

root 8995 0.0 0.4 17236 4052 - Ss 29Oct21 2:48.09 /usr/local/bin/q 1 8995 8995 0

The output is truncated, but on this OPNSense the only binary beginning with "q" in /usr/local/bin is qemu-ga.
I think, the problem is described for FreeBSD on GitHub

https://github.com/aborche/qemu-guest-agent/issues/17

and there already seems to be a fix for it

https://github.com/aborche/qemu-guest-agent/commit/71edc56b1476bf6c45d1d461bbfa9fe987a8974e

Unfortunately this fix hasn't made it into FreeBSD and therefore in OPNSense yet.
I was able to fix the problem for me by disabling the guest-get-fsinfo RPC call in OPNSense configuration, see attachment.

Just in case, if somebody else is running into the same problem

3

German - Deutsch / [gelöst] UnBound DNS Problem mit web.impfnachweis.info

« on: June 26, 2021, 06:18:22 pm »

Die DNS Adresse web.impfnachweis.info wird aktuell von deutschen Arztpraxen zum Ausstellen digitaler Covid19 Impfzertifikate verwendet.
Über Public Nameserver (getestet mit 1.1.1.1, 8.8.8.8 und 9.9.9.9) ist die Adresse problemlos auflösbar:

Code: [Select]

me@laptop02 ~
$ host web.impfnachweis.info 9.9.9.9
Using domain server:
Name: 9.9.9.9
Address: 9.9.9.9#53
Aliases:

web.impfnachweis.info has address 100.102.17.10

Ich hatte gerade bei einem Kunden das Problem, dass die Adresse von seinem OPNSense Gateway nicht aufgelöst wurde und habe ein wenig getestet:

• UnBound auf OPNSense >=21.1.6 löst die Adresse nicht auf, getestet auf 4 Gateways
• Unbound auf OPNSense <= 21.1.2 löst die Adresse korrekt auf, getestet auf einem Gateway
• DNSSEC abzuschalten, macht's nicht besser

Es geht nur um die DNS Auflösung, die Website https://web.impfnachweis.info ist dann nur aus der Arztpraxis via VPN über die Telematik Infrastruktur erreichbar.

Ich habe das Problem bereits im englischen Forum unter https://forum.opnsense.org/index.php?topic=23694.0 gepostet und frag jetzt hier nochmal nach, weils ein "typisch deutsches" Problem ist 

Mein Hotfix ist bisher, einen Override für impfnachweis.info zeigend auf einen Public DND Server zu setzen, das funktioniert.
Hat jemand das gleiche Problem beobachtet und vielleicht eine Lösung ?

4

21.1 Legacy Series / Strange UnBound DNS problem with web.impfnachweis.info

« on: June 26, 2021, 05:24:40 pm »

I just stumbled across a strange DNS problem probably related to UnBound DNS.

A customer has to resolve web.impfnachweis.info, which is used by physicians in Germany to issue digital Covid19 vaccination certificates.

The address resolves fine using public DNS servers

Code: [Select]

me@laptop02 ~
$ host web.impfnachweis.info 8.8.8.8
Using domain server:
Name: 8.8.8.8
Address: 8.8.8.8#53
Aliases:

web.impfnachweis.info has address 100.102.17.10

(same result for 1.1.1.1 and 9.9.9.9)

For some reasons it does not resolve using UnBound on OPNSense 21.1.6 (tested on three different gateways)

Code: [Select]

me@laptop02 ~
$ host web.impfnachweis.info 10.42.1.1
Using domain server:
Name: 10.42.1.1
Address: 10.42.1.1#53
Aliases:

whereas it does on UnBound still using OPNSense 21.1.2

Code: [Select]

me@laptop02 ~
$ host web.impfnachweis.info 10.70.71.254
Using domain server:
Name: 10.70.71.254
Address: 10.70.71.254#53
Aliases:

web.impfnachweis.info has address 100.102.17.10

So far I have tried to restart UnBound, disabled DNSSEC on the affected gateways and increased the Log Level with no effect and no further insight what is happening.
Upgrading one of the affected gateways to OPNSense 21.1.7_1 did also not solve the problem.

The problem can easily be solved by defining an override for impfnachweis.info pointing to a public DNS server, but I would be very interested in what is happening here.

Has anybody experienced this and can provide an explanation ?

5

21.1 Legacy Series / Testing wireguard-kmod on 21.1.4 / problems with re-connecting

« on: April 05, 2021, 02:04:35 pm »

We are running several larger production installations with a virtual OPNSense "central" gateway (cloud based, high speed internet connection >1Gbit/s) and "branch offices" (DSL based provider internet, typically 100/40Mbit/s) connected via wireguard.
The branch offices are using either OPNSense or OpenWRT hardware routers.
Wireguard uses IPv6 for the tunnel and Dual-Stack inside the tunnel.

Two of theese installations I switched to the new, experimental wireguard-kmod driver coming with 21.1.4 whereever OPNSense was used.
This lead to a substantial increase in downstream rate (from central gateway to branch office, typically factor 1.5 - 2.5) and also a slight increase in upstream rate as expected.
The tunnels are running fine without major issues.

The only problem I can see as far:

• After restarting the central (OPNSense) gateway, wireguard on OPNSense branch office routers does not re-connect unless restarting wireguard. Pinging the central gateway does not help.
  On the branch office gateways the last handshake displayed on the "list configuration" tab stays on a time before restarting the central gateway, the central gateway shows no handshake.
  

This worked fine before, when all the OPNSense gateways used the wireguard-go implementation.
The branch offices using OpenWRT (kernel based wireguard on linux) are not affected and re-connect just fine.

Might be a bug in the wireguard-kmod driver

6

21.1 Legacy Series / IPv6 IPSEC tunnel does not pass data in NAT-T mode

« on: March 24, 2021, 12:14:07 pm »

I am trying to establish an IPv6 IPSEC Tunnel between two OPNSense boxes that both have a working IPv4 as well as an IPv6 upstream connection.

When using IPv4 the tunnel works flawlessly.
With IPv6 the tunnel goes up, but transports no data if NAT-T (ESP-UDP Transport via port 4500) is used.
IPv6 and ESP Transport (without NAT-T) works fine.

My first idea was to disable NAT-T (it is usually not needed in an IPv6 setup) by setting "NAT Traversal" to "Disable" in IPSEC phase 1 configuration. The problem is, that disabling NAT-T does not work with strongSwan since Realease 5.0.0 as described here https://wiki.strongswan.org/projects/strongswan/wiki/NatTraversal.
So the option "Disable" seems to be pointless in OPNSense, I opened a github issue on this today.

strongSwan seems to enable NAT-T everytime when MOBIKE extensions are enabled, I found it hard to impossible to get a reliable IPv6-ESP connection without NAT-T when a mobile VPN is configured on the same WAN interface.

Has anyone experienced this behaviour ?
Is it possible that FreeBSD does not support NAT-T with IPv6 ?