Messages

May this is a similar issue I have too: https://forum.opnsense.org/index.php?topic=35707.0

Thanks for the hint I will try and repport back. I set it to 8.8.8.8. Would be nice if there is an option to set multiple external Monitor IPs and just do something when all of them are down. Otherwise a service outage can impact your internet connectivity as well.

[The Setup:]

Hi

The Setup:

OPNSense (VM) -> ESX Host -> Switch (Cisco) -> XGS-PON Bridge (Nokia) -> WAN FTTH

Everything here is powered by UPS.

Problem:

As of some re-wireing stuff I've disconnected my XGS-PON Bridge Power & Fiber. I've replaced it and then reconnected it again.
After re-wireing was finished I had no WAN connectivity at all. So I logged in to my opnsense FW to check whats happened.
I saw, that the WAN interface was stuck without IPv4 address with just the IPv6 address assigned.
But no connectivity at all. So I opend "Interfaces -> Overview -> WAN" and klicked "Release/Renew". After that everything was working again.

I assume that the problem is, as the WAN link never went really down in the view of opnsense. As it runs as a VM, the NIC was never disconnected. So may it never really noticed that the link was down and a new DHCP request is needed.

Any Idea how this could be solved propperly?

Okay, thanks for the info. Didn't found anything on the internet about that.

Code Select Expand


root@fw:~ # pkg info | grep wire
os-wireguard-1.11              WireGuard VPN service
wireguard-go-0.0.20220316_3,1  WireGuard implementation in Go
wireguard-kmod-0.0.20220615    WireGuard implementation for the FreeBSD kernel
wireguard-tools-1.0.20210914_1 Fast, modern and secure VPN Tunnel


Code Select Expand

root@fw:~ # opnsense-revert wireguard-kmod
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
The following packages will be fetched:

New packages to be FETCHED:
        wireguard-kmod: 0.0.20220615 (40 KiB: 100.00% of the 40 KiB to download)

Number of packages to be fetched: 1

40 KiB to be downloaded.
Fetching wireguard-kmod-0.0.20220615.pkg: 100%   40 KiB  40.6kB/s    00:01   
wireguard-kmod-0.0.20220615: already unlocked
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be REINSTALLED:
        wireguard-kmod-0.0.20220615

Number of packages to be reinstalled: 1
[1/1] Reinstalling wireguard-kmod-0.0.20220615...
[1/1] Extracting wireguard-kmod-0.0.20220615: 100%

Code Select Expand

root@fw:~ # kldstat
Id Refs Address                Size Name
1   51 0xffffffff80200000  215db98 kernel
2    1 0xffffffff8235e000     4b58 if_enc.ko
3    1 0xffffffff82363000    181d0 if_lagg.ko
4    2 0xffffffff8237c000     3538 if_infiniband.ko
5    1 0xffffffff82380000     e318 pfsync.ko
6    3 0xffffffff8238f000    741a8 pf.ko
7    1 0xffffffff82404000     3b18 pflog.ko
8    1 0xffffffff82408000     ba48 if_gre.ko
9    1 0xffffffff82414000     e4d0 if_bridge.ko
10    2 0xffffffff82423000     7870 bridgestp.ko
11    1 0xffffffff8242c000     f460 carp.ko
12    1 0xffffffff82a11000     3530 fdescfs.ko
13    1 0xffffffff82a15000     4480 vmmemctl.ko
14    1 0xffffffff82a1a000     4b58 vmblock.ko
15    1 0xffffffff82a1f000     3218 intpm.ko
16    1 0xffffffff82a23000     2180 smbus.ko
17    1 0xffffffff82a26000     7490 vmci.ko
18    1 0xffffffff82a2e000    34568 if_wg.ko
19    2 0xffffffff82a63000    27048 ipfw.ko
20    1 0xffffffff82a8b000    12520 dummynet.ko

Interesting is...

- uninstall wireguard-kmod
- opnsense-revert os-wireguard
- reboot
- reinstall wireguard-kmod
- reboot

now it works again as expected. Strange :(
Sorry for disturbing.

Hi

Just updated to 22.7 and noticed that wireguard-kmod is not working anymore.
As I found on google it needs a rebuild/recompile on 13.1 as the kernel module is not compatible anymore.

Anyone alreday did that? Any how to do this?

BR

Hi

I'm running opnsense since a long time and using dynamic DNS with cloudflare to update my DNS record to the current public IP.

Unfortunately my FW can not directly connect to the providers network so I have to use the given ISP router in front of my opnsense.

Since the internet speed is not so good, the ISP got me a LTE Extention to the ISP router.
All TCP traffic that reaches the router will be balanced between both links. (Multipath TCP)

The problem I now facing in is that dyndns with cloudflare uses the public source IP to define the dynDNS record.
As the TCP https requests is also balanced I randomly get the public IP of the cellular network on the dynDNS record. This doesn't work as it uses carrier grade NAT.

So I was looking for another solution. I found out, that with a simple curl command in bash I can ask the router for the current public IP on the DSL.

My question now is: Is it possible to use some curl command to get information of the public IP first and use this IP to update dynamic DNS record?

I do see a similar/same issue.

Upgraded to 27.1.3 and can see very high cpu load. python3.8 consumes around 60-90% CPU all the time. I rebooted the VM but it didn't help.

I reverted to the VM snapshot I did before the upgrade an CPU load is back to normal.

can you please check again what kind of requests the unbound forwards  exactly?
for example, the request for

Code Select Expand

server1.internal.my-domain.tld.my-domain.tld
should be forwarded. unbound does not own "tld.my-domain.tld" zone

I can do... but:

tld.my-domain.tld is not a local-zone, this is correct. But why it searchs for this? "server1.internal.my-domain.tld is local-zone and there is local-data for this host.

But one example (there are multiple):

Code Select Expand

real hostname: accesspoint1.internal.my-domain.tld
dns request on public resolver: accesspoint1.internal.my-domain.tld.my-domain.tld

The client who has made this DNS query is my monitoring system where hostname is specified as "accesspoint1.internal.my-domain.tld"

a more radical way is also possible if there are few records for the domain on the external servers. in this case, you can take the entire "my-domain.tld" domain to static local-zone and add the required records to the host override

This is not possible as some of them are dynamic and are changes by cloud LB automaticly. In this case I had to change them on opnsense every time it changes in the cloud.

I put it in the "Custom options" field in the WebGUI.

But may it would be easier to explain what I want to realize and may we will find another way.

I've a TLD (e.g. my-domain.tld). This domain is used in the internet (e.g. public.my-domain.tld).

I use the subdomains internal.my-domain.tld and dmz.my-domain.tld for local systems (behind opnsense).

So my idea was to put it as local-zone in the "Custom options" field like that:

Code Select Expand


local-zone: "internal.my-domain.tld" static
local-zone: "dmz.my-domain.tld" static


Then I added Host Overrides for each server in the Overrides section of unbound.

All of this is working fine. I can resolve the names local by using unbound on opnsense and I can't resolve the local names when using the internet DNS. (which is expected). -> OK

Beside this I've configured unbound to forward all to a DoT DNS server.
On this DoT-DNS server I can see in the logs, that unbound tries to resolve names for internal devices like:

Code Select Expand


server1.dmz.my-domain.tld.internal.my-domain.tld
server1.internal.my-domain.tld.my-domain.tld
server1.internal.my-domain.tld.dmz.my-domain.tld
google.com.my-domain.tld


And now I want to find out why unbound tries to resolve such strange names?
Specially when server1.dmz.my-domain.tld is already a valid one and unbound had local data (host override) for?

In the end I just want that everything with "internal.my-domain.tld" and "dmz.my-domain.tld" NEVER is forwarded to an internet DNS server.

so that requests to addresses for these domains are not forwarded imho you need to make local-zones with the "static" type for this domains (looks like there is no gui param for making this for domains from DHCP "domain search list" option. for System Domain it generates local-zone with "local zone type" from unbound general settings. ie at least for the system domain, you can set the zone type to 'static' in Services: Unbound DNS: General)

I've already:

Code Select Expand


local-zone: "domain.tld" static
local-zone: "otherdomain.tld" static
local-zone: "use-application-dns.net" always_nxdomain


But this doesn't help and I still see that requests are made to things like

Code Select Expand

googleapis.com.domain.tld :(

This is why I'm confused as I thought that set it a local-zone = static should not forward it and only get a result when local-data is configured.

I really don't know what happens there :/

Okay.... I now dumped around 30 mins of DNS traffic on 3 different VLANs and looks like you're right.
I can see that some clients resolving such sh**:

Code Select Expand


local-server1.domain.tld.domain.tld
www.googleapis.com.domain.tld


Yes I defined a Domain for DHCP clients and a search list. And also a static one in resolv.conf when not DHCP is used.

hi
just add closing "." when checking via nslookup )

The problem dosn't occour just on "nslookup" ... I can see more than 1k of such DNS requests from different hosts.
And this does only happen with unbound. If I test by just using dnscrypt-proxy is doesn't happen anymore. So unbound adds the additional (second) domain.tld.

So it is not the DNS client which does request this... as when dnscrypt-proxy is running on port 53 you can see that this never happens.

When I use unbound and forward to dnscrypt-proxy I can see that porblem again with host1.domain.tld.domain.tld

I configured my unbound & dnscrypt-proxy as described in the first post.

Unfortunatley it doesn't work right.
While some domains are working fine others can't be resolved at all.

query.log of dnscrypt-proxy shows like this:

Code Select Expand


[2021-07-12 16:46:13]   127.0.0.1       office365.com   DS      PASS    11ms    NextDNS-Primary
[2021-07-12 16:46:13]   127.0.0.1       office365.com   DS      PASS    0ms     -
[2021-07-12 16:46:13]   127.0.0.1       office365.com   DS      PASS    0ms     -
[2021-07-12 16:46:13]   127.0.0.1       office365.com   DS      PASS    0ms     -
[2021-07-12 16:46:13]   127.0.0.1       office365.com   DS      PASS    0ms     -
[2021-07-12 16:46:13]   127.0.0.1       office365.com   DS      PASS    0ms     -
[2021-07-12 16:46:22]   127.0.0.1       community.librenms.org  A       PASS    0ms     -
[2021-07-12 16:46:22]   127.0.0.1       librenms.org    DS      PASS    0ms     -
[2021-07-12 16:46:22]   127.0.0.1       librenms.org    DS      PASS    0ms     -
[2021-07-12 16:46:22]   127.0.0.1       librenms.org    DS      PASS    0ms     -
[2021-07-12 16:46:22]   127.0.0.1       librenms.org    DS      PASS    0ms     -
[2021-07-12 16:46:22]   127.0.0.1       librenms.org    DS      PASS    0ms     -
[2021-07-12 16:46:22]   127.0.0.1       librenms.org    DS      PASS    0ms     -
[2021-07-12 16:46:22]   127.0.0.1       beacons.gvt2.com        A       PASS    0ms     -


As you can see the first request was working while all the others end with "PASS" but with the "-" in the end of the line instead the selected DNS-Server profile.

Unbound custom options:

Code Select Expand


server:
tls-cert-bundle: "/etc/ssl/cert.pem"
local-zone: "use-application-dns.net" always_nxdomain
do-not-query-localhost: no
forward-zone:
name: "."
forward-addr: 127.0.0.1@5354


dnscrypt-proxy is listening on 127.0.0.1:5354 and does get the requests forwarded by unbound, so this shouldn't be the issue.

Any ideas why this does happen?

Edit:
I can solve the issue by uncheck "Enable DNSSEC Support" in unbound settings.
I do not like to disable DNSSEC support. So is there a other way to get it working?

Hi

I noticed a strange behavior with unbound when I checked my upstream DNS logs.

Unbound tries to resolve names like:

Code Select Expand

host1.localdomain.tld.localdomain.tld

or

Code Select Expand

google.com.localdoman.tld

So... I don't get why this happens. :(
I already tcpdumped on some of my servers to check if the server does such reqeusts but they look all fine.
So for me it looks like unbound does add the "localdomain.tld" to a FQDN.

Any hint is likly welcome!


Edit:
I guess it depends as I added the following lines to unbound custom config to use DoT service:

Code Select Expand

forward-zone:
  name: "."
  forward-tls-upstream: yes
  forward-addr: x.y.z.a@853#dnsserver.domain.tld

For me it is still unclear why this happens.
Why unbound forward "host1.localdomain.tld.localdomain.tld" to the upstream DNS even there is a local entry for "host1.localdomain.tld"

If I used Opnsense for prod , I would have CERPed it. Snapshoting the memory is quite pointless.

It just doesn't matter as this issue occours on ANY FreeBSD 12.1-RELEASE and not just opnsense. As described in the issue linked above.

So it may be pointless for your usecases but this may not be true for everyone else.