***call for testing*** DNS TLS encryption using Quad9 and Cloudflare DNS servers

[elektroinside]

Hmm.. Maybe a packet capture can shed some light what on what is going on there.. have you tried that?

[mjh]

A few weeks back I was having the same cipher-drop issue, since then I've upgraded a few times and seeing this topic start I decided to give DNS over TLS another try. So far this morning it started out great and then I ran into a similar issue as others, the specific error is:

unbound: notice: ssl handshake failed 149.112.112.112 port 853

unbound: [pid:0] error ssl handshake failed crypto error:140020B5:SSL routines:CONNECT_CW_CLNT_HELLO:no ciphers available


I'm running:

OPNsense 18.7.a_186-amd64
FreeBSD 11.1-RELEASE-p8
LibreSSL 2.6.4
Unbound 1.6.8_2

My unbound custom options follows Calomel's Unbound DNS (https://www.calomel.org/unbound_dns.html) pretty closely:


server:
    hide-trustanchor: yes
    harden-large-queries: yes
    minimal-responses: yes
    harden-algo-downgrade: yes
    qname-minimisation-strict: yes
    ignore-cd-flag: yes
    use-caps-for-id: yes

    ssl-upstream: yes

private-domain: "example.com"
private-domain: "lab.example.com"

domain-insecure: "onion"
private-domain: "onion"
do-not-query-localhost: no
local-zone: "onion." nodefault

forward-zone:
    name: "onion"
    forward-addr: 127.0.0.1@9053

# forward-zone:
#  name: "."
# forward-addr:9.9.9.9 #quad9 non-encrypted
# forward-addr:149.112.112.112 #quad9 non-encrypted secondary

#include: /var/unbound/ad-blacklist.conf 

forward-zone:
 name: "."
    forward-addr: 9.9.9.9@853         # quad9.net primary
    forward-addr: 149.112.112.112@853 # quad9.net secondary
    forward-addr: 145.100.185.18@853 # Surfnet dnsovertls3.sinodun.com
    forward-addr: 145.100.185.17@853 # Surfnet dnsovertls2.sinodun.com

Edit: Currently trying OpenSSL flavor, will report, currently capturing tcp/853.
Edit 2: After >1hr of testing on OpenSSL and LibreSSL, this error hasn't replicated. Still capturing tcp/853 and will post if err.

[slackadelic]

Keep in mind, something is going on with Cloudflare's DNS-over-TLS implementation.. sometime last night it stopped answering queries properly... can't find any info on the cause yet...

[elektroinside]

Still going strong here... absolutely no issues...

[RickNY]

So, I initially had this working yesterday using directives that were given in a thread of the pfSense forum, using the following advanced options:

Code: [Select]

server:
ssl-upstream: yes
do-tcp: yes
forward-zone:
  name: "."
  forward-addr: 1.1.1.1@853
  forward-addr: 1.0.0.1@853
The primary difference here being that the instructions there also included the "do-tcp: yes" directive.. This mostly worked -- however my Unbound logfile was showing sporadic instances of "unbound: [50404:1] error: outgoing tcp: connect: Address already in use for 1.1.1.1"

I went back to enabling forwarding mode, and deleted the advanced options..  I have the 1.1.1.1 and 1.0.0.1 addresses in System:Settings:General, and in nonsecured mode - they work.

However -- if I now try to re-enable TLS using the configuration instructions here (or even the original one that was working yesterday), all of my DNS queries are failing.  The logfile is showing entries like:

Code: [Select]

Apr 4 10:43:35 unbound: [81143:0] debug: tcp error for address ip4 1.0.0.1 port 853 (len 16)
Apr 4 10:43:35 unbound: [81143:0] debug: outnettcp got tcp error -1
Not sure what happened between yesterday and today.. Im on OPNsense 18.1.5-amd64
FreeBSD 11.1-RELEASE-p8
OpenSSL 1.0.2n 7 Dec 2017

Thanks,
Rick

[RickNY]

Keep in mind, something is going on with Cloudflare's DNS-over-TLS implementation.. sometime last night it stopped answering queries properly... can't find any info on the cause yet...

Hmmm.. OK, this would make sense.. I just tried the configuration using just the Quad9 9.9.9.9 address, and TLS works.. Its just the Cloudflare addresses that are failing.

[mjh]

Two OPNsense devices running currently, one on LibreSSL and one on OpenSSL, both with DNS-over-TLS (see config above) and aside from the first glitch mentioned earlier all since then as been running smoothly.

Edit: Same error as in reply #16 to 9.9.9.9 on the LibreSSL system. No errors on OpenSSL yet.
Edit 2: Here's the last entries before Unbound (LibreSSL) crashed:
Edit 3: Removed output.

Unbound Debug

8<---->8

Packet Capture (short)

8<--->8

[elektroinside]

Much more to gain using Quad9 than Cloudflare anyway. Those 10-15 ms faster performance with Cloudflare is nothing compared to the security offered by Quad9. Regarding privacy.. that's something for you to decide.

[nle]

Do I have to open any ports in NAT/Firewall to make this work? Just setting up as per OP did not work.

[elektroinside]

Do I have to open any ports in NAT/Firewall to make this work?

No.

[opnfwb]

The primary difference here being that the instructions there also included the "do-tcp: yes" directive..

This is a good observation, I noticed this as well when I was setting this up yesterday and reading the Calomel Unbound tutorial. However, I checked the default unbound.conf file and it already included "do-tcp: yes" in the config on my box so I assumed it was baked in to OPNsense already. It probably doesn't hurt to list it again in the advanced options but in my case, it was not necessary because it was already included in the baseline config.

For those interested, this is my unbound.conf file, you can see the Advanced options appended to the bottom by OPNsense for the DNS/TLS servers. I'm only using Quad9 at the moment.

Also worth noting, my unbound.conf also includes additional tweaks that were configured via Services/Unbound/Advanced. So it may look a little different that a 100% stock file but, the do-tcp: yes value was there even before customization.

Code: [Select]

##########################
# Unbound Configuration
##########################

##
# Server configuration
##
server:
chroot: /var/unbound
username: unbound
directory: /var/unbound
pidfile: /var/run/unbound.pid
use-syslog: yes
port: 53
verbosity: 1
hide-identity: yes
hide-version: yes
harden-referral-path: no
do-ip4: yes
do-ip6: yes
do-udp: yes
do-tcp: yes
do-daemonize: yes
module-config: "validator iterator"
cache-max-ttl: 86400
cache-min-ttl: 0
harden-dnssec-stripped: yes
serve-expired: yes
outgoing-num-tcp: 20
incoming-num-tcp: 20
num-queries-per-thread: 1024
outgoing-range: 2048
infra-host-ttl: 900
infra-cache-numhosts: 10000
unwanted-reply-threshold: 0
jostle-timeout: 500
msg-cache-size: 100m
rrset-cache-size: 200m
num-threads: 4
msg-cache-slabs: 4
rrset-cache-slabs: 4
infra-cache-slabs: 4
key-cache-slabs: 4

auto-trust-anchor-file: /var/unbound/root.key
prefetch: yes
prefetch-key: yes
# Statistics
# Unbound Statistics
statistics-interval: 0
extended-statistics: yes
statistics-cumulative: yes

# Interface IP(s) to bind to
interface: 0.0.0.0
interface: ::0
interface-automatic: yes



# DNS Rebinding
# For DNS Rebinding prevention
#
# All these addresses are either private or should not be routable in the global IPv4 or IPv6 internet.
#
# IPv4 Addresses
#
private-address: 0.0.0.0/8       # Broadcast address
private-address: 10.0.0.0/8
private-address: 100.64.0.0/10
private-address: 127.0.0.0/8     # Loopback Localhost
private-address: 169.254.0.0/16
private-address: 172.16.0.0/12
private-address: 192.0.0.0/24    # IANA IPv4 special purpose net
private-address: 192.0.2.0/24    # Documentation network TEST-NET
private-address: 192.168.0.0/16
private-address: 198.18.0.0/15   # Used for testing inter-network communications
private-address: 198.51.100.0/24 # Documentation network TEST-NET-2
private-address: 203.0.113.0/24  # Documentation network TEST-NET-3
private-address: 233.252.0.0/24  # Documentation network MCAST-TEST-NET
#
# IPv6 Addresses
#
private-address: ::1/128         # Loopback Localhost
private-address: 2001:db8::/32   # Documentation network IPv6
private-address: fc00::/8        # Unique local address (ULA) part of "fc00::/7", not defined yet
private-address: fd00::/8        # Unique local address (ULA) part of "fc00::/7", "/48" prefix group
private-address: fe80::/10       # Link-local address (LLA)


# Access lists
include: /var/unbound/access_lists.conf

# Static host entries
include: /var/unbound/host_entries.conf

# DHCP leases (if configured)
include: /var/unbound/dhcpleases.conf

# Domain overrides
include: /var/unbound/domainoverrides.conf

# Unbound custom options
ssl-upstream: yes
forward-zone:
name: "."
forward-addr: 9.9.9.9@853
forward-addr: 149.112.112.112@853
forward-addr: 2620:fe::fe@853




###
# Remote Control Config
###
include: /var/unbound/remotecontrol.conf


[csmall]

Trying to use cloudflare doesn't work for me. I tried six ways from Sunday and no luck.

I switched to 9.9.9.9 (Quad9) and tls works fine.

I also switched to OpenSSL from LibreSSL prior to switching to Quad9.

I verified with a WAN packet capture in Wireshark that it is TLS/encrypted.

I'm not sure why Cloudflare doesn't work.

[elektroinside]

Cloudflare is some sorts of proxy for anything behind them, they intercept all queries before reaching the destination. Including encrypted communication. I didn't try, but without dnssec this may work with Cloudflare as well (although dnssec is not encrypted DNS communication, it is a signed response, with which Cloudflare may interfere).

[nle]

Is there anyone who only uses the Cloudflare DNS servers successfully in this setup? If they don't work that would explain why it did not work for me.

[unsound]

Is there anyone who only uses the Cloudflare DNS servers successfully in this setup? If they don't work that would explain why it did not work for me.

I was also trying to use Cloudflare DNS exclusively and wasn't having any luck with 1.1.1.1 or 1.0.0.1 via TLS.

Quad9 is working great though using the provided examples.