DNS-over-TLS - are ISPs interfering?

Started by chemlud, September 30, 2021, 06:16:15 PM

Previous topic - Next topic
Print
Go Down Pages1
User actions
September 30, 2021, 06:16:15 PM Last Edit: September 30, 2021, 06:18:22 PM by chemlud
Hi!

Have here two OPNsense latest (LibreSSL), both with DoT configured for longer time.

One simply fails to resolve from the beginning:

Code Select
2021-09-30T17:59:02 unbound[30141] [30141:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T17:59:02 unbound[30141] [30141:1] notice: ssl handshake failed 185.95.218.42 port 853
2021-09-30T17:59:02 unbound[30141] [30141:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T17:59:01 unbound[30141] [30141:1] notice: ssl handshake failed 185.95.218.42 port 853
2021-09-30T17:59:01 unbound[30141] [30141:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T17:59:01 unbound[30141] [30141:1] notice: ssl handshake failed 185.95.218.42 port 853
2021-09-30T17:59:01 unbound[30141] [30141:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T17:59:01 unbound[30141] [30141:1] notice: ssl handshake failed 185.95.218.42 port 853
2021-09-30T17:59:01 unbound[30141] [30141:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T17:59:01 unbound[30141] [30141:1] notice: ssl handshake failed 185.95.218.42 port 853
2021-09-30T17:59:01 unbound[30141] [30141:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T17:59:01 unbound[30141] [30141:1] info: generate keytag query _ta-4f66. NULL IN
2021-09-30T17:59:01 unbound[30141] [30141:1] notice: ssl handshake failed 185.150.99.255 port 853
2021-09-30T17:59:01 unbound[30141] [30141:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T17:59:01 unbound[30141] [30141:1] notice: ssl handshake failed 185.150.99.255 port 853
2021-09-30T17:59:01 unbound[30141] [30141:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T17:59:01 unbound[30141] [30141:1] notice: ssl handshake failed 5.9.164.112 port 853
2021-09-30T17:59:01 unbound[30141] [30141:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T17:59:01 unbound[30141] [30141:1] notice: ssl handshake failed 5.9.164.112 port 853
2021-09-30T17:59:01 unbound[30141] [30141:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T17:59:00 unbound[30141] [30141:1] notice: ssl handshake failed 5.9.164.112 port 853
2021-09-30T17:59:00 unbound[30141] [30141:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T17:59:00 unbound[30141] [30141:1] notice: ssl handshake failed 5.9.164.112 port 853
2021-09-30T17:59:00 unbound[30141] [30141:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T17:59:00 unbound[30141] [30141:1] notice: ssl handshake failed 5.1.66.255 port 853
2021-09-30T17:59:00 unbound[30141] [30141:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T17:59:00 unbound[30141] [30141:1] notice: ssl handshake failed 5.1.66.255 port 853
2021-09-30T17:59:00 unbound[30141] [30141:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T17:59:00 unbound[30141] [30141:1] notice: ssl handshake failed 185.150.99.255 port 853
2021-09-30T17:59:00 unbound[30141] [30141:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T17:59:00 unbound[30141] [30141:1] notice: ssl handshake failed 185.150.99.255 port 853
2021-09-30T17:59:00 unbound[30141] [30141:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T17:59:00 unbound[30141] [30141:1] notice: ssl handshake failed 5.1.66.255 port 853
2021-09-30T17:59:00 unbound[30141] [30141:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T17:59:00 unbound[30141] [30141:1] notice: ssl handshake failed 5.1.66.255 port 853
2021-09-30T17:59:00 unbound[30141] [30141:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T17:58:58 unbound[30141] [30141:0] info: start of service (unbound 1.13.2).

Therefore the other OPNsense is configured as DNS via a tunnel. This worked until today. Then this afternoon all of a sudden DNS failed on the other OPNsense completely, although 5 DoT servers are configured.

When I add 9.9.9.9 or mulvad DNS it starts to work again an the remote DNS (but not on the OPNsense initially failing during TLS handshake).

When I do a package capture on WAN (port 853) it looks like normal TLS 1.2 and 1.3 traffic with at least one remote IP (95.216.212.177), but DNSleaktest.com reports no functional DNS servers and in browser nothing loads (again: my usually working 5 DoT servers still configured).

The OPNsense is located in Northern Europe, most DNS servers I configured are located in Central Europe, but is there some kind of iron curtain for DoT troughout Europe?

WHY? Is the ISP interfering with DoT?
kind regards
chemlud
____
"The price of reliability is the pursuit of the utmost simplicity."
C.A.R. Hoare

felix eichhorns premium katzenfutter mit der extraportion energie

A router is not a switch - A router is not a switch - A router is not a switch - A rou....
September 30, 2021, 06:17:35 PM #1 Last Edit: September 30, 2021, 06:29:44 PM by chemlud
It's getting even worse. On the initially failing OPNsense with 5 DoT servers I added 9.9.9.9 #853. Now I get DNS resolved in dnsleaktest.com, but on an obscure IP in the UK (see pic attached), but there is nothing in the unbound log:

Code Select
2021-09-30T18:20:02 unbound[4249] [4249:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T18:20:02 unbound[4249] [4249:1] notice: ssl handshake failed 185.150.99.255 port 853
2021-09-30T18:20:02 unbound[4249] [4249:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T18:20:02 unbound[4249] [4249:1] notice: ssl handshake failed 185.95.218.42 port 853
2021-09-30T18:20:02 unbound[4249] [4249:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T18:20:02 unbound[4249] [4249:1] notice: ssl handshake failed 185.95.218.42 port 853
2021-09-30T18:20:02 unbound[4249] [4249:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T18:20:01 unbound[4249] [4249:1] info: generate keytag query _ta-4f66. NULL IN
2021-09-30T18:20:01 unbound[4249] [4249:1] notice: ssl handshake failed 185.150.99.255 port 853
2021-09-30T18:20:01 unbound[4249] [4249:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T18:20:01 unbound[4249] [4249:1] notice: ssl handshake failed 185.150.99.255 port 853
2021-09-30T18:20:01 unbound[4249] [4249:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T18:20:01 unbound[4249] [4249:1] notice: ssl handshake failed 5.9.164.112 port 853
2021-09-30T18:20:01 unbound[4249] [4249:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T18:20:01 unbound[4249] [4249:1] notice: ssl handshake failed 5.9.164.112 port 853
2021-09-30T18:20:01 unbound[4249] [4249:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T18:20:01 unbound[4249] [4249:1] notice: ssl handshake failed 5.9.164.112 port 853
2021-09-30T18:20:01 unbound[4249] [4249:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T18:20:01 unbound[4249] [4249:1] notice: ssl handshake failed 5.9.164.112 port 853
2021-09-30T18:20:01 unbound[4249] [4249:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T18:20:01 unbound[4249] [4249:1] notice: ssl handshake failed 185.95.218.42 port 853
2021-09-30T18:20:01 unbound[4249] [4249:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T18:20:00 unbound[4249] [4249:1] notice: ssl handshake failed 185.95.218.42 port 853
2021-09-30T18:20:00 unbound[4249] [4249:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T18:20:00 unbound[4249] [4249:1] notice: ssl handshake failed 5.1.66.255 port 853
2021-09-30T18:20:00 unbound[4249] [4249:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T18:20:00 unbound[4249] [4249:1] notice: ssl handshake failed 5.1.66.255 port 853
2021-09-30T18:20:00 unbound[4249] [4249:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T18:20:00 unbound[4249] [4249:1] notice: ssl handshake failed 5.1.66.255 port 853
2021-09-30T18:20:00 unbound[4249] [4249:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T18:20:00 unbound[4249] [4249:1] notice: ssl handshake failed 5.1.66.255 port 853
2021-09-30T18:20:00 unbound[4249] [4249:1] error: ssl handshake failed crypto error:14FFF086:SSL routines:(UNKNOWN)SSL_internal:certificate verify failed
2021-09-30T18:19:55 unbound[4249] [4249:0] info: start of service (unbound 1.13.2).
kind regards
chemlud
____
"The price of reliability is the pursuit of the utmost simplicity."
C.A.R. Hoare

felix eichhorns premium katzenfutter mit der extraportion energie

A router is not a switch - A router is not a switch - A router is not a switch - A rou....
September 30, 2021, 06:38:40 PM #2
They are using probably LE certs. OPNsense seems to have an issue with them since the DST Root CA X3
expired today. Updating OPNsense also fails, if the mirror is using a LE certs. https://forum.opnsense.org/index.php?topic=24968.msg119835#msg119835
I checked unicast.censurfridns.dk and anycast.censurfridns.dk and they are failing for me and using LE certs. Seems like a pattern to me.

Just tested:
Code Select
[admin@OPNsense ~]$ fetch -o mimugmail.conf https://www.routerperformance.net/mimugmail.conf
Certificate verification failed for /O=Digital Signature Trust Co./CN=DST Root CA X3
5843273977856:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/usr/src/crypto/openssl/ssl/statem/statem_clnt.c:1915:
fetch: https://www.routerperformance.net/mimugmail.conf: Authentication error

A simple fetch also fails.
September 30, 2021, 06:59:08 PM #3
hmmm, many thanks for reply! Then it would make sense... ;-)

How to end this catch 22 if no update (of certs) possible due to failing handshake? Any mirrors not using LE certs?
kind regards
chemlud
____
"The price of reliability is the pursuit of the utmost simplicity."
C.A.R. Hoare

felix eichhorns premium katzenfutter mit der extraportion energie

A router is not a switch - A router is not a switch - A router is not a switch - A rou....
September 30, 2021, 08:18:04 PM #4
Use http (not https) mirrors.
In theory there is no difference between theory and practice. In practice there is.
September 30, 2021, 10:02:33 PM #5
Hi,

let me guess, you updated your LE certs yesterday, before the ACME Client update was applied?
I had to delete all LE CA certs unter System: Trust: Authorities, even the new ones from yesterday.
I then reissued all LE certs on my OPNsense and the R3 (ACME Client) CA was added again and now my system works again as expected. Maybe a reboot also will work. My ensurfridns.dk are working again now.

KH
September 30, 2021, 10:04:22 PM #6
Nope, no LE certs or authorities present on my boxes....
kind regards
chemlud
____
"The price of reliability is the pursuit of the utmost simplicity."
C.A.R. Hoare

felix eichhorns premium katzenfutter mit der extraportion energie

A router is not a switch - A router is not a switch - A router is not a switch - A rou....
September 30, 2021, 10:06:51 PM #7
Ok, then you can try the solution from Felix: https://forum.opnsense.org/index.php?topic=24950.msg119873#msg119873

KH
October 01, 2021, 09:11:10 AM #8
Hmm... I could update to 21.7.3_3 without any intervention from https update server...

Let's see what the DNS does over the day.
kind regards
chemlud
____
"The price of reliability is the pursuit of the utmost simplicity."
C.A.R. Hoare

felix eichhorns premium katzenfutter mit der extraportion energie

A router is not a switch - A router is not a switch - A router is not a switch - A rou....
Print
Go Up Pages1
User actions