Unbound, stubby and pi-hole

[cookiemonster]

Hello.
I have just built my first proper router and firewall with opnsense, replacing a router running DD-WRT.
This is an introduction and request for friendly assistance.
I am familiar with linux and freebsd but not with the subsystems OPNSense uses. I am on a learning curve.

My goal is to have functionally the setup I had before, sending LAN clients for their DNS lookups to a pi-hole, which in turn sends the allowed queries onward to the router, which in turn uses getdns stubby to go to public TLS enabled resolvers for DNSoverTLS.

I am introducing the parts one by one and testing instead of all at once. Currently I want to get stubby and unbound to work, leaving pi-hole out. The pi-hole ip is 192.168.5.154

My problem appears as soon as I change unbound for the setup.

I have read and followed the various posts on this subject on this forum but either I am missing a step or OPSense has changed enough since they were written that I need to start from scratch.

I am using OPNSense version OPNsense 21.1.5-amd64. The host is a PC Engines APU4D2.
The steps so far taken:
1. Install getdns stubby by pkg install on OPNSense. The version is getdns-1.5.2_4. The router ip is static 192.168.5.160
It is configured to run on 127.0.0.1@8053 with only two upstream resolvers until the setup is working. The config is this:

Code: [Select]

root@OPNsense:/usr/local/etc/stubby # cat /usr/local/etc/stubby/stubby.yml

resolution_type: GETDNS_RESOLUTION_STUB

dns_transport_list:
  - GETDNS_TRANSPORT_TLS

tls_authentication: GETDNS_AUTHENTICATION_REQUIRED

tls_query_padding_blocksize: 128

edns_client_subnet_private : 1

round_robin_upstreams: 0

idle_timeout: 10000

tls_ca_path: "/etc/ssl/"

tls_cipher_list: "EECDH+AESGCM:EECDH+CHACHA20"

tls_ciphersuites: "TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256"

tls_min_version: GETDNS_TLS1_2

tls_max_version: GETDNS_TLS1_3

listen_addresses:
  - 127.0.0.1@8053
#  - 0::1@8053

dnssec: GETDNS_EXTENSION_TRUE

appdata_dir: "/var/cache/stubby"

tls_ca_file: "/usr/local/share/certs/ca-root-nss.crt"

upstream_recursive_servers:

  - address_data: 9.9.9.9
    tls_auth_name: "dns.quad9.net"
    tls_port: 853

  - address_data: 1.1.1.1
    tls_auth_name: "cloudflare-dns.com"
    tls_port: 853


2. Pi-hole is not running a DHCP server. It is set to use a single upstream DNS server with the router ip of 192.168.5.160 . It is listening only on its eth0 interface. There are no other interfaces configured. It is a amd64 virtual machine on an ESXi host. It's IP address is 192.168.5.154

3, On OPNSense:
3..1 On " System: Settings: General" > Networking > DNS servers: I have set 127.0.0.1. "DNS server options" I have none selected, none ticked.
3.2 On " Services: Unbound DNS: General" :
- Enable Unbound is ticked.
- Network interfaces I have both LAN and WAN selected. I don't have an option of ALL.
- DHCP registration is ticked
- DHCP static mappings is ticked

In this setup my LAN clients are getting their DNS server allocation from DHCP pointing them to 192.168.5.160. The OPNSense router is resolving DNS for them as a "normal" setup.

As soon as I enable these additional options in Unbound, my LAN clients fail to get their DNS queries resolved:
server:

Code: [Select]

do-not-query-localhost: no
forward-zone:
name: "."
forward-addr: 192.168.5.160@8053I have tried setting these too but they seemed to cause Unbound to fail to start.
#forward-tls-upstream: yes
#logservfail: yes

Unfortunately with no logging available on stubby I'm really stumped on where/how to see where the failure occurs.

Can I ask for some pointers to start my investigation please.

[cookiemonster]

I'm going to ask a what I need to know.

What do I need to do on OPNSense to:
a) tell the lan clients requesting an ip address to use 192.168.5.154 as their DNS resolver
b) when 192.168.5.154 sends those queries to OPNSense on port 53, to have OPNSense pushing them over to it's own loopback on the stubby port 127.0.0.1@8053
c) are there any firewall rules that need to allow this? I have not created any.
d) additionally, for any lan clients trying to bypass the setup and send their own DNS queries directly (port 53 of course), what do I need to do to capture them and drop+notify.

Thank you.

[gpb]

I have a similar setup but instead of stubby, I use dnscrypt-proxy and cloudflared (both for encryption using DoH) on my rpi's.  This for me at least simplifies, OPNsense tells hosts on my network to resolve at RPi addresses.  The Rpi's fully handle the connection to resolve the DNS requests.  If a port 53 request comes from a non Rpi device, then it is NAT'd to one of the RPi (not dropped or blocked) for processing.

a: DHCP under Services in OPNsense...for each network segment it applies to.
b: ?
c: Not that I'm aware.
d: as mentioned above, use a NAT rule to reroute any dest port 53 that are coming from NOT your pihole host.

Hope that helps.

[cookiemonster]

It does help. I will be staying with stubby or another solution for DNSoverTLS. I really prefer DoT over DoH.
Admittedly I am pulling my hairs out and my family are really cross that I keep breaking the internet for everyone. I might have to go to back to DD-WRT with stubby but I really want to keep OPNSense. I'll persevere.

For a):
Is that in: Services > DHCPv4 > [LAN] > DNS servers  field?
The help for that field reads "Leave blank to use the system default DNS servers: This interface IP address if a DNS service is enabled or the configured global DNS servers." I've re-read a few times and I struggle to understand from "This interface.." in the contextual help
If that is the place, great. If you could confirm please.

thanks for the answer for d)

[gpb]

DNSCrypt-Proxy supports both, as well as their own DNSCrypt...just to clarify.

a: correct, if you leave it blank it will use what you have in your general settings dns field...so...it's flexible.  Interface refers to "LAN1", "LAN2", "VLAN1"....etc.  So they can be the physical jacks on your router or virtual in the case of VLANs.  Yeah that wording a bit confusing...but it's the parameter(s) the DHCP server sends to hosts requesting an IP address...so it can be specified for each interface.

Good luck! 

[cookiemonster]

thank you. One piece of the puzzle.

[cookiemonster]

I'm happy to report success. My first challenge was to get stubby working, testing it within the router wasn't easy. I didn't want to install tools that pulled dependencies that could break the OS.
The working config is this in case someone needs it:

Code: [Select]

resolution_type: GETDNS_RESOLUTION_STUB
dns_transport_list:
  - GETDNS_TRANSPORT_TLS
tls_authentication: GETDNS_AUTHENTICATION_REQUIRED
dnssec_return_status: GETDNS_EXTENSION_TRUE
tls_query_padding_blocksize: 128
edns_client_subnet_private : 1
round_robin_upstreams: 1
idle_timeout: 10000
tls_ca_path: "/etc/ssl/"
tls_cipher_list: "EECDH+AESGCM:EECDH+CHACHA20"
tls_ciphersuites: "TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256"
tls_min_version: GETDNS_TLS1_2
tls_max_version: GETDNS_TLS1_3

listen_addresses:
  - 127.0.0.1@8053
#  - 0::1@8053

#dnssec: GETDNS_EXTENSION_TRUE
appdata_dir: "/var/cache/stubby"

#dnssec_trust_anchors: "/usr/local/sbin/unbound-anchor"
dnssec_trust_anchors:
  - "/usr/local/etc/unbound/root.key"

tls_ca_file: "/usr/local/share/certs/ca-root-nss.crt"

upstream_recursive_servers:

## Quad 9 'secure' service - Filters, does DNSSEC, doesn't send ECS
  - address_data: 9.9.9.9
    tls_auth_name: "dns.quad9.net"
    tls_port: 853

## Cloudflare 1.1.1.1 and 1.0.0.1
  - address_data: 1.1.1.1
    tls_auth_name: "cloudflare-dns.com"
    tls_port: 853
I've left a couple of commented out lines so we can see the format that is needed in.

Then on Unbound custom options:

Code: [Select]

server:
do-not-query-localhost: no
forward-zone:
name: "."
forward-addr: 127.0.0.1@8053
If I entered my OPNSense LAN address it wouldn't work. All tests from OPNSense would give servfail and unfortunately there are no logs to tell why. It was trial and error. Enable DNSSEC support in Unbound is not selected.

In System, General, Settings the DNS has 127.0.0.1. Again the LAN ip wouldn't work. There are no other settings selected on this page.
Edit 28/06/2021: I am unsure what changed but after some time and normal reboots I noticed I had no name resolution on the firewall itself i.e. for checking packages, system or plugins updates. I had to change in System, General, Settings and select (tick) "Do not use the local DNS service as a nameserver for this system".

And from gpb advice I've set in Services, DHCPv4, DNS Servers for my single pool the LAN ip of the pi-hole. This ip is a reserved one, also on this page.

Finally, in pi-hole, in Settings, DNS I've set a "Custom Upstream DNS Servers" with the (again static) LAN ip address of OPNSense.

Now the basic setup is done I can move on to analysing any rogues dns queries with firewall rules. Also I look forward to utilising the rest of OPNSense capabilities.

[opnfwb]

I'm not using Stubby with OPNsense (just Unbound for DoT) however, it's great to see someone posting the available solution. Stubby works great, I've used it when I had to use OpenWRT. Very handy to see how to integrate it on OPNsense if needed.

[cookiemonster]

Hi opnfwb. I used Stubby on DD-WRT and a pi-hole and they worked great for me.

There are posts here and other places to get stubby on OPNSense but I couldn't get their advised config to work, but they start from a local build off a git clone, not a binary pkg install. That and being 3 yrs old posts might be the reason. I couldn't find any other integrating pi-hole into the setup.
Thank you for the positive note.