Why is custom options for Unbound removed in 21.7 ?

[134]

I'm currently using this feature in Opnsense (and previously with pfSense) to achieve split-horizon DNS with Unbound. It just came to my attention that 21.7 onward will no longer support it. May I ask why? I'm afraid this would be the deal breaker for me and some other users.

[Patrick M. Hausen]

You can still use the config directory mentioned in the release notes. It's just removed from the UI. If you absolutely must use this feature, you probably know how to use ssh?

[134]

I can. But this would defeat the goal of using single configuration file to restore everything. In my opinion it should be moved to Advanced section rather than removing entirely.

[mimugmail]

Open a feature request in GitHub about the config lines, I can add the to misc submenu

[allebone]

Hi I am also wondering about this. Im not clued up to what has happened so am hoping someone can explain to me. Sorry I just dont know much about why it was removed and how to make the changes going forward without it that it all. Im happy to change if there is an easy method to follow that performs the same thing.

"Does anyone know why this has been removed. Im sad it has been removed because I used that page when setting up unbound. Will it still be easy to set the options that were on that page for me? I ask out of ignorance as I dont know why it was removed or how to configure the same options going forward. Im hoping it will be easy."

P

[franco]

1. Ever since the OpenVPN custom options privilege escalation debacle in 2019 that affected *sense and prior widespread use of "just let us have custom configuration fields for all services" we decided to remove these ticking time bombs proactively and block their inclusion... slowly but steadily.

https://github.com/opnsense/changelog/blob/17ab9aee2c11fcaf811245b0b9a5e23a7c48a34f/community/19.1/19.1.8#L36

2. From a product perspective advanced users will add their custom glue and deprive meaningful use cases from the not so advanced users. It's better to work together and find GUI-driven solutions to problems everybody has.

3. For anyone saying "The GUI can't do this but when I edit the config file it gets overwritten" we usually advise to avoid using the GUI (core or plugin) and just use the service like anyone would on FreeBSD. Most decline, hence (2) is better in the long run anyway.

4. For everyone else and our plugin infrastructure pluggable services are the greatest thing we could possibly offer. Custom blob in the config.xml has no meaning, no structure and nobody wants to write a plugin for that, but imagine you can build (your private if you insist) plugin to do your current tasks more effectively and transparently. Hell, you can even write a plugin that retains custom options if that's the sort of thing that solves all issues.. at least locally.

5. For Unbound in particular only a handful of people improved the code we have over the last 6 years. It sort of goes back to problem (2) and how improvements have been lacking for having custom options that squelch most valuable feedback.

6. Most of what I just said has been said countless times before over the years and most recently we removed Dnsmasq custom options and also put Unbound custom options on the roadmap and the 21.1 release notes. If you don't take this seriously I'm not sure how we can be of help anymore than we've done.

https://github.com/opnsense/changelog/blob/17ab9aee2c11fcaf811245b0b9a5e23a7c48a34f/community/21.1/21.1#L24-L26


Cheers,
Franco

[opnfwb]

Quick question on this. If I restore a full backup config from OPNsense 21.1 and I'm using Unbound Advanced options for DoT, will these import to a 21.7 install?

[chemlud]

We had a threat on the DoH option in the first half of Q1 2021. It never happend, unfortunately. I will not update to 21.7 if it breaks DoH in unbound....

[franco]

Ticket for missing options is still open and will be done come 21.7. Nobody is being forced to upgrade and I think that's common knowledge...

https://github.com/opnsense/core/issues/4327

If there is something missing let me know.

Obviously custom options can't be migrated to any meaningful model which is part of the original problem.


Cheers,
Franco

[mimugmail]

4. For everyone else and our plugin infrastructure pluggable services are the greatest thing we could possibly offer. Custom blob in the config.xml has no meaning, no structure and nobody wants to write a plugin for that, but imagine you can build (your private if you insist) plugin to do your current tasks more effectively and transparently. Hell, you can even write a plugin that retains custom options if that's the sort of thing that solves all issues.. at least locally.

If someone is interested, I can add such a plugin to my community repo, this is really easy

[mimugmail]

We had a threat on the DoH option in the first half of Q1 2021. It never happend, unfortunately. I will not update to 21.7 if it breaks DoH in unbound....

You mean DoT? Yes, there is no grid for adding cert checks, to me, personally, I'd just allow a hash sign in the validation and dont migrate to grid. Franco, what do you think?

[opnfwb]

Yes, there is no grid for adding cert checks, to me, personally, I'd just allow a hash sign in the validation and dont migrate to grid. Franco, what do you think?

This sounds like a great solution and would cover my use case (and I suspect most others). It's the only reason I can't use the built-in DoT function in OPNsense. We aren't getting the full benefit by just forwarding queries to ipaddress@853. Using the additional validation that the answers coming back are actually coming from our chosen DoT provider is worth it, IMHO. That's why I continue to use a custom config just for basic DoT forwarding. FWIW, pfSense implemented something similar in the General Settings if you specify the IP and domain name of the DNS entries, and check the box to "enable forwarding mode" and "encrypt DNS outbound over TLS" it automatically converts those entries to IP@PORT#provider.name in the running unbound config.

Also a side note, I installed 21.7RC2 on a VM and it's quite easy to just create a new .conf with my same custom options from the Unbound config in 21.1. So if this isn't added to the GUI it isn't the end of the world, I'll even post a quick guide in the HOWTO section here. Just trying to get an idea if the .conf is the only available option or if there's a plan to add IP@PORT#provider.name in the DOT syntax.

It's good to know about a migration plan. For people like me that rely on DoT, I'll need to do some additional work to replicate the existing functionality from 21.1 that I am relying on.

[CloudHoppingFlowerChild]

This is worth nothing so don't waste time reading it.

I'm not interesting in protecting the incompetent from themselves by concealing functionality. I have screwed my self in the router department many times over the years; it's how I learned. What I am interested in is a highly configurable router, which is why my WRT54G was my last off-the-shelf router.

1) It's not as if someone can type gibberish into the custom options field and hit save and their internet stops working. They're going to have to research actual options and enter something correctly formatted to have any effect. That's a far greater barrier to damage than having check boxes and drop down menus that can be scrambled like a Rubik's Cube.
2) They had to elect to use Unbound in the first place. If they break it, they can fix it or switch back on the default resolver.
3) There's already an marvelous undo button in the form of the System > Configuration > History page which lets one very effectively roll back the clock.

A GUI enhances the accessibility and usability of the underlying services; where does the mandate to curtail and conceal functionality come from?

Given these points, I find the given reason of trying to protect the incapable from themselves to be irrational.

edit: I forgot about reading this shit two weeks ago and hit the upgrade button. 2 seconds later I remembered. Now I wish I could set something on fire. You just culled functionality in the name of user friendliness. FUCK. THAT. fuckyoufuckyourmom.

[allebone]

Hi again all,

@franco thank you for your detailed explanation for why this feature is being removed. Makes sense to me :) I am happy to roll with the change however I also think I misunderstood what was changing exactly.

From the changelog you posted on the announcement post you wrote:
"Unbound advanced configuration has been removed.  Local override directory /usr/local/etc/unbound.opnsense.d exists."

From this I took this to mean the page in "Services - Unbound DNS - Advanced" is being removed. However in reading the above comments I believe that the only thing being removed is actually "Services - Unbound DNS - general - Custom options" which is quite different.

In fact this section does not even affect me which if is the case is quite a different section being removed than what I took the announcement to mean. If thats the case, sorry for the misunderstanding :)

P

[chemlud]

We had a threat on the DoH option in the first half of Q1 2021. It never happend, unfortunately. I will not update to 21.7 if it breaks DoH in unbound....

You mean DoT? Yes, there is no grid for adding cert checks, to me, personally, I'd just allow a hash sign in the validation and dont migrate to grid. Franco, what do you think?

Yes, I mean DNS-over-TLS. DNS-over-HTTPS is malicous. :-)

https://forum.opnsense.org/index.php?topic=21153.msg98892#msg98892

We started early (January!), could we please have the necessary check boxes in the GUI, please... ;-)