Messages

The other option to register reservations ("Register DHCP Static Mappings") is selected, as I need it to declare all the needed hosts.  Not sure if it interferes with query forwardings, as long as I'm not using ISC and all of them migrated to Kea and uninstalled the plugin.

Disable this too, and restart unbount. This option won't work with Kea anyway, it's ISC specific and even if ISC is disabled it will interfere with Unbound based on disabled ISC configuration.

Disable `Register ISC DHCP4 Leases` option in Unbound. Even if ISC is disabled, this option will still affect Unbound, and register dummy tables, which in turn breaks query forwarding.

True, but ISC plugin also isn't removed unless I'm mistaken here as well.

Incidentally I proposed the following change so that the ISC page is accessible again as long as the plugin is installed:

https://github.com/opnsense/core/pull/9755

It's not removed, but once you disable it or enable dnsmasq, you may be searching all day where those static entries are comming form, and they would be nowhere to be found.

pull/9755 works to show it. Not sure how hard would it be to just check the state of ISC and not actually process those entries in it's config, if disabled. Either way, it's not critical issue, more of a annoyance for people who are migrating, because not only you need to set everything in new place, but also manually clean the old place. Which may be problematic if you need to temporary switch back to ISC or something. Now that I know what those things are happening, it's easy to clean those entires, but I though to bring this up to discussion.

To some degree I understand the problem at hand

I think you are missing the point here.

but I'm wondering why you find your manually configured static ARP entries suboptimal?

Because, I don't want them "anymore". The configuration page for them is NOT accessible when ISC is disabled. I need to disable Dnsmasq for the given interface, to access this configuration again.

All I need is static DHCP assignments, with ISC static arp was bonus on top.

Are you still using ISC?

No. That is the point.

Do you need these static entries?

No. But after disabling ISC, it is not clear where they are coming from and how to remove those entries. UI elements are completely not available, even more so if you remove the ISC DHCP plugin completely.

You could also remove the MAC address which would make them not feed into the Neighbor table.

I know how to workaround the issue, but once ISC is disabled it is no longer possible, without reenabling it (at least temporary). It makes stale configuration, no longer accessible from user interface to affect the arp table.

EDIT: and ofc, on top of that there is the fact that static arp is not possible with ISC alternatives, currently.

Trimmed to single entry. Note that the "static arp" on the interface was not enabled/checked. The static assignments are the issue in my case.

Code Select Expand

root@OPNsense:~ # ifconfig vmx0_vlan10
vmx0_vlan10: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: IoT (opt2)
        options=4000000<MEXTPG>
        ether <mac>
        inet 192.168.10.1 netmask 0xffffff00 broadcast 192.168.10.255
        groups: vlan
        vlan: 10 vlanproto: 802.1q vlanpcp: 0 parent interface: vmx0
        media: Ethernet autoselect
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>


Code Select Expand

root@OPNsense:~ # arp -a
? (192.168.10.100) at <mac> on vmx0_vlan10 permanent [vlan]

This is caused by this entry, in ISC DHCP interface options.

And resulting mapping here, from permanent arp entry.

The first issue just needs a reboot, then the static arp flag will be removed from interfaces it has been set on.

https://github.com/opnsense/core/issues/9838

Static assignments remain persistent, even after reboot.

EDIT: Even removing the ISC plugin and rebooting, doesn't help.

Hi,

I migrated my configuration to Dnsmasq, but I am experiencing some annoyances. There are implicit side-effects of, now legacy, ISC dhcp server configuration, which are hard to track. Even when it is disabled.

1. Neighbors: Static assignments are still being applied from the ISC configuration, even though ISC is disabled and the interfaces are now managed by Dnsmasq. However, Dnsmasq does not apply its own static assignments. ISC has a "static ARP" option.

2. When "Register ISC DHCP4 Leases" is enabled in Unbound, it still overwrites the local domain settings even though ISC itself is disabled. After migrating to Dnsmasq, local hostnames start returning NXDOMAIN. This happens even when "Query Forwarding" for local domains is configured correctly. It works for a few seconds after restarting Unbound, but once a "sync" from the (disabled) ISC service occurs, it begins returning NXDOMAIN again.

First issue can partially be workaround by manually clearing ISC configuration, but this shouldn't be necessary if it is disabled. Partially, because it wouldn't apply static assignments from Dnsmasq.

Likewise, second issue can be workaround by not setting "Register ISC DHCP4 Leases", which is fair, but there should be checks if source of this information (ISC) is actually used/valid. It took me quite some time to find that this option was the culprit.

- Kacper

By doing it like so:

1. Unbound is your main DNS resolver. It either resolves internet DNS by itself, working as a resolving DNS or you configure it to use an upstream server, like 8.8.8.8 via normal DNS or DNS-over-TLS. You also tell it to "Do not forward private reverse lookups". The import part is that you instruct it to forward specific domains, namely, you private domains, to 127.0.0.1:53053. This includes the reverse domains, say "168.192.in-addr.arpa".

2. You configure DNSmasq to run on port 53053 and set it up to resolve your internal domains, it will use the system name servers as upstream servers. These do not even have to use 127.0.0.1 (Unbound).

Thus, regular queries go to Unbound first and are either forwarded to DNSmasq (if they match fordwarded domains) or resolved by Unbound.
Because the forwarded, local domains can be resolved by DNSmasq, you will either get an IP or an NXDOMAIN. And since DNSmasq is only ever asked for internal domains (by Unbound only), its upstream server will never get used, even if it is Unbound by accident.

I don't think this is working like that currently, as evident by multiple reports about it. I think it is already resolved as https://github.com/opnsense/core/issues/8708 and related changes. I will try on next release.

I believe it's unbound because when running a nslookup on my windows machine to a host that is in unbound's overrides, it times out twice before finally succeeding, suggesting unbound is choking on...something because it's a local lookup to unbound.

I'm seeing the same thing. I think current documentation is incomplete in a way of setting up this unbound to dnsmasq forward. And in fact I don't even see how it suppose to work in practice. Symptoms are like @Unspec described, timeouts and generally instability, it sometimes works, other times does not. It queries unbound multiple times, also with host.lan.internal.lan.internal.

If you think about it, unless I'm missing something it is lucky it even manages to return anything sometimes, breaking recursion.

If dnsmasq dns is set on alternative port, it will forward queries back to unbound which is main dns server. Unbound then sees that this query should be forwarded (back) to dnsmasq, which ask unbound again... and we get infinite recursion. I hope I missed something in configuration, because as it stand now it hardly can work like that.

I'm still confused about this one. Can someone explain how should a infinite recursion of dns resolution between unbound and dnsmasq prevented?

I'm not going to run two DNS services just to be able to resolve internal host names. This whole deprecation of ISC has been a mess. ISC+Unbound is exceedingly simple and functional, Kea and dnsmasq are both half-baked.

I felt the same until I read through the updated docs. DNSmasq is primarily being introduced for dhcpd. Using it also for local name resolution (via an unbound forwarding) means no unbound restarts on updated leases. You retain a recursive resolver and still only have two daemons running in order to provide DNS/DHCP. If it all works as described in the docs, I will be more than happy to switch since I was fond of dnsmasq from previous experience.

In theory it looks ok, but unfortunately in practice it is not stable currently. As reported in the other thread.

Wouldn't it be possible to register local leases in unbound in some other way? Running two DNS servers just for this seems like a low-effort workaround for limitation that shouldn't exist.

I believe it's unbound because when running a nslookup on my windows machine to a host that is in unbound's overrides, it times out twice before finally succeeding, suggesting unbound is choking on...something because it's a local lookup to unbound.

I'm seeing the same thing. I think current documentation is incomplete in a way of setting up this unbound to dnsmasq forward. And in fact I don't even see how it suppose to work in practice. Symptoms are like @Unspec described, timeouts and generally instability, it sometimes works, other times does not. It queries unbound multiple times, also with host.lan.internal.lan.internal.

If you think about it, unless I'm missing something it is lucky it even manages to return anything sometimes, breaking recursion.

If dnsmasq dns is set on alternative port, it will forward queries back to unbound which is main dns server. Unbound then sees that this query should be forwarded (back) to dnsmasq, which ask unbound again... and we get infinite recursion. I hope I missed something in configuration, because as it stand now it hardly can work like that.

Any news about this? Do we want to fix this before 22.7?

Oh, I just lost whole finished post when trying to send it, because it log me out. I guess we go again, probably shorter version this time and less coherent...

Indeed I can work with snapshots. I didn't know it is such painless to move to dev version. So I moved to 22.7.a_413 (7fdc163bf).

Issue 1. from OP is fixed, enabling interface seems to work as expected.

But there are still issues. Sorry for being chaotic before I wasn't sure how it should work exactly.

Disabling gateway does not adjust routes correctly. To recap I have gif0 device assigned to WANv6 interface that is used as upstream gateway for IPv6.

I will be disabling WANv6 interface which also controls associated gateway.

Code Select Expand

netstat -6rWn | grep gif0$

1. Nominal case, after reboot (WANv6 interface enabled)

Code Select Expand

Destination                       Gateway                       Flags   Nhop#    Mtu    Netif
default                           <ipv6>                        UGS        13   1480     gif0
<ipv6>                            link#10                       UH         10   1480     gif0
fe80::%gif0/64                    link#10                       U          12   1480     gif0
2a. After disabling gateway only (no change to 1)

Code Select Expand

Destination                       Gateway                       Flags   Nhop#    Mtu    Netif
default                           <ipv6>                        UGS        13   1480     gif0
<ipv6>                            link#10                       UH         10   1480     gif0
fe80::%gif0/64                    link#10                       U          12   1480     gif0
2b. After disabling WANv6 interface

Code Select Expand

Destination                       Gateway                       Flags   Nhop#    Mtu    Netif
default                           <ipv6>                        UGS        13   1480     gif0
fe80::%gif0/64                    link#10                       U          12   1480     gif0
3. After reboot (WANv6 interface disabled or only gateway disabled)

Code Select Expand

Destination                       Gateway                       Flags   Nhop#    Mtu    Netif
<ipv6>                            link#10                       UH          9   1280     gif0
fe80::%gif0/64                    link#10                       U          11   1280     gif0
4. After enabling WANv6 interface

Code Select Expand

Destination                       Gateway                       Flags   Nhop#    Mtu    Netif
default                           <ipv6>                        UGS        11   1480     gif0
<ipv6>                            link#10                       UH          8   1480     gif0
fe80::%gif0/64                    link#10                       U          10   1480     gif0

As we can see it is not consistent. 1 and 4 is expected. In 2 and 3 case I would expect there are no routes. Traffic is still routed through this gateway while I would expect there is no default route for IPv6. After disabling WANv6 interface firewall rules for this interface does not apply to traffic that is still routed there, but this is side effect. Also note that after reboot in case 3 MTU is set to default 1280, again side effect, because if device is associated to interface while this interface is disabled I would expect no routes created.

I hope I didn't forget anything, let me know if more detailed info is needed.

Thanks,
Kacper

First we would ideally be able to confirm the fix works

Can I test it without much hassle? Can I do opnsense-patch on them? Not sure they would apply cleanly? You mentioned before that some other changes are required, so probably I need to wait for 22.7?