I've been running opnsense 23.1 flawlessly I think pretty much from release (I have had the odd issue with opnsense prior to 23.1, but effectively 99.9% uptime for many years since switching from pfsense).
Yesterday I updated to 23.7.3 and immediately starting having issues.
First off was that I was not getting any DNS responses (I could connect via IPs OK) - for some reason "Enable Unbound" was not ticked anymore. Ticked, now getting DNS responses once unbound started.
HOWEVER, how my custom domain does no longer work.
For example, if I had server1.customdomain it would originally have responded with the IP address.
Now I get:
Server: 10.10.1.1
Address: 10.10.1.1#53
** server can't find server1.customdomain: NXDOMAIN
server1.local DOES however work.
All my overrides still work too.
Thankfully as I run this as a VM I rebooted into my 23.1 instance - everything working again.
What has changed in 23.7 that A) meant that unbound DNS did not automatically start because it had been unchecked to do so and B) Why is my custom domain stuff working?
In the mean time I've reverted back to 23.1, as a number of things rely on the device.customdomain tag and thus dont work properly...!
I've had a fiddle, but at the end of the day this is a home setup so I've run out of debug time and skills!
Many Thanks!
I for one actually had the same experience post-upgrade and also rolled back without too much investigative analysis. VM as well.
Don't really have anything to add sorry, just mirroring your experience. This was back in 23.7.0 though, so first release.
Will keep an eye on this thread for insights. I'm in no rush to upgrade home again, but would like to. Just waiting patiently,
QuoteJust waiting patiently
There are currently no open issues regarding DNS post 23.7.3
Quote from: newsense on September 04, 2023, 08:21:34 AM
QuoteJust waiting patiently
There are currently no open issues regarding DNS post 23.7.3
This may be the case, but the upgrade for me has caused two different issues, both DNS related.
Some guidance as to why the domain stuff isn't working (or what to look for) may yield clues, but for now I'm sticking with 23.1.
Edit: Switched between 23.7.3 and 23.1 a few times. Black and white working/not working for me.
23.1.11 device.customdomain works
23.7.3 device.customdomain does NOT work
So there is clearly something between 23.1.11 and 23.7.3 that is causing this, if someone can provide some guidance as to where the problem might lie then I can do some more digging.
I believe a few people had trouble where unbound got disabled during the upgrade but I don't think it was a common thing or that it was ever determined why.
How do you have your custom domain configured and what do the Unbound logs show?
Seems like a pretty significant bug if multiple users have seen it happen ;D
As for custom domain, I think the only place it is set is here:
SYSTEM: SETTINGS: GENERAL
Domain "customdomain"
I'll have to boot back into 23.7 to get some unbound logs.
It's probably fixable under 30 minutes with the right error message attached.
If you have the 23.7 where Unbound is still disabled right after upgrade you can try running:
# /usr/local/opnsense/mvc/script/run_migrations.php
# opnsense-log | grep run_migrations
Cheers,
Franco
I had issues starting Unbound after updating to 23.7. The logs gave no clues, but I stumbled on to a post (can't find it ATM) where they discussed a bug when running Unbound on "not all" interfaces, as I was.
The fix was to:
Go to Services -> Unbound DNS -> General
Choose to Clear All beneath the Network Interfaces drop down, Apply
Choose your preffered interface(s) again, and reapply.
Worked for me, hope this helps.
Yeah this one was fixed in 23.7.1:
https://github.com/opnsense/changelog/blob/01889aa7eb9c4e75c3aff6dde9abeca18f16ea55/community/23.7/23.7.1#L28
Cheers,
Franco
I'll spend a bit of time messing about with this this weekend to get answers.
Couldn't turn the internet off yet, I'd get an unhappy other half. ;D
Same issue since updating cant acess any servers on a local hostname... any fixes or should i just to back to old version.
@Franco
Hi,
I have the same issue, after upgrade to 23.7.3 Unboud was disabled so I enable it but all my overrides dosn't work.
I run the following command:
/usr/local/opnsense/mvc/script/run_migrations.php
** OPNsense\Unbound\Unbound Migration failed, check log for details
and then
opnsense-log | grep run_migrations
<147>1 2023-09-13T08:23:04+02:00 localhost config 2076 - [meta sequenceId="29"] #1 /usr/local/opnsense/mvc/script/run_migrations.php(54): OPNsense\Base\BaseModel->runMigrations()
<147>1 2023-09-13T08:27:54+02:00 opnsense-casa.proximanet.net config 80369 - [meta sequenceId="8"] #1 /usr/local/opnsense/mvc/script/run_migrations.php(54): OPNsense\Base\BaseModel->runMigrations()
<147>1 2023-09-13T09:56:22+02:00 opnsense-casa.proximanet.net config 50858 - [meta sequenceId="6"] #1 /usr/local/opnsense/mvc/script/run_migrations.php(54): OPNsense\Base\BaseModel->runMigrations()
Then I tried also to enable the access list and insert my subnets, but it still doesn't work...
Can you help me, please?
Thank you
Can you give the full log and the config.xml section of your overrides? That would help reproduce this quickly. You can also PM me or better yet send to franco@opnsense.org
Cheers,
Franco
Hi Franco,
I sent you the unbound section of config.xml via email.
Thak you
With mic's help we now have a POC:
https://github.com/opnsense/core/pull/6844
I don't want to advertise it too much though. It needs to be discussed internally first and not everyone is at the office at the moment.
Cheers,
Franco
Quote from: BasilBasil on September 07, 2023, 11:32:58 AM
Seems like a pretty significant bug if multiple users have seen it happen ;D
Depends on the scale. Was it handful of users in a group of 10 or handful of users in a group of 100000?
Quote from: franco on September 13, 2023, 04:45:54 PM
With mic's help we now have a POC:
https://github.com/opnsense/core/pull/6844
I don't want to advertise it too much though. It needs to be discussed internally first and not everyone is at the office at the moment.
Just to clarify for my own edification, the issue is that there are interfaces that had been added to the access list but disabled and that's what causes the Unbound failures? I'm not familiar enough with the inner workings of OPNSense to tell what's going on from the discussion and commit.
The way this was designed was that interfaces were added to the list, but if you deleted them in the interfaces section they ended up as "garbage" entries in the unbound configuration. These are not problematic per se, but once moved to MVC the data models will realize that one interface is not a valid option and prevent setting it in the config. Unfortunately this also affects data migration from one configuration location to the next, which was carried out in 23.7 also moving the path of the "enable" flag of unbound ending up unbound not being enabled because data could not be migrated.
This was all done in the spirit of providing a full API for Unbound, which wasn't the case before.
I've spent more time on this for discarding invalid values on migration and 23.7.5 will have all the fixes. The main commit is https://github.com/opnsense/core/commit/6898bc883 but don't try to opnsense-patch this individually. The topic is a bit more complex than hoped for and other changes in the area were required as well.
Cheers,
Franco
Quote from: franco on September 20, 2023, 02:22:39 PM
The way this was designed was that interfaces were added to the list, but if you deleted them in the interfaces section they ended up as "garbage" entries in the unbound configuration. These are not problematic per se, but once moved to MVC the data models will realize that one interface is not a valid option and prevent setting it in the config. Unfortunately this also affects data migration from one configuration location to the next, which was carried out in 23.7 also moving the path of the "enable" flag of unbound ending up unbound not being enabled because data could not be migrated.
This was all done in the spirit of providing a full API for Unbound, which wasn't the case before.
I've spent more time on this for discarding invalid values on migration and 23.7.5 will have all the fixes. The main commit is https://github.com/opnsense/core/commit/6898bc883 but don't try to opnsense-patch this individually. The topic is a bit more complex than hoped for and other changes in the area were required as well.
Cheers,
Franco
Thanks for the clarification. This isn't something I ran into but just wanted to understand.
Finally managed to look into this today.
Upgrading to: OPNsense 23.7.1_3-amd64 from 23.1.11_2 via the GUI.
In SERVICES: UNBOUND DNS: GENERAL
--> Enable Unbound - Gets unchecked during the upgrade
Checking this allows for DNS resolution to work again but not device.customdomain.
--> Register DHCP Static Mappings however also gets unchecked during the upgrade.
Checking that gets me back to how it was in 23.1 in that device.customdomain works.
Edit: It sounds like the above from Franco will resolve this issue. I've still got my 23.1 VM so I'll try the fix when 23.7.5 comes out by doing a full upgrade again.