Unbound DNS keeps crashing every 2-3 mins.

[apoorv569]

I used to use Pi-Hole as my network wide ad-blocker and for local DNS so I can have some like nas.homeserver.lan for all my services I host at home, but I recently learned about Unbound DNS in OPNsense and switched to it, I use DNS over TLS with quad9 for my upstream DNS, I also have various blocklists enabled,

• Abuse.ch - ThreatFox IOC
• AdGuard List
• EasyList
• EasyPrivacy
• Steven Black List
• YoYo List
• Ads Blocklist
• NSFW Blocklist
• Threat Intelligence Feeds
• Fake - scams / fakes
• Pop-Up Ads
• Badware Hoster blocking

and I have various overrides mimicking what I had in Pi-Hole for all my services such as nas.homeserver.lan also added few of my domains for split horizon DNS so the traffic can stay local when I am at home. I also have a floating rule that blocks all port 53 traffic on all my interfaces/VLANs except WAN and and WG interfaces and I also have a destination NAT rule that forwards all port 53 traffic to 127.0.0.1 to force all traffic via Unbound basically.
Now everything works fine, except every few mins, IDK if it crashes or what but DNS resolutions keep failing and I see SERVFAIL error in the Unbound logs and the only solution to this is to restart the Unbound DNS service couple of times and the cycle continues of it crashing and me restarting it fix the issue.
I have attached some photos for my various settings and rules I have regarding this.