OPNsense Forum
Archive => 23.1 Legacy Series => Topic started by: senser on January 27, 2023, 10:57:28 pm
-
:)
Thanks!
-
This needs some explanation:
-
https://discourse.pi-hole.net/t/cant-add-https-app-measurement-com-sdk-exp-to-blocking-list/38888/4
-
Ah. It’s a feature borrowed from the pi-hole project. Could it somehow make sense that only the first query qualifies as blocked? It is certainly unintuitive but maybe cache hits do not show as blocked, even though localhost is returned!? I can see how this might be difficult to catch…
-
sorry, I thought you were interested in a strange request with "https://" in the hostname :)
link more about it..
not sure if "function is borrowed", maybe the "idea is seen" ?
Could it somehow make sense that only the first query qualifies as blocked?
yep, looks weird (if something has not changed in 15 seconds between requests that allowed request and needed recursion?).
Maybe if you enable additional fields (Type and Return Code), something will become clearer?
I think tuto2, as the author, knows all the insides of the new feature )
-
Pretty cool, indeed! :)
Anybody figured out how to show a complete DNS query? Most of the FQDNs are cut off for me, and hovering over does not show more, either.
-
@athurdent
can you test with
opnsense-patch -a kulikov-a 3b22b18please?
-
Looks a lot better now, thank you. I only see truncated entries on .in-addr.arpa now
Ideally the columns would be resizable, and hopefully we'll get there someday.
-
I only see truncated entries on .in-addr.arpa now
hm, should be no difference for ptr requests..can you share a screenshot?
columns would be resizable
iirc bootgrid doesn't support this (it would require a rather complicated migration to something like datatables) and imho this is not a universal solution - fqdn can be too long ;)
-
@athurdent
can you test with
opnsense-patch -a kulikov-a 3b22b18please?
That is awesome, thank you very much! Works perfectly fine here. :)
-
@athurdent
thanks for the feedback!
I'll try to make a pr if there are no problems with the ptr requests records (i did not notice such, but @newsense mentioned)
-
@athurdent
thanks for the feedback!
I'll try to make a pr if there are no problems with the ptr requests records (i did not notice such, but @newsense mentioned)
Thank you for the fix! :)
My PTR all look OK.
-
For whatever reason it stopped working after two hours. Perhaps at the same time I uploaded a local file and reloaded unbound. Any idea on how to get it started again?
-
@wtremmel
may be REPORTING: SETTINGS -> "Reset DNS Data" may help
-
hm, should be no difference for ptr requests..can you share a screenshot?
Sure, this is what I'm seeing.
-
This needs some explanation:
Hi there,
Maybe if you enable additional fields (Type and Return Code), something will become clearer?
I think tuto2, as the author, knows all the insides of the new feature )
Indeed, I'm not seeing the type here which could explain the behaviour. To be specific, I've noticed clients using the dns record type HTTPS right after normal A/AAAA queries, which is a relatively new (and incomplete) RFC standard. HTTPS does not return an IP address and as such does not qualify for blocking since clients wouldn't have enough information to establish a connection. As far as I know the only thing it returns is a CNAME, which in turn is part of the blocklist :)
For context, HTTPS record types are used to detect whether clients can immediately establish an HTTPS connections, instead of an upgrade from HTTP.
In the future we could consider being more stringent with more record types, but the reason this isn't done is because Unbound's behaviour is a bit unpredictable in whether the information we need to make the reporting happen is available in the first place based on the record type.
Cheers,
Stephan
-
For whatever reason it stopped working after two hours. Perhaps at the same time I uploaded a local file and reloaded unbound. Any idea on how to get it started again?
If you notice such things it might be best to take a look at the Unbound log and see if anything has happened there. I've taken great care in the implementation to make sure that edge cases are at the very least reported there so we can improve on them based on your feedback :)
-
If you notice such things it might be best to take a look at the Unbound log and see if anything has happened there. I've taken great care in the implementation to make sure that edge cases are at the very least reported there so we can improve on them based on your feedback :)
Done so. Thanks for the great feature!
See https://github.com/opnsense/core/issues/6284
-
Hi.
I also really love the new unbound reporting feature. And if that's only the beginning I'm really looking forward for more to come. Thx for that!
I do have one issue within my stats.
It's showing a lot of
localhost PTR 10.1.168.192.in-addr.arpa. Pass Local-data NOERROR 0ms 0 None.... messages.
192.168.1.10 is my notebook within the network - and actually the only device, as it's a testing setup for OPNsense.
It's by far No. 1 within my top passed domains.
What am I doing wrong?
-
Mine has also stopped working a few times since 23.1 release, all i do is restart unbound and reporting works again. DNS resolution never stops working, just the reporting
For whatever reason it stopped working after two hours. Perhaps at the same time I uploaded a local file and reloaded unbound. Any idea on how to get it started again?
-
Mine has also stopped working a few times since 23.1 release, all i do is restart unbound and reporting works again. DNS resolution never stops working, just the reporting
For whatever reason it stopped working after two hours. Perhaps at the same time I uploaded a local file and reloaded unbound. Any idea on how to get it started again?
Unbound is decoupled from the reporting logic to prevent unnecessary DNS issues in a network. That said, it would be helpful if you're able to share logs specifically at the point of failure from either the GUI or /var/log/resolver/.
-
Hi.
I also really love the new unbound reporting feature. And if that's only the beginning I'm really looking forward for more to come. Thx for that!
You're welcome :) Feedback and suggestions are welcome.
I do have one issue within my stats.
It's showing a lot of
localhost PTR 192.186.1.10.in-addr.arpa. Pass Local-data NOERROR 0ms 0 None.... messages.
192.168.1.10 is my notebook within the network - and actually the only device, as it's a testing setup for OPNsense.
It's by far No. 1 within my top passed domains.
What am I doing wrong?
You're not doing anything wrong, some process is trying to figure out the hostname of that specific client using a reverse DNS lookup. If its Unbound itself you could help pinpoint the issue by running # opnsense-patch 44e9dc25band optionally reset the DNS data, but restarting Unbound.
Relevant commit: https://github.com/opnsense/core/commit/44e9dc25b8c1dd8138733658eff260dca7d61edb
And report back if the number of queried PTR records is reduced.
-
...
and optionally reset the DNS data, but restarting Unbound.
Thx. Will test it. Patch applied and Unbound restarted. But how do I reset the DNS data of the new Unbound Interface?
-
And report back if the number of queried PTR records is reduced.
Just to report back - the number went to nearly zero. The patch is working perfectly!
But how can I reset stats to get rid of all these earlier PTR records within my top domains.
-
And report back if the number of queried PTR records is reduced.
Just to report back - the number went to nearly zero. The patch is working perfectly!
But how can I reset stats to get rid of all these earlier PTR records within my top domains.
Good to hear, thanks for testing! You can reset the DNS data in Reporting -> Settings -> Reset DNS data.
-
Just to say I find the new Unbound reporting very useful, and I would like to suggest a couple of enhancements
1. Auto refresh
2, On the list, when filtering with the term 'Block' all the clients are localhost, whereas without a term, or even with the term 'Pass' the correct client is shown. Having the client in the Block situation would assist in tracing malicious queries.
-
1. Auto refresh
To be honest I'm not seeing a big use case here. Why and where would you consider this to be most useful?
2, On the list, when filtering with the term 'Block' all the clients are localhost, whereas without a term, or even with the term 'Pass' the correct client is shown. Having the client in the Block situation would assist in tracing malicious queries.
I cannot reproduce this, I think this is mostly a sorting issue as a lot of queries from localhost might mean they show up first. What you can do is sort on "Block" (toggle the sorting caret in the column header) and search on specific clients, or do the exact opposite and sort on clients.
-
1. I often when testing the firewall gaze at the Live View firewall log, which auto updates, just makes life a bit easier.
2. No for me, all local even with 'All' set as the option for display. Possibly it may be how I have Unbound setup.
-
Good to hear, thanks for testing! You can reset the DNS data in Reporting -> Settings -> Reset DNS data.
Hi. I can say that the patch works as it should! All nonsense requests gone within stats! Really nice job.
So will this patch make it into final or is there any other issue which has to be solved?
As a OPNsense newbie - what do I have to do if it will make it into a official patch release? Deleting the patch and updating OPNsense - or just do nothing?
-
So will this patch make it into final or is there any other issue which has to be solved?
There are other minor fixes being prepared as a batch for the next minor release. This one will be a part of it.
\
As a OPNsense newbie - what do I have to do if it will make it into a official patch release? Deleting the patch and updating OPNsense - or just do nothing?
No need to do anything. Just update as you would when a new minor release becomes available.
-
2. No for me, all local even with 'All' set as the option for display. Possibly it may be how I have Unbound setup.
How is Unbound configured? Forwarding, special advanced options, DoT etc.
The total result set of the "live" view is limited to a 1000 entries for performance reasons. It might be that localhost is taking them all up. If there are specific clients you wish to view you can also click on a client in the "overview" page in the client graph and it will present you with a view of this clients' activity within that specific time period.
-
DOT
Thanks for the advice.
-
Unbound reporting is really cool. I'm a new user to OPNsense this month and am loving this new feature. One very minor suggestion would be to change the background font for the details tab to work better with the dark mode themes (like cicada or vicuna). I usually have to change to the default OPNsense theme to read that tab. Otherwise, fantastic work!
-
@SpinningRust
https://github.com/opnsense/plugins/issues/3290
-
Wonderful, I'm glad it has been put into the backlog.
-
Hi,
is this only a "design" glitch, or why does it block the A records but not the HTTPS records?
metrics.icloud.com A record blocked
metrics.icloud.com HTTPS record NOT blocked
-
Like the others I love the new reporting, however, with this new implementation with the python module that handles the DNSBL, what's the work around to allow bypassing the DNSBL?
It used to be using tags or views, but those won't apply now that the dnsbl file is in .json format.
-
@dumbo
not sure about the 'glitch'
HTTPS RR is pretty new. for now dnsbl is applied to A/AAA/CNAME records
-
Hi,
is this only a "design" glitch, or why does it block the A records but not the HTTPS records?
metrics.icloud.com A record blocked
metrics.icloud.com HTTPS record NOT blocked
https://forum.opnsense.org/index.php?topic=32127.msg155508#msg155508
-
Like the others I love the new reporting, however, with this new implementation with the python module that handles the DNSBL, what's the work around to allow bypassing the DNSBL?
It used to be using tags or views, but those won't apply now that the dnsbl file is in .json format.
If you're referring to single domains, you can use the "whitelist domains" field.
If you're referring to networks, not really possible.
-
@tuto2
im afraid https rr could provide ip via hints ..
https://datatracker.ietf.org/doc/html/draft-ietf-dnsop-svcb-https-01#section-6.4
-
@tuto2
im afraid https rr could provide ip via hints ..
https://datatracker.ietf.org/doc/html/draft-ietf-dnsop-svcb-https-01#section-6.4
Ok, if we're going to add HTTPS as a record type to block, would you mind helping by testing this locally? I'll put up a patch tomorrow.
-
@tuto2
sure )
-
@tuto2
sure )
@Fright Can you test with # opnsense-patch e0469001a?
-
@Fright Can you test with # opnsense-patch e0469001a?
Is this the patch with also blocking HTTPS requests?
-
Is this the patch with also blocking HTTPS requests?
Yes, https://github.com/opnsense/core/commit/e0469001a672cf67cec126b7fe80e20bac6bfea1.
-
@tuto2
5 hours work (the last 3 - with partial forwarding from production DNS servers to a test server) - everything looks fine. dig shows NODATA (rcode 0 answer:0) for blocked https RRs
thanks!
-
dig shows NODATA (rcode 0 answer:0) for blocked https RRs
Thanks for taking the time to test it :)
-
I can also confirm that this patch is working on my system for HTTPS type now.
# opnsense-patch e0469001a
-
Is this the patch with also blocking HTTPS requests?
Yes, https://github.com/opnsense/core/commit/e0469001a672cf67cec126b7fe80e20bac6bfea1.
Thx. Can confirm - patch is working.
Another question concerning Unbound:
What did you all choose as ' Local Zone Type'?
Transparent or Static?
And does it make any difference concerning the Unbound reporting?
-
Hi,
enclosed a screenshot of my top passed domains from my testing system.
Why do I have so many _dns.resolver.arpa traffic and this other lb._dns-sd.udp.... traffic (the Subnet is one of my main VLANs)?
Am I doing something wrong or is this normal behavior?
-
hi
Am I doing something wrong
no
_dns.resolver.arpa is for Discovery of Designated Resolvers (DDR) (https://datatracker.ietf.org/doc/draft-ietf-add-ddr/)
lb._dns-sd.udp. is for DNS Service Discovery (DNS-SD) (https://www.rfc-editor.org/rfc/rfc6763)
-
Another question concerning Unbound:
What did you all choose as ' Local Zone Type'?
Transparent or Static?
And does it make any difference concerning the Unbound reporting?
Doesn't make a difference for Unbound reporting. It only relates to the system domain and it only configures how Unbound should respond to a query for this domain, which is picked up either way in the reporting section.
-
The only hing that's disappointing to me is when using the color scheme "os-theme-rebellion". The details page is barely readable.
-
The only hing that's disappointing to me is when using the color scheme "os-theme-rebellion". The details page is barely readable.
Contributions on the community plugins are welcome :)
-
Looks like with todays update the unbound reporting tool is broken. "No results found!" for me.
Tried to reset DNS Data, tried a reboot... Doesn't work.
-
Looks like with todays update the unbound reporting tool is broken. "No results found!" for me.
Tried to reset DNS Data, tried a reboot... Doesn't work.
When mine rebooted after this latest upgrade, the reporting tool showed no numbers, but after a second reboot, it seems to be working well. (The startup beep sequence also seemed slow the first time, but had its normal cadence on the second reboot. I've heard that slow beep sequence occasionally on previous upgrades - I'm not sure what it means, but when I hear it, I typically reboot again, "just in case".)
-
... but after a second reboot, it seems to be working well. (The startup beep sequence also seemed slow the first time, but had its normal cadence on the second reboot. I've heard that slow beep sequence occasionally on previous upgrades - I'm not sure what it means, but when I hear it, I typically reboot again, "just in case".)
Thx for your feedback. already restarted several times - but doesn't work. :(
-
Unbound Reporting broke for me as well after the latest 23.1.1 update, however I managed to get it working again.
I had to disable Unbound reporting (uncheck the check box under Reporting > Settings and click save), I also cleared the unbound statistics.
I then made sure the following log settings under Unbound > Advanced were checked/Enabled
Log Queries
Log Replies
Tag Queries and Replies
After that I rebooted OPNsense, waited for a few minutes for everything to settle down and then went back to Reporting > Settings and enabled Unbound reporting again, clicked save and wa-la Unbound Reporting is working again.
-
Look here (https://forum.opnsense.org/index.php?topic=32494.msg157229#msg157229).
Patch opnsense-patch -a kulikov-a 404b9d5
-
Hi,
the new patch works like it should. Thanks for that.
Another issue I found out is, that after a day or so the hostname resolution stops working. Tried to restart unbound and DHCPv4 - but doesn't help at all. It stopped showing hostnames of my network devices and now only shows ip addresses.
Nothing special to see within Unbound log (or I didn't find it).
Anybody else also having those issues?
-
Right. Same here. But not every Client is affected. For example some Phones (Android) are OK.
-
Is anyone only getting recursive results? I have unbound configured to use DoT but every request in Details shows Recursion.
Is this a bug in reporting or is Unbound ignoring my DoT config?