The new unbound reporting is pretty cool

Started by senser, January 27, 2023, 10:57:28 PM

Previous topic - Next topic
Quote from: senser on January 27, 2023, 11:17:52 PM
This needs some explanation:

Hi there,

Quote from: Fright on January 28, 2023, 08:13:05 AM
Maybe if you enable additional fields (Type and Return Code), something will become clearer?

I think tuto2, as the author, knows all the insides of the new feature )


Indeed, I'm not seeing the type here which could explain the behaviour. To be specific, I've noticed clients using the dns record type HTTPS right after normal A/AAAA queries, which is a relatively new (and incomplete) RFC standard. HTTPS does not return an IP address and as such does not qualify for blocking since clients wouldn't have enough information to establish a connection. As far as I know the only thing it returns is a CNAME, which in turn is part of the blocklist :)

For context, HTTPS record types are used to detect whether clients can immediately establish an HTTPS connections, instead of an upgrade from HTTP.

In the future we could consider being more stringent with more record types, but the reason this isn't done is because Unbound's behaviour is a bit unpredictable in whether the information we need to make the reporting happen is available in the first place based on the record type.

Cheers,
Stephan

Quote from: wtremmel on January 29, 2023, 10:35:27 AM
For whatever reason it stopped working after two hours. Perhaps at the same time I uploaded a local file and reloaded unbound. Any idea on how to get it started again?

If you notice such things it might be best to take a look at the Unbound log and see if anything has happened there. I've taken great care in the implementation to make sure that edge cases are at the very least reported there so we can improve on them based on your feedback :)

Quote from: tuto2 on January 30, 2023, 10:46:15 AM

If you notice such things it might be best to take a look at the Unbound log and see if anything has happened there. I've taken great care in the implementation to make sure that edge cases are at the very least reported there so we can improve on them based on your feedback :)

Done so. Thanks for the great feature!
See https://github.com/opnsense/core/issues/6284
Want to know about IP or BGP? www.de-cix.net/academy

January 30, 2023, 04:04:23 PM #18 Last Edit: January 30, 2023, 05:05:52 PM by dumbo
Hi.

I also really love the new unbound reporting feature. And if that's only the beginning I'm really looking forward for more to come. Thx for that!

I do have one issue within my stats.

It's showing a lot of
localhost PTR 10.1.168.192.in-addr.arpa. Pass Local-data NOERROR 0ms 0 None....
messages.

192.168.1.10 is my notebook within the network - and actually the only device, as it's a testing setup for OPNsense.

It's by far No. 1 within my top passed domains.

What am I doing wrong?

Mine has also stopped working a few times since 23.1 release, all i do is restart unbound and reporting works again. DNS resolution never stops working, just the reporting

Quote from: wtremmel on January 29, 2023, 10:35:27 AM
For whatever reason it stopped working after two hours. Perhaps at the same time I uploaded a local file and reloaded unbound. Any idea on how to get it started again?

Quote from: danderson on January 30, 2023, 04:18:32 PM
Mine has also stopped working a few times since 23.1 release, all i do is restart unbound and reporting works again. DNS resolution never stops working, just the reporting

Quote from: wtremmel on January 29, 2023, 10:35:27 AM
For whatever reason it stopped working after two hours. Perhaps at the same time I uploaded a local file and reloaded unbound. Any idea on how to get it started again?

Unbound is decoupled from the reporting logic to prevent unnecessary DNS issues in a network. That said, it would be helpful if you're able to share logs specifically at the point of failure from either the GUI or /var/log/resolver/.

Quote from: dumbo on January 30, 2023, 04:04:23 PM
Hi.

I also really love the new unbound reporting feature. And if that's only the beginning I'm really looking forward for more to come. Thx for that!
You're welcome :) Feedback and suggestions are welcome.
Quote from: dumbo on January 30, 2023, 04:04:23 PM
I do have one issue within my stats.

It's showing a lot of
localhost PTR 192.186.1.10.in-addr.arpa. Pass Local-data NOERROR 0ms 0 None....
messages.

192.168.1.10 is my notebook within the network - and actually the only device, as it's a testing setup for OPNsense.

It's by far No. 1 within my top passed domains.

What am I doing wrong?

You're not doing anything wrong, some process is trying to figure out the hostname of that specific client using a reverse DNS lookup. If its Unbound itself you could help pinpoint the issue by running # opnsense-patch 44e9dc25b
and optionally reset the DNS data, but restarting Unbound.

Relevant commit: https://github.com/opnsense/core/commit/44e9dc25b8c1dd8138733658eff260dca7d61edb

And report back if the number of queried PTR records is reduced.

Quote from: tuto2 on January 30, 2023, 05:03:05 PM
...
and optionally reset the DNS data, but restarting Unbound.


Thx. Will test it. Patch applied and Unbound restarted. But how do I reset the DNS data of the new Unbound Interface?

Quote from: tuto2 on January 30, 2023, 05:03:05 PM
And report back if the number of queried PTR records is reduced.

Just to report back - the number went to nearly zero. The patch is working perfectly!

But how can I reset stats to get rid of all these earlier PTR records within my top domains.

Quote from: dumbo on January 30, 2023, 07:20:28 PM
Quote from: tuto2 on January 30, 2023, 05:03:05 PM
And report back if the number of queried PTR records is reduced.

Just to report back - the number went to nearly zero. The patch is working perfectly!

But how can I reset stats to get rid of all these earlier PTR records within my top domains.

Good to hear, thanks for testing! You can reset the DNS data in Reporting -> Settings -> Reset DNS data.

Just to say I find the new Unbound reporting very useful, and I would like to suggest a couple of enhancements

1. Auto refresh

2, On the list, when filtering with the term 'Block' all the clients are localhost, whereas without a term, or even with the term 'Pass' the correct client is shown. Having the client in the Block situation would assist in tracing malicious queries.

Quote from: aimdev on January 31, 2023, 07:52:07 AM
1. Auto refresh

To be honest I'm not seeing a big use case here. Why and where would you consider this to be most useful?

Quote from: aimdev on January 31, 2023, 07:52:07 AM
2, On the list, when filtering with the term 'Block' all the clients are localhost, whereas without a term, or even with the term 'Pass' the correct client is shown. Having the client in the Block situation would assist in tracing malicious queries.

I cannot reproduce this, I think this is mostly a sorting issue as a lot of queries from localhost might mean they show up first. What you can do is sort on "Block" (toggle the sorting caret in the column header) and search on specific clients, or do the exact opposite and sort on clients.

1. I often when testing the firewall gaze at the Live View firewall log, which auto updates, just makes life a bit easier.

2. No for me, all local even with 'All' set as the option for display. Possibly it may be how I have Unbound setup.


Quote from: tuto2 on January 30, 2023, 08:35:39 PM
Good to hear, thanks for testing! You can reset the DNS data in Reporting -> Settings -> Reset DNS data.

Hi. I can say that the patch works as it should! All nonsense requests gone within stats! Really nice job.

So will this patch make it into final or is there any other issue which has to be solved?

As a OPNsense newbie - what do I have to do if it will make it into a official patch release? Deleting the patch and updating OPNsense - or just do nothing?

Quote from: dumbo on January 31, 2023, 08:56:08 AM
So will this patch make it into final or is there any other issue which has to be solved?

There are other minor fixes being prepared as a batch for the next minor release. This one will be a part of it.

Quote from: dumbo on January 31, 2023, 08:56:08 AM\
As a OPNsense newbie - what do I have to do if it will make it into a official patch release? Deleting the patch and updating OPNsense - or just do nothing?

No need to do anything. Just update as you would when a new minor release becomes available.