Messages

106

21.7 Legacy Series / Re: DNS leaks, even after DNS catch-all port forward

« on: September 16, 2021, 06:37:39 pm »

To be more specific, chrome could be using DoH = DNS over Https, which would bypass your redirect of port 53 traffic.

Also consider that you may need to block/redirect ports 784 / 853 (?) For DNS over Quic / TLS traffic.

107

21.7 Legacy Series / Re: Unbound issues

« on: September 16, 2021, 06:35:39 pm »

Thank you both! Indeed I have strict QNAME minimisation enabled.

Going on holiday this weekend so will only be able to follow up later, but thank you very much for the pointers!

108

21.7 Legacy Series / Re: Unbound issues

« on: September 14, 2021, 11:08:37 pm »

Thanks -

1. I was afraid of this question - it was a website for Germany's far-right party (I don't like them, I did not vote them, but I had to look up their site nonetheless). Need to reset Unbound to resolver mode to check.

I have also had these issues, I think, with Apple and Microsoft sites (though no DNSBL enabled).

I have Adguard Home as primary DNS on :53 which then forwards to unbound at 127.0.0.1:5553.

Response seen in Adguard Home e.g. for www.bing.com:

Response details
Status
Allowed
DNS server
127.0.0.1:5553
Elapsed
115 ms
Response code
NXDOMAIN
Rule(s)

www.bing.com
hl2guide

Response
CNAME: a-0001.a-afdentry.net.trafficmanager.net. (ttl=21600)

Later Adguard shows proper resolution:

Response details
Status
Allowed
Elapsed
0.06 ms
Response code
NOERROR
Rule(s)

    www.bing.com
hl2guide

Response
CNAME: a-0001.a-afdentry.net.trafficmanager.net. (ttl=60)
CNAME: www-bing-com.dual-a-0001.a-msedge.net. (ttl=60)
CNAME: dual-a-0001.a-msedge.net. (ttl=60)
A: 13.107.21.200 (ttl=60)
A: 204.79.197.200 (ttl=60)

2. Statistics got reset to zero and resolve times shot up again. So: No thorough checking.

109

21.1 Legacy Series / Re: Let's Encrypt: Automation: SCP fails

« on: September 11, 2021, 10:25:48 pm »

Unfortunately (1) that is not my issue and (2) the error persists (on 21.7.2).

The key pair and scp/sftp work.

I see "ca.pem" being copied over, and the upload test manages to put the test file on the sftp server:

[root@esxi:/etc/vmware/ssl] ls -la
[...]
-rw-------    1 root     root            11 Sep 11 20:40 sftp-upload-4Q85Il
-rw-------    1 root     root            11 Sep 11 20:40 sftp-upload-HC3pkO
-rw-------    1 root     root            11 Sep 11 20:39 sftp-upload-Q5ZeG2

[root@esxi:/etc/vmware/ssl] cat sftp-upload-Q5ZeG2
upload-test

The issue seems to be that the upload script fails to execute the proper command on the sftp server (ESXi) / that ESXi does not support fsetstat:

2021-09-11T22:23:26   opnsense[84570]   /usr/local/opnsense/scripts/OPNsense/AcmeClient/upload_sftp.php: Failed uploading file '/tmp/sftp-upload-YzGRyN' to 'ca.pem' ; Cause: {"file_not_found":true,"error":"Couldn't fsetstat: No such file or directory"}

How can I get the ACME client to execute a a short script to scp the files over? i.e. how do I change the "Run Command" of the ACME client from the limited pre-defined actions to executing a local script?

110

Tutorials and FAQs / Re: HOWTO - Redirect all DNS Requests to Opnsense

« on: September 11, 2021, 09:45:23 pm »

Redirecting DNS to 127.0.0.1 seem to fail for me:

Whereas a redirect to the relevant LAN/VLAN's gateway (e.g. 192.168.1.1:53) works, a redirect to 127.0.0.1:53 does not:

nslookup www.ft.com ns2.google.com.
DNS request timed out.
    timeout was 2 seconds.
Server:  UnKnown
Address:  216.239.34.10

DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
*** Zeitüberschreitung bei Anforderung an UnKnown.

The DNS request shows up in my (adguard home) logs, but apparently the response to my client is faulty.

If redirecting to 192.168.1.1, the redirect works and the client "does not notice":

nslookup www.ft.com ns2.google.com.
Server:  ns2.google.com
Address:  216.239.34.10

Nicht autorisierende Antwort:
Name:    ft2.map.fastly.net
Addresses:  151.101.2.209
          151.101.66.209
          151.101.130.209
          151.101.194.209
Aliases:  www.ft.com

What could be the reason?

Redirecting to 127.0.0.1 would be favourable as I could apply it globally to all local LAN/VLAN/VPN interfaces, whereas redirecting to a gateway address would require individual rules for each interface (bleh!).

111

21.7 Legacy Series / Unbound issues

« on: September 11, 2021, 08:56:20 pm »

While running Unbound as a local resolver, I had come across three issues:

1. Unbound fails to resolve certain domains. I have no DNSBL in Unbound. Unbound only delivers the CNAME, but no A record. When using Unbound as DoT forwarder, it resolves the hostname normally.

2. Even though I have unchecked "Flush DNS cache during reload", the statistics and cache are cleared with every Unbound reload. This should not happen.

3. Unbound failing to start, see https://github.com/opnsense/core/issues/5150 -  I don't experience this anymore, though.

Has anyone experienced similar issues or could suggest possible (configuration) errors?

112

Intrusion Detection and Prevention / Re: Enable IPS prevents DHCP on VLANs

« on: August 27, 2021, 12:51:38 pm »

Had the same issue. It can be resolved as follows:
- disable VLAN hardware filtering. REBOOT (!!).
- enable IDS, enable promiscuous mode and only apply IDS on physical interfaces.

Then it works.

Can't stress the "REBOOT" bit enough.

113

21.1 Legacy Series / Re: Upgraded to 21.1.4 last night. DHCP woes.

« on: August 26, 2021, 08:09:19 pm »

I'll put it here in case someone finds it useful:

IDS (Suricata) in promiscuous mode and VLANs work under 21.7.1, if you (1) disable VLAN hardware filtering AND (2) reboot.

The latter (reboot) is often not explicitly stated and has caused me woes ...

114

Intrusion Detection and Prevention / Re: E-mail when alert or drop

« on: August 25, 2021, 12:33:36 pm »

I also had more success with

content<space>=<space>alert

i.e. without using quotation marks ""

115

21.7 Legacy Series / Re: Unbound behavior in General

« on: August 08, 2021, 08:22:44 pm »

Yeah, behaviour is the same -- Unbound is a resolver by nature, whether running on pfsense or opnsense (or any other OS). It won't "forward" unless you tell it to.

Further I am pretty curious about the dns behavior If I start using Unbound DNS: DNS over TLS - Does this overrule all other dns rel. settings and if in which way ?

Yes - In my (limited but existing) experience, DNS over TLS mode turns unbound into a forwarder (via DNS-over-TLS) and causes it to ignore the other settings.

116

21.7 Legacy Series / Re: Unbound won't start from GUI after 21.7.1 upgrade (from 21.7)

« on: August 08, 2021, 08:19:36 pm »

FYI, I have opened a bug on the issue tracker: https://github.com/opnsense/core/issues/5150

117

Intrusion Detection and Prevention / Re: Regarding IDS Rules not Downloading and Updating

« on: August 08, 2021, 02:39:25 pm »

I would delete the following two directories (via SSH access):

Code: [Select]

/usr/local/etc/suricata/opnsense.rules
/usr/local/etc/suricata/rules
Then reboot and download again.

Thank you -- downloading works (same as before) but the IDS -> Administration -> Rules tab still shows

Code: [Select]

No results found!
and enabling Suricata shows GUI error "Error loading IDS rules" (or something like that, copy&pasting the error did not work)

Log does not help:

Code: [Select]

2021-08-08T13:46:09 suricata[49327] [100545] <Notice> -- Stats for 'igb3': pkts: 0, drop: 0 (nan%), invalid chksum: 0
2021-08-08T13:46:09 suricata[49327] [100545] <Notice> -- Stats for 'igb2': pkts: 0, drop: 0 (nan%), invalid chksum: 0
2021-08-08T13:46:09 suricata[49327] [100545] <Notice> -- Stats for 'igb1': pkts: 22674, drop: 0 (0.00%), invalid chksum: 227
2021-08-08T13:46:08 suricata[49327] [100545] <Notice> -- Signal Received. Stopping engine.

118

Intrusion Detection and Prevention / Re: Regarding IDS Rules not Downloading and Updating

« on: August 08, 2021, 01:57:20 pm »

problems here as well -- after upgrade to 21.7, I wanted to try IDS again.

rulesets are apparently downloaded, but the "rules" tab shows no rules, suricata throws error messages about being unable to import rules and the log shows

Code: [Select]

2021-08-08T13:27:54 suricata[49327] [100545] <Warning> -- [ERRCODE: SC_ERR_NO_RULES_LOADED(43)] - 1 rule files specified, but no rule was loaded at all!
2021-08-08T13:27:54 suricata[49327] [100545] <Warning> -- [ERRCODE: SC_ERR_NO_RULES(42)] - No rule files match the pattern /usr/local/etc/suricata/opnsense.rules/suricata.rules

119

21.7 Legacy Series / Re: Unbound won't start from GUI after 21.7.1 upgrade (from 21.7)

« on: August 08, 2021, 01:19:29 pm »

Same issue here - unbound would not start automatically and the GUI button did not start it either.  Working configuration imported from 21.1 and unbound-checkconf does not complain.

The GUI "start" button only generated the following unbound log entry:

Code: [Select]

2021-08-08T13:16:31 unbound[57542] daemonize unbound dhcpd watcher.
whereas running

Code: [Select]

unbound -c /var/unbound/unbound.confcaused Unbound to start up fine.

Reboot did  not solve the issue:

Initially, Unbound started up - GUI showed it as running and log entries seem to confirm startup, but then it died again:

Code: [Select]

2021-08-08T13:28:07 unbound[2474] daemonize unbound dhcpd watcher.
2021-08-08T13:28:05 unbound[2663] daemonize unbound dhcpd watcher.
2021-08-08T13:28:00 unbound[38473] daemonize unbound dhcpd watcher.
2021-08-08T13:27:52 unbound[60119] daemonize unbound dhcpd watcher.
2021-08-08T13:27:52 unbound[25826] [25826:0] info: mesh has 0 recursion states (0 with reply, 0 detached), 0 waiting replies, 0 recursion replies sent, 0 replies dropped, 0 states jostled out
2021-08-08T13:27:52 unbound[25826] [25826:0] info: server stats for thread 3: requestlist max 0 avg 0 exceeded 0 jostled 0
2021-08-08T13:27:52 unbound[25826] [25826:0] info: server stats for thread 3: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2021-08-08T13:27:52 unbound[25826] [25826:0] info: mesh has 0 recursion states (0 with reply, 0 detached), 0 waiting replies, 0 recursion replies sent, 0 replies dropped, 0 states jostled out
2021-08-08T13:27:52 unbound[25826] [25826:0] info: server stats for thread 2: requestlist max 0 avg 0 exceeded 0 jostled 0
2021-08-08T13:27:52 unbound[25826] [25826:0] info: server stats for thread 2: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2021-08-08T13:27:52 unbound[25826] [25826:0] info: mesh has 0 recursion states (0 with reply, 0 detached), 0 waiting replies, 0 recursion replies sent, 0 replies dropped, 0 states jostled out
2021-08-08T13:27:52 unbound[25826] [25826:0] info: server stats for thread 1: requestlist max 0 avg 0 exceeded 0 jostled 0
2021-08-08T13:27:52 unbound[25826] [25826:0] info: server stats for thread 1: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2021-08-08T13:27:52 unbound[25826] [25826:0] info: mesh has 0 recursion states (0 with reply, 0 detached), 0 waiting replies, 0 recursion replies sent, 0 replies dropped, 0 states jostled out
2021-08-08T13:27:52 unbound[25826] [25826:0] info: server stats for thread 0: requestlist max 0 avg 0 exceeded 0 jostled 0
2021-08-08T13:27:52 unbound[25826] [25826:0] info: server stats for thread 0: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2021-08-08T13:27:51 unbound[25826] [25826:0] info: service stopped (unbound 1.13.1).
2021-08-08T13:27:51 unbound[25826] [25826:0] info: control cmd: dump_cache
2021-08-08T13:27:45 unbound[25826] [25826:0] info: start of service (unbound 1.13.1).
2021-08-08T13:27:45 unbound[25826] [25826:0] notice: init module 2: iterator
2021-08-08T13:27:45 unbound[25826] [25826:0] notice: init module 1: validator
2021-08-08T13:27:45 unbound[25826] [25826:0] notice: init module 0: dns64
2021-08-08T13:27:39 unbound[25826] [25826:0] notice: Restart of unbound 1.13.1.
2021-08-08T13:27:39 unbound[25826] [25826:0] info: mesh has 0 recursion states (0 with reply, 0 detached), 0 waiting replies, 0 recursion replies sent, 0 replies dropped, 0 states jostled out
2021-08-08T13:27:39 unbound[25826] [25826:0] info: server stats for thread 3: requestlist max 0 avg 0 exceeded 0 jostled 0
2021-08-08T13:27:39 unbound[25826] [25826:0] info: server stats for thread 3: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2021-08-08T13:27:39 unbound[25826] [25826:0] info: mesh has 0 recursion states (0 with reply, 0 detached), 0 waiting replies, 0 recursion replies sent, 0 replies dropped, 0 states jostled out
2021-08-08T13:27:39 unbound[25826] [25826:0] info: server stats for thread 2: requestlist max 0 avg 0 exceeded 0 jostled 0
2021-08-08T13:27:39 unbound[25826] [25826:0] info: server stats for thread 2: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2021-08-08T13:27:39 unbound[25826] [25826:0] info: mesh has 0 recursion states (0 with reply, 0 detached), 0 waiting replies, 0 recursion replies sent, 0 replies dropped, 0 states jostled out
2021-08-08T13:27:39 unbound[25826] [25826:0] info: server stats for thread 1: requestlist max 0 avg 0 exceeded 0 jostled 0
2021-08-08T13:27:39 unbound[25826] [25826:0] info: server stats for thread 1: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2021-08-08T13:27:39 unbound[25826] [25826:0] info: mesh has 0 recursion states (0 with reply, 0 detached), 0 waiting replies, 0 recursion replies sent, 0 replies dropped, 0 states jostled out
2021-08-08T13:27:39 unbound[25826] [25826:0] info: server stats for thread 0: requestlist max 0 avg 0 exceeded 0 jostled 0
2021-08-08T13:27:39 unbound[25826] [25826:0] info: server stats for thread 0: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2021-08-08T13:27:38 unbound[25826] [25826:0] info: service stopped (unbound 1.13.1).
2021-08-08T13:27:38 unbound[25826] [25826:0] info: start of service (unbound 1.13.1).
2021-08-08T13:27:38 unbound[25826] [25826:0] notice: init module 2: iterator
2021-08-08T13:27:38 unbound[25826] [25826:0] notice: init module 1: validator
2021-08-08T13:27:38 unbound[25826] [25826:0] notice: init module 0: dns64
2021-08-08T13:27:31 unbound[66648] daemonize unbound dhcpd watcher.
2021-08-08T13:27:31 unbound[2967] [2967:0] info: mesh has 0 recursion states (0 with reply, 0 detached), 0 waiting replies, 0 recursion replies sent, 0 replies dropped, 0 states jostled out
2021-08-08T13:27:31 unbound[2967] [2967:0] info: server stats for thread 3: requestlist max 0 avg 0 exceeded 0 jostled 0
2021-08-08T13:27:31 unbound[2967] [2967:0] info: server stats for thread 3: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2021-08-08T13:27:31 unbound[2967] [2967:0] info: mesh has 0 recursion states (0 with reply, 0 detached), 0 waiting replies, 0 recursion replies sent, 0 replies dropped, 0 states jostled out
2021-08-08T13:27:31 unbound[2967] [2967:0] info: server stats for thread 2: requestlist max 0 avg 0 exceeded 0 jostled 0
2021-08-08T13:27:31 unbound[2967] [2967:0] info: server stats for thread 2: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2021-08-08T13:27:31 unbound[2967] [2967:0] info: mesh has 0 recursion states (0 with reply, 0 detached), 0 waiting replies, 0 recursion replies sent, 0 replies dropped, 0 states jostled out
2021-08-08T13:27:31 unbound[2967] [2967:0] info: server stats for thread 1: requestlist max 0 avg 0 exceeded 0 jostled 0
2021-08-08T13:27:31 unbound[2967] [2967:0] info: server stats for thread 1: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2021-08-08T13:27:31 unbound[2967] [2967:0] info: mesh has 0 recursion states (0 with reply, 0 detached), 0 waiting replies, 0 recursion replies sent, 0 replies dropped, 0 states jostled out
2021-08-08T13:27:31 unbound[2967] [2967:0] info: server stats for thread 0: requestlist max 0 avg 0 exceeded 0 jostled 0
2021-08-08T13:27:31 unbound[2967] [2967:0] info: server stats for thread 0: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
2021-08-08T13:27:31 unbound[2967] [2967:0] info: service stopped (unbound 1.13.1).
2021-08-08T13:27:31 unbound[2967] [2967:0] info: control cmd: dump_cache
2021-08-08T13:27:19 unbound[2967] [2967:0] info: control cmd: list_local_data
2021-08-08T13:27:19 unbound[2967] [2967:0] info: start of service (unbound 1.13.1).
2021-08-08T13:27:19 unbound[2967] [2967:0] notice: init module 2: iterator
2021-08-08T13:27:19 unbound[2967] [2967:0] notice: init module 1: validator
2021-08-08T13:27:19 unbound[2967] [2967:0] notice: init module 0: dns64
2021-08-08T13:27:12 unbound[87717] daemonize unbound dhcpd watcher.
[*** REBOOT ***]
This recurs several times in the log, thereafter it is only the

Code: [Select]

unbound[57542] daemonize unbound dhcpd watcher.
Manually starting unbound works.

Could this be an issue with Unbound having issues with running on a different port than 53 (I have adguard home running as general DNS server), even though Unbound does not complain about ports?

120

21.1 Legacy Series / Re: 21.1.9 Upgrade & Insight Aggregator CPU Usage

« on: July 28, 2021, 12:41:24 pm »

roughly similar experience here, too:
1. update -> 100% CPU on one core, some Python 3.7 process.
2. reboot solved the issue instantly, normal CPU usage since.