Topics

1

22.7 Legacy Series / Chrony - NTS server

« on: December 06, 2022, 10:37:20 pm »

I noticed that the standard installation of the chrony plugin may not work as an NTS server. Can't tell for sure as I am not using a clean install. So putting this out here in case (1) it helps someone and/or (2) someone can help me improve / simplify my setup.

my issues:
chrony clients could not connect because of certificate errors.
1. the chrony plugin in server mode needs read access to the certificate and key. Without it, I got the following error:

Code: [Select]

# chronyd -Q -t 3 'server [myserver] iburst nts maxsamples 1'
...
... TLS handshake with [myserver] failed : The TLS connection was non-properly terminated.
...
2. the certificate needs to be the full chain. Without it, I got

Code: [Select]

# chronyd -Q -t 3 'server [myserver] iburst nts maxsamples 1'
...
... TLS handshake with [myserver] failed : Error in the certificate verification. The certificate is NOT trusted. The certificate issuer is unknown....
3. the standard OPNsense certificate & key at /var/etc/cert.pem / ..key.pem are read root only. I did not want to grant chrony access to it or include chrony in the "wheel" group.

What did I do?

1. set up a DNS alias for time.MYDOMAIN.TLD
2. set up a separate letsencrypt certificate for time.MYDOMAIN.TLD
3. set up an automation to have the certificate (full chain + key) be copied over to /var/lib/chrony and chmod those to chronyd and restart chrony:
a) set up the following backend action (https://docs.opnsense.org/development/backend/configd.html):

Code: [Select]

[NTS_renewal]
command:cp /var/etc/acme-client/home/time.XXXXX/fullchain.cer /var/lib/chrony/chrony.fullchain.cer && cp /var/etc/acme-client/home/time.XXXXX/time.XXXXX.key /var/lib/chrony/chrony.key && cp /var/etc/acme-client/home/time.XXXXX/time.XXXXX.cer /var/lib/chrony/chrony.cer && chown chronyd /var/lib/chrony/chrony.key /var/lib/chrony/chrony.cer /var/lib/chrony/chrony.fullchain.cer && /usr/local/etc/rc.d/chronyd stop && /usr/local/etc/rc.d/chronyd start
type:script
message: copy NTS certificates into chrony directory and make them readable
description: Renew NTS certificate & restart chrony (all in one go)b) enable it:

Code: [Select]

service configd restartc) setup an automation using this action in the ACME client for the NTS certificate.
4. Point chrony to his certificate and key:

Code: [Select]

ntsservercert /var/lib/chrony/chrony.fullchain.cer
ntsserverkey /var/lib/chrony/chrony.key(not sure this will persist an update of the plugin though ...)

Now it seems to work. Hooray! 

resources:
- https://docs.opnsense.org/development/backend/configd.html
- https://unix.stackexchange.com/questions/712784/chrony-fails-to-sync-with-nts-enabled

pinging @mimugmail in case some of this could be intregrated into the plugin itself (as it purports to create a working NTS server which it _may_ not do).

Especially, it seems that the manual edits to the config file - specifically the "ntsservercert" / "ntsserverkey" part -- seem to be over-written ...

2

21.7 Legacy Series / chrony NTS issues

« on: December 02, 2021, 02:19:44 pm »

Since [some time], chrony hardly connects to any servers anymore:

Code: [Select]

MS Name/IP address         Stratum Poll Reach LastRx Last sample               
===============================================================================
^* time.cloudflare.com           3   6    37    11    -36us[ +469us] +/-   17ms
^? sth1.nts.netnod.se            0   8     0     -     +0ns[   +0ns] +/-    0ns
^? sth2.nts.netnod.se            0   8     0     -     +0ns[   +0ns] +/-    0ns
^? ptbtime1.ptb.de               0   8     0     -     +0ns[   +0ns] +/-    0ns
^? ptbtime2.ptb.de               0   8     0     -     +0ns[   +0ns] +/-    0ns
^? ptbtime3.ptb.de               0   8     0     -     +0ns[   +0ns] +/-    0ns
^- nts1.time.nl                  2   6    37    10  -2907us[-2907us] +/-   39ms
^? nts.ntp.se                    0   8     0     -     +0ns[   +0ns] +/-    0ns
^? ntp2.glypnod.com              0   8     0     -     +0ns[   +0ns] +/-    0ns
^? ntpmon.dcs1.biz               0   8     0     -     +0ns[   +0ns] +/-    0ns
^? netmon2.dcs1.biz              0   8     0     -     +0ns[   +0ns] +/-    0ns
^? sth-ts.nts.netnod.se          0   8     0     -     +0ns[   +0ns] +/-    0ns

I can DNS-resolve all and ping most of the above domains

It seems to be an issue with file access rights? System log shows:

Code: [Select]

2021-12-02T14:15:43 chronyd[5971] Selected source 162.159.200.123 (time.cloudflare.com)
2021-12-02T14:15:41 chronyd[5971] Selected source 94.198.159.11 (nts1.time.nl)
2021-12-02T14:15:36 chronyd[5971] Could not set credentials : Error while reading file.
2021-12-02T14:15:35 chronyd[5971] Could not set credentials : Error while reading file.
2021-12-02T14:15:35 chronyd[5971] Could not set credentials : Error while reading file.
2021-12-02T14:15:35 chronyd[5971] Could not set credentials : Error while reading file.
2021-12-02T14:15:35 chronyd[5971] Could not set credentials : Error while reading file.
2021-12-02T14:15:35 chronyd[5971] Could not set credentials : Error while reading file.
2021-12-02T14:15:34 chronyd[5971] Could not set credentials : Error while reading file.
2021-12-02T14:15:34 chronyd[5971] Could not set credentials : Error while reading file.
2021-12-02T14:15:34 chronyd[5971] Could not set credentials : Error while reading file.
2021-12-02T14:15:34 chronyd[5971] Could not set credentials : Error while reading file.
2021-12-02T14:15:34 chronyd[5971] Source 194.58.202.203 changed to 194.58.202.202 (nts.netnod.se)
2021-12-02T14:15:20 configctl[3020] event @ 1638450920.24 exec: system event config_changed
[ Chrony restart ]
It used to run fine; so I am suspecting the latest updates 21.7.5 or 21.7.6 -- or the recent update of my SSL certificate by the new ACME?

3

Virtual private networks / Wireguard traffic treated as flowing into WAN port?

« on: October 19, 2021, 06:15:02 pm »

Why does OPNsense treat traffic on my WireGuard interface as "in" traffic on WAN?

Code: [Select]

WAN Oct 19 18:08:48 10.10.100.1:5353 224.0.0.251:5353 udp Block private networks from WAN

Code: [Select]

__timestamp__ Oct 19 18:08:48
action [block]
anchorname
datalen 69
dir [in]
dst 224.0.0.251
dstport 5353
ecn
id 46266
interface igb0
interface_name WAN
ipflags none
ipversion 4
label Block private networks from WAN
length 89
offset 0
protoname udp
protonum 17
reason match
rid 1eb94a38e58994641aff378c21d5984f
rulenr 69
src 10.10.100.1
srcport 5353
subrulenr
tos 0x0
ttl 1
This is the mDNS repeater listening on my LAN, VLAN and WireGuard interfaces which all form part of my LocalNet interface group and are generally considered as local interfaces. My OPNsense wireguard interface/endpoint has IP 10.10.100.1.

I would have expected the mDNS repeater repeating DNS traffic emitting from the LAN/VLAN interfaces to the WireGuard interface and vice versa, however this shows as WireGuard mDNS traffic flowing "in WAN" which seems wrong.

Why does this trigger a "WAN in" rule? Seems like a bug to me ....?

4

Intrusion Detection and Prevention / Abuse.ch SSBL - more performant TLS ruleset?

« on: October 13, 2021, 10:46:35 am »

https://sslbl.abuse.ch/blacklist/ states:

In addition, SSLBL provides a more performant Suricata ruleset that uses tls_cert_fingerprint instead of tls.fingerprint. Please use either the ruleset above (sslblacklist.rules) OR sslblacklist_tls_cert.rules from below. Do not use both of them at the same time.

...

In order to use the more perfomant Suricata ruleset avilable for download below, you must run Suricata 4.1.0 or newer.

https://sslbl.abuse.ch/blacklist/sslblacklist_tls_cert.rules

Would it be possible to replace the current SSLBL ruleset with the more performant TLS ruleset? Or add it as a custom ruleset?

(the "user defined" tab only seems to permit adding custom rules, not custom rulesets)

5

21.7 Legacy Series / Unbound issues

« on: September 11, 2021, 08:56:20 pm »

While running Unbound as a local resolver, I had come across three issues:

1. Unbound fails to resolve certain domains. I have no DNSBL in Unbound. Unbound only delivers the CNAME, but no A record. When using Unbound as DoT forwarder, it resolves the hostname normally.

2. Even though I have unchecked "Flush DNS cache during reload", the statistics and cache are cleared with every Unbound reload. This should not happen.

3. Unbound failing to start, see https://github.com/opnsense/core/issues/5150 -  I don't experience this anymore, though.

Has anyone experienced similar issues or could suggest possible (configuration) errors?

6

21.1 Legacy Series / Let's Encrypt: Automation: SCP fails

« on: May 03, 2021, 11:24:43 pm »

My automation for copying the Let's Encrypt certificate to my local ESXI server fails:

1. automation task set up and "test connection" claims everything is fine: Connection and upload test succeeded.

 Naming "cert.pem" --> rui.crt
 Naming "key.pem" --> rui.key
 Naming "ca.pem" --> ca.pem
 Naming "fullchain.pem" --> castore.pem

2. Manually SCP'ing the files to ESXi works. ESXi finds and uses the copied certificate:

Code: [Select]

  # scp ./cert.pem root@esxi.XXX.de:/etc/vmware/ssl/rui.crt
  # scp ...
(for the avoidance of doubt, no password needs to be entered when doing scp, i.e. key authentication has been set up and is working)

3. Running automation from the "certificate" submenu fails to copy the certificate and yields the following errors in the logfile:

Code: [Select]

opnsense[82577] /usr/local/opnsense/scripts/OPNsense/AcmeClient/upload_sftp.php: Command execution failed, exit code 2. Last input was: {"host":"esxi.XXX.de","host-key":"","port":"22","identity-type":"rsa","user":"root","remote-path":"/etc/vmware/ssl","chgrp":"","chmod":"","chmod-key":"","cert-name":"rui.crt","key-name":"rui.key","ca-name":"ca.pem","fullchain-name":"castore.pem","certificates":"xxx","automation-id":"xxx"}
opnsense[82577] /usr/local/opnsense/scripts/OPNsense/AcmeClient/upload_sftp.php: Failed on {"source":"/tmp/sftp-upload-3UmGMx","target":"ca.pem","mode":"0440","group":false,"delete_source":true}
opnsense[82577] /usr/local/opnsense/scripts/OPNsense/AcmeClient/upload_sftp.php: Failed uploading file '/tmp/sftp-upload-3UmGMx' to 'ca.pem' ; Cause: {"file_not_found":true,"error":"Couldn't fsetstat: No such file or directory"}
opnsense[69190] AcmeClient: running automation: uploadESXI
opnsense[69190] AcmeClient: running automation: restartGUI
opnsense[69190] AcmeClient: running automations for certificate: *.xxx.de

Strangely, ca.pem is the only file that is actually copied over to ESXi when running this ...!

Is this an error in my setup, or is upload_sftp.php broken?

7

21.1 Legacy Series / Laptop loses connection to gateway

« on: April 27, 2021, 11:33:15 pm »

Hi, strange behaviour that I have come across on 21.1.5, and not sure where to begin to troubleshoot:

I have one(!) laptop that loses connection to the OPNsense gateway intermittently since a few days:

Setup:

WAN -> Cable Modem -> OPNsense -> Unifi switch - LAN clients / wifi clients

Issue:

My work laptop suddenly loses connection to the gateway: It can still ping all LAN devices, as well as the switch, but no longer access the OPNsense IP (nor WAN of course). A reboot of OPNsense fixes this - for a few minutes and then connection _for this client_ fails again.

All other devices (LAN and wifi) continue to be able to access OPNsense and WAN. So the problem seems to be either in OPNsense or the switch.

How could I narrow down the issue? Is it ...
- related to RSTP issues?
- related to routing (I have the FRR routing plugin installed)?
- related to firewall / shaping (why do the other 192.168.1.0/24 clients have access though)?
- ...?
 -