Messages

I want to use my OPNsense router as a proxy for two internal DNS resolvers. Preferably, I want a HA-setup where on the OPNsense a proxy runs that tests if my two internal DNS-es are alive and routes the UDP port 53 to an alive one. That way, I can let the DHCP of the OPNsense router hand out the OPNsense router's IP address as DNS to the DHCP clients.

Reason: I run two internal DNS resolvers. Currently, the DHCP on OPNsense hands out both to clients. It turns out I have many clients that will stick to the one they select first (especially iOS/macOS devices, but it may be the same for others). Recently, I have had availability issues on both where one failed because a switch in front of it had trouble, and the other failed because it had an ethernet hardware issue. Not at the same time, but that doesn't matter, because when I client had settled on one, it would stubbornly keep trying. that one, not switching to the other one. I think that is a problem with macOS/iOS, but as this is what I have to deal with (good luck in getting Apple to fix anything), I want my setup to be robust under the scenario that one of my internal DNS resolvers is unavailable.

I accept that makes the OPNsense into a SPOF, but if the router is down, not much will work anyway.

What is the best way to do this on an OPNsense business edition?

I have two internal DNS resolvers running on two different servers (different OS too). I currently give the IP addresses of both to the clients via DHCP, so each client gets two IP addresses to use as resolver (e.g. 192.168.1.5 and 192.168.1.6). But when one of these servers dies, the clients tend to remain stuck on that server for their DNS needs, and thus a lot of stuff starts failing. In general, it seems my clients (mostly Apple) don't really react to one of the DNS resolvers being unavailable, or at least not quickly.

I would like to add a virtual IP-address to OPNsense (e.g. 192.168.1.53) that passes traffic on to either 192.168.1.5 or 192.168.1.6, specifically UDP on port 53 of course, depending on availability. Is that possible and if so, how? I am running 24.10 business edition.

I'd like to see the 'Mode' (None/Active/Backup/Disabled) of a server in the Real Servers overview. Added this to github.

OK, my issue is now solved, but I am not sure why.

• I started with a GoDaddy-authenticated cert for my router, but the cert I ask for is a wildcard (not strictly necessary, but my router doesn't have a public DNS name and I like to keep it that way).
• I was hit my GoDaddy's sudden dropping everyone <50 certs from API access. My cert was still valid (until July 12)
• I spent a day trying to get CloudFlare and NameSilo running on OPNsense/ACME.sh, but failed. I did a few resets of teh ACME.sh plugin for OPNsense
• I moved to CNAME + self-hosted acme-dns, first getting that to work on another systems (Linux+Docker+certbot)
• I got acme-dns to work with the Linux+Docker+certbot system, but then for some reason, a test cert could be had via the ACME.sh plugin, but a production cert not
• I got the certbot+acme-dns working on macOS
• I still did not get it to work (production) on OPNsense. The log shows that curl fails to talk to my acme-dns server because of the cert acme-dns is setup with, but the other machine have no problem with that (fine) cert.

This was where I was when I posted the question.

Now, I did some editing and trial runs again. Staging cert worked. Then I tried production cert mostly to get logging for my problem-hunt. And lo and behold, suddenly it worked. It was able to use acme-dns, update the TXT record, and LE could validate.

Happy enough that it works now. But not really an idea why acme.sh from OPNsense first had problems connecting to my self-hosted acme-dns (with its curl throwing up the error with value 60, whereas the certbots on my LAN had no issue with it) where now it does work. A mystery.

I got acme-dns and certbot+plugin to work. My acme-dns service is fine.

I am starting to suspect OPNsense has an outdated Intermediate cert that is no longer used by LetsEncrypt.

ACME fails to create an update in my acme-dns service, which other machines on my LAN can do it and acme-dns seems to work properly.

From an inside machine (not the router), two successive updates with a slightly different value for TXT:

Code Select Expand

gerben@hermione% curl -X POST https://acmedns-service-lan.rna.nl:943/update -H "X-Api-User: <snip>" -H "X-Api-Key: <snip>" --data '{"subdomain": "1f1c3244-8dbd-4d9e-8fa9-ee3bda01f9fe", "txt": "___validation_token_recieved_from_the_CA___"}'| python3 -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   161  100    54  100   107    646   1280 --:--:-- --:--:-- --:--:--  1939
{
    "txt": "___validation_token_recieved_from_the_CA___"
}
gerben@hermione% curl -X POST https://acmedns-service-lan.rna.nl:943/update -H "X-Api-User: <snip>" -H "X-Api-Key: <snip>" --data '{"subdomain": "1f1c3244-8dbd-4d9e-8fa9-ee3bda01f9fe", "txt": "___validation_token_recEIved_from_the_CA___"}'| python3 -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   161  100    54  100   107    710   1407 --:--:-- --:--:-- --:--:--  2146
{
    "txt": "___validation_token_recEIved_from_the_CA___"
}
The logging from acme-dns says:

Code Select Expand

time="2024-06-10T12:57:04Z" level=info msg="Handler: Actual request"
time="2024-06-10T12:57:04Z" level=info msg="  Actual request no headers added: missing origin"
time="2024-06-10T12:57:04Z" level=debug msg="TXT updated" subdomain=1f1c3244-8dbd-4d9e-8fa9-ee3bda01f9fe txt=___validation_token_recieved_from_the_CA___
time="2024-06-10T12:57:31Z" level=info msg="Handler: Actual request"
time="2024-06-10T12:57:31Z" level=info msg="  Actual request no headers added: missing origin"
time="2024-06-10T12:57:31Z" level=debug msg="TXT updated" subdomain=1f1c3244-8dbd-4d9e-8fa9-ee3bda01f9fe txt=___validation_token_recEIved_from_the_CA___

These updates work, so my acme-dns is functioning. I can check that by resolving from the outside:

Code Select Expand

$ dig @acmedns-service.rna.nl -t txt 1f1c3244-8dbd-4d9e-8fa9-ee3bda01f9fe

; <<>> DiG 9.11.36-RedHat-9.11.36-14.el8_10 <<>> @acmedns-service.rna.nl -t txt 1f1c3244-8dbd-4d9e-8fa9-ee3bda01f9fe
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12969
;; flags: qr rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;1f1c3244-8dbd-4d9e-8fa9-ee3bda01f9fe. IN TXT

;; ANSWER SECTION:
1f1c3244-8dbd-4d9e-8fa9-ee3bda01f9fe. 1 IN TXT "___validation_token_recieved_from_the_CA___"
1f1c3244-8dbd-4d9e-8fa9-ee3bda01f9fe. 1 IN TXT "___validation_token_recEIved_from_the_CA___"

;; Query time: 37 msec
;; SERVER: 213.125.118.50#53(213.125.118.50)
;; WHEN: Mon Jun 10 14:59:38 CEST 2024
;; MSG SIZE  rcvd: 249

And the logging from acme-dns says:

Code Select Expand

time="2024-06-10T12:59:38Z" level=debug msg="Answering question for domain" domain=1f1c3244-8dbd-4d9e-8fa9-ee3bda01f9fe. qtype=TXT rcode=NOERROR

So it seems my acme-dns service is working properly.

But ACME from OPNsense cannot handle it. The System log says:

Code Select Expand

2024-06-10T13:17:13 opnsense-business AcmeClient: validation for certificate failed: *.rna.nl
2024-06-10T13:17:13 opnsense-business AcmeClient: domain validation failed (dns01)
2024-06-10T13:17:13 opnsense-business /usr/local/opnsense/scripts/OPNsense/AcmeClient/lecert.php: AcmeClient: The shell command returned exit code '1': '/usr/local/sbin/acme.sh --issue --syslog 7 --debug 3 --server 'letsencrypt' --dns 'dns_acmedns' --home '/var/etc/acme-client/home' --cert-home '/var/etc/acme-client/cert-home/6666dff9dbca50.73529818' --certpath '/var/etc/acme-client/certs/6666dff9dbca50.73529818/cert.pem' --keypath '/var/etc/acme-client/keys/6666dff9dbca50.73529818/private.key' --capath '/var/etc/acme-client/certs/6666dff9dbca50.73529818/chain.pem' --fullchainpath '/var/etc/acme-client/certs/6666dff9dbca50.73529818/fullchain.pem' --domain '*.rna.nl' --days '1' --force --keylength '4096' --accountconf '/var/etc/acme-client/accounts/63c416d30df460.27753549_prod/account.conf''

and the AMCE Log says:

Code Select Expand

2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] skip dns.
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] dns_entries
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] _clearupdns
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] No need to restore nginx, skip.
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] pid
#define WITH_DEFAULT_IPV 4
#define WITH_MSGLEVEL 0 /*debug*/
#define WITH_RETRY 1
#define WITH_FILAN 1
#define WITH_SYCLS 1
#define WITH_LIBWRAP 1
#undef WITH_FIPS
#define WITH_OPENSSL 1
#define WITH_PTY 1
#undef WITH_TUN
#undef WITH_READLINE
#define WITH_EXEC 1
#define WITH_SHELL 1
#define WITH_SYSTEM 1
#define WITH_PROXY 1
#undef WITH_NAMESPACES
#undef WITH_VSOCK
#define WITH_SOCKS5 1
#define WITH_SOCKS4A 1
#define WITH_SOCKS4 1
#undef WITH_POSIXMQ
#define WITH_LISTEN 1
#define WITH_UDPLITE 1
#define WITH_DCCP 1
#define WITH_SCTP 1
#define WITH_UDP 1
#define WITH_TCP 1
#undef WITH_INTERFACE
#define WITH_GENERICSOCKET 1
#define WITH_RAWIP 1
#define WITH_IP6 1
#define WITH_IP4 1
#undef WITH_ABSTRACT_UNIXSOCKET
#define WITH_UNIX 1
#define WITH_SOCKETPAIR 1
#define WITH_PIPE 1
#define WITH_TERMIOS 1
#define WITH_GOPEN 1
#define WITH_CREAT 1
#define WITH_FILE 1
#define WITH_FDNUM 1
#define WITH_STDIO 1
#define WITH_STATS 1
#define WITH_HELP 1
features:
running on FreeBSD version FreeBSD 13.2-RELEASE-p11 stable/24.1-n255007-1d6e165fb40 SMP, release 13.2-RELEASE-p11, machine amd64
socat version 1.8.0.0 on Apr 16 2024 13:14:23
socat by Gerhard Rieger and contributors - see www.dest-unreach.org
socat:
nginx doesn't exist.
nginx:
apache doesn't exist.
apache:
OpenSSL 1.1.1t-freebsd 7 Feb 2023
openssl:openssl
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] Diagnosis versions:
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] code='200'
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] _ret='0'
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] _CURL='curl --silent --dump-header /var/etc/acme-client/home/http.header -L --trace-ascii /tmp/tmp.rto0x1MF -g '
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] _post_url='https://acme-v02.api.letsencrypt.org/acme/chall-v3/362089707667/vY4KAg'
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] POST
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] payload='{}'
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] url='https://acme-v02.api.letsencrypt.org/acme/chall-v3/362089707667/vY4KAg'
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] =======Begin Send Signed Request=======
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] Please add '--debug' or '--log' to check more details.
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] _on_issue_err
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] Error add txt for domain:_acme-challenge.rna.nl
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] invalid response of acme-dns
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] response
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] _ret='60'
0140: ......
0100: .5.....|^..V...~.......S......./s?...n......?.IR..E^..7..e...5[C
00c0: .........$...Zy..M...5l..~.M.4.....W.....M. T...V.n}..+..{..KK.R
0080: ..240711215245Z0.1.0...U....*.rna.nl0.."0...*.H.............0...
0040: ...U....US1.0...U....Let's Encrypt1.0...U....R30...240412215246Z
0000: ...........0...0..............W..s........cf0...*.H........021.0
<= Recv SSL data, 2581 bytes (0xa15)
== Info: TLSv1.3 (IN), TLS handshake, Certificate (11):
0000: .
<= Recv SSL data, 1 bytes (0x1)
0000: ....&
<= Recv SSL data, 5 bytes (0x5)
0000: .............h2
<= Recv SSL data, 15 bytes (0xf)
== Info: TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
0000: .
<= Recv SSL data, 1 bytes (0x1)
0000: ....
<= Recv SSL data, 5 bytes (0x5)
0000: .....
<= Recv SSL data, 5 bytes (0x5)
0040: .....'3......+.....3.$... ...O.U.U.w...H.`...uM..t.o..I..2
0000: ...v....|...!B....&....M.hy.....M.L!pz ...B.....0/.+T!..)...-4..
<= Recv SSL data, 122 bytes (0x7a)
== Info: TLSv1.3 (IN), TLS handshake, Server hello (2):
0000: ....z
<= Recv SSL data, 5 bytes (0x5)
01c0: ................................................................
0180: ................................................................
0140: ....*....S...36.S...............................................
0100: ......................+............-.....3.&.$... ....+&.?.X=...
00c0: ................h2.http/1.1.........1.....*.(...................
0080: <.5./.....u.........acmedns-service-lan.........................
0040: .....'3.>.......,.0.........+./...$.(.k.#.'.g.....9.....3.....=.
0000: .......). ./'[.~...M.i2.o...D...K..... ...B.....0/.+T!..)...-4..
=> Send SSL data, 512 bytes (0x200)
== Info: TLSv1.3 (OUT), TLS handshake, Client hello (1):
0000: .....
=> Send SSL data, 5 bytes (0x5)
== Info: ALPN: curl offers h2,http/1.1
== Info: Connected to acmedns-service-lan (192.168.2.125) port 943
== Info: Trying 192.168.2.125:943...
== Info: IPv4: 192.168.2.125
== Info: IPv6: (none)
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] == Info: Host acmedns-service-lan:943 was resolved.
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] Here is the curl dump log:
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] Please refer to https://curl.haxx.se/libcurl/c/libcurl-errors.html for error code: 60
2024-06-10T13:17:12 acme.sh [Mon Jun 10 13:17:12 CEST 2024] _CURL='curl --silent --dump-header /var/etc/acme-client/home/http.header -L --trace-ascii /tmp/tmp.rto0x1MF -g '
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] _post_url='https://acmedns-service-lan:943/update'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] POST
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] data='{"subdomain":"1f1c3244-8dbd-4d9e-8fa9-ee3bda01f9fe", "txt": "hxH0Ioya1YDUFDpt9U8qk9V87xSWRJVU_guMsFnbl0s"}'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] txtvalue hxH0Ioya1YDUFDpt9U8qk9V87xSWRJVU_guMsFnbl0s
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] fulldomain _acme-challenge.rna.nl
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] Using acme-dns
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] Adding txt value: hxH0Ioya1YDUFDpt9U8qk9V87xSWRJVU_guMsFnbl0s for domain: _acme-challenge.rna.nl
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] Found domain api file: /usr/local/share/examples/acme.sh/dnsapi/dns_acmedns.sh
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] d_api='/usr/local/share/examples/acme.sh/dnsapi/dns_acmedns.sh'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] txt='hxH0Ioya1YDUFDpt9U8qk9V87xSWRJVU_guMsFnbl0s'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] txtdomain='_acme-challenge.rna.nl'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] _d_alias
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] d='*.rna.nl'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] vlist='*.rna.nl#o48qno-LyORkD7Y5YnDOi1BYtIkyBQyKWBogFRIVtXQ.Myo2wog0rUg4AoAJAY_dxLhBjjDhZ3QUo-swjma-_QM#https://acme-v02.api.letsencrypt.org/acme/chall-v3/362089707667/vY4KAg#dns-01#dns_acmedns#https://acme-v02.api.letsencrypt.org/acme/authz-v3/362089707667,'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] d
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] dvlist='*.rna.nl#o48qno-LyORkD7Y5YnDOi1BYtIkyBQyKWBogFRIVtXQ.Myo2wog0rUg4AoAJAY_dxLhBjjDhZ3QUo-swjma-_QM#https://acme-v02.api.letsencrypt.org/acme/chall-v3/362089707667/vY4KAg#dns-01#dns_acmedns#https://acme-v02.api.letsencrypt.org/acme/authz-v3/362089707667'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] keyauthorization='o48qno-LyORkD7Y5YnDOi1BYtIkyBQyKWBogFRIVtXQ.Myo2wog0rUg4AoAJAY_dxLhBjjDhZ3QUo-swjma-_QM'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] uri='https://acme-v02.api.letsencrypt.org/acme/chall-v3/362089707667/vY4KAg'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] token='o48qno-LyORkD7Y5YnDOi1BYtIkyBQyKWBogFRIVtXQ'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] entry='"type":"dns-01","status":"pending","url":"https://acme-v02.api.letsencrypt.org/acme/chall-v3/362089707667/vY4KAg","token":"o48qno-LyORkD7Y5YnDOi1BYtIkyBQyKWBogFRIVtXQ"'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] _authz_url='https://acme-v02.api.letsencrypt.org/acme/authz-v3/362089707667'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] _currentRoot='dns_acmedns'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] _w='dns_acmedns'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] Getting webroot for domain='*.rna.nl'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] d='*.rna.nl'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] code='200'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] _ret='0'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] _CURL='curl --silent --dump-header /var/etc/acme-client/home/http.header -L --trace-ascii /tmp/tmp.rto0x1MF -g '
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] _post_url='https://acme-v02.api.letsencrypt.org/acme/authz-v3/362089707667'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] POST
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] payload
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] url='https://acme-v02.api.letsencrypt.org/acme/authz-v3/362089707667'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] =======Begin Send Signed Request=======
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] Le_OrderFinalize='https://acme-v02.api.letsencrypt.org/acme/finalize/1770125187/277106615897'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] Le_LinkOrder='https://acme-v02.api.letsencrypt.org/acme/order/1770125187/277106615897'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] code='201'
2024-06-10T13:17:11 acme.sh [Mon Jun 10 13:17:11 CEST 2024] _ret='0'
2024-06-10T13:17:10 acme.sh [Mon Jun 10 13:17:10 CEST 2024] _CURL='curl --silent --dump-header /var/etc/acme-client/home/http.header -L --trace-ascii /tmp/tmp.rto0x1MF -g '
2024-06-10T13:17:10 acme.sh [Mon Jun 10 13:17:10 CEST 2024] _post_url='https://acme-v02.api.letsencrypt.org/acme/new-order'
2024-06-10T13:17:10 acme.sh [Mon Jun 10 13:17:10 CEST 2024] POST
2024-06-10T13:17:10 acme.sh [Mon Jun 10 13:17:10 CEST 2024] _ret='0'
2024-06-10T13:17:10 acme.sh [Mon Jun 10 13:17:10 CEST 2024] _CURL='curl --silent --dump-header /var/etc/acme-client/home/http.header -L --trace-ascii /tmp/tmp.rto0x1MF -g -I '
2024-06-10T13:17:10 acme.sh [Mon Jun 10 13:17:10 CEST 2024] _post_url='https://acme-v02.api.letsencrypt.org/acme/new-nonce'
2024-06-10T13:17:10 acme.sh [Mon Jun 10 13:17:10 CEST 2024] HEAD
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] RSA key
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] payload='{"identifiers": [{"type":"dns","value":"*.rna.nl"}]}'
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] url='https://acme-v02.api.letsencrypt.org/acme/new-order'
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] =======Begin Send Signed Request=======
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] d
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] Getting domain auth token for each domain
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] Single domain='*.rna.nl'
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] _createcsr
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] Read key length:4096
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] _saved_account_key_hash is not changed, skip register account.
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] d
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] _currentRoot='dns_acmedns'
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] Check for domain='*.rna.nl'
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] d='*.rna.nl'
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] Le_LocalAddress
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] _chk_alt_domains
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] _chk_main_domain='*.rna.nl'
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] _on_before_issue
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] Using CA: https://acme-v02.api.letsencrypt.org/directory
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] ACME_NEW_NONCE='https://acme-v02.api.letsencrypt.org/acme/new-nonce'
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] ACME_AGREEMENT='https://letsencrypt.org/documents/LE-SA-v1.4-April-3-2024.pdf'
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] ACME_REVOKE_CERT='https://acme-v02.api.letsencrypt.org/acme/revoke-cert'
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] ACME_NEW_ACCOUNT='https://acme-v02.api.letsencrypt.org/acme/new-acct'
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] ACME_NEW_ORDER='https://acme-v02.api.letsencrypt.org/acme/new-order'
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] ACME_NEW_AUTHZ
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] ACME_KEY_CHANGE='https://acme-v02.api.letsencrypt.org/acme/key-change'
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:09 CEST 2024] ret='0'
2024-06-10T13:17:09 acme.sh [Mon Jun 10 13:17:08 CEST 2024] _CURL='curl --silent --dump-header /var/etc/acme-client/home/http.header -L --trace-ascii /tmp/tmp.J8dLzTVp -g '
2024-06-10T13:17:08 acme.sh [Mon Jun 10 13:17:08 CEST 2024] timeout=
2024-06-10T13:17:08 acme.sh [Mon Jun 10 13:17:08 CEST 2024] url='https://acme-v02.api.letsencrypt.org/directory'
2024-06-10T13:17:08 acme.sh [Mon Jun 10 13:17:08 CEST 2024] GET
2024-06-10T13:17:08 acme.sh [Mon Jun 10 13:17:08 CEST 2024] _init api for server: https://acme-v02.api.letsencrypt.org/directory
2024-06-10T13:17:08 acme.sh [Mon Jun 10 13:17:08 CEST 2024] Using ACME_DIRECTORY: https://acme-v02.api.letsencrypt.org/directory
2024-06-10T13:17:08 acme.sh [Mon Jun 10 13:17:08 CEST 2024] Le_NextRenewTime
2024-06-10T13:17:08 acme.sh [Mon Jun 10 13:17:08 CEST 2024] DOMAIN_PATH='/var/etc/acme-client/cert-home/6666dff9dbca50.73529818/*.rna.nl'
2024-06-10T13:17:08 acme.sh [Mon Jun 10 13:17:08 CEST 2024] ACME_DIRECTORY='https://acme-v02.api.letsencrypt.org/directory'
2024-06-10T13:17:08 acme.sh [Mon Jun 10 13:17:08 CEST 2024] Using config home:/var/etc/acme-client/home
2024-06-10T13:17:08 acme.sh [Mon Jun 10 13:17:08 CEST 2024] _alt_domains='no'
2024-06-10T13:17:08 acme.sh [Mon Jun 10 13:17:08 CEST 2024] _main_domain='*.rna.nl'
2024-06-10T13:17:08 acme.sh [Mon Jun 10 13:17:08 CEST 2024] Running cmd: issue
2024-06-10T13:17:08 acme.sh [Mon Jun 10 13:17:08 CEST 2024] Using server: https://acme-v02.api.letsencrypt.org/directory
The curl call from acme.sh fails, it seems, on a certificate issue (error 60). But why? It seems I am so so close, but I can't get it to work.

My OPNsense router doesn't use the inside DNS so it isn't dependent on it. This means it always goes outside for resolving. What I did for now is add a private IP A entry in my public DNS. That way, I can explicitly tell the router to go to the inside machine for the /update API. This works.

I have another issue, I'll create a separate post for that.

I do not fully understand your suggestion, sorry (not too regular work for me, this).

Suppose the router is at 192.168.2.2.
There is a mail server running on 192.168.2.15:25, which on the outside is 100.100.100.53:25 and in between is haproxy on 192.168.2.2
There is a DNS server running on 192.168.2.16:953, which on the outside is 100.100.100.54:53
There is a DNS server running on 192.168.2.16:53, which is LAN-only
There is a HTTPS server running on 192.168.2.16:443, which on the outside is 100.100.100.54:443
If I go outside to the WAN, to somewhere else, my outgoing traffic seems to come from

• 100.100.100.53 if it is going to port 25 (outgoing NAT rule so that my mail server's DNS name fits the reverse DNS when connecting other mail servers
• 100.100.100.51 for all other traffic

I want system 192.168.2.10 go to 192.168.2.16:953, but it should arrive there as if it comes from src 100.100.100.51 and it tries to connect to 100.100.100.54:53

Currently, with my setup, I can connect to the mail server from the LAN to the WAN address
```
gerben@192.168.2.10% nc -v 100.100.100.53 25
Connection to 100.100.100.53 port 25 [tcp/smtp] succeeded!
220 mail.rna.nl
```
But the postfix mail server's log says (uit knows about haproxy)
```
Jun 10 14:23:54 hermione smtp_haproxy/postscreen[95404]: CONNECT from [192.168.2.10]:61001 to [192.168.2.2]:25
```
And all other public IPs and ports fail, e.g.
```gerben@192.168.2.10 nc -v 100.100.100.54 443``` simply hangs.

From an outside machine it works:
```
$ nc -v -z 100.100.100.54 443
Ncat: Version 7.92 ( https://nmap.org/ncat )
Ncat: Connected to 100.100.100.54:443.
Ncat: 0 bytes sent, 0 bytes received in 0.04 seconds.
```

How setting up a NAT rule on the LAN helps here, I do not understand.

I need certain systems on my LAN to be able to go to a WAN interface (so take the 'outside' route). They have to connect to a server that has two DNS services, one normal for the LAN and one is ACME-DNS for letsencrypt, it is running on port 953. The system that connects only connects to port 53.

I.e. I have a NAT rule that allows (w = wan, public IP addresses, p = lan, private IP addresses)

I have a NAT rule: w1.w2.w3.w4 53 --> p1.p2.p3.p4 953 and from the outside that works.

But now I need machines on the inside to be able to do this:

LAN:p1.p2.p3.p5 -> WAN:w1.w2.w3.w4 53 --> LAN:p1.p2.p3.p4 953

In effect I cannot change their use of port 53, and I want to use NAT to make it possible. Can I?

I've added an acme-dns service on my LAN to support Letsencrypt certification. The router needs to use this too to write the secret received from LE there via the API (runs on port 943 on an internal server)

When I try to connect to the API to deliver the secret (in the challenge type), OPNsense (the router) resolves the name with the external DNS, so gets the external IP. But from outside, this API port is blocked for security reasons.

If I give OPNsense the internal IP address in the challenge type it fails too, because the service has a certificate that covers the name, but not the— internal — IP address

How do I make either of the following true:

• Make ACME service use an internal DNS to resolve the server's name to get to the API
• Make ACME ignore the wrong certificate
• Open up the port on the outside Allow only the router to use it?

Thanks.

I ran a security audit and got this result:

Code Select Expand

***GOT REQUEST TO AUDIT SECURITY***
Currently running OPNsense 23.10.3 at Sat Apr 13 11:37:48 CEST 2024
vulnxml file up-to-date
suricata-6.0.17 is vulnerable:
  suricata -- multiple vulnerabilities
  CVE: CVE-2024-23837
  CVE: CVE-2024-24568
  CVE: CVE-2024-23835
  CVE: CVE-2024-23836
  CVE: CVE-2024-23839
  WWW: https://vuxml.FreeBSD.org/freebsd/979dc373-d27d-11ee-8b84-b42e991fc52e.html

openssl111-1.1.1w is vulnerable:
  OpenSSL -- DoS in DH generation
  CVE: CVE-2023-5678
  WWW: https://vuxml.FreeBSD.org/freebsd/a5956603-7e4f-11ee-9df6-84a93843eb75.html

2 problem(s) in 2 installed package(s) found.
***DONE***
But according to https://suricata.io/2024/02/08/suricata-7-0-3-and-6-0-16-released/ the CVEs were already fixed in Suricata 6.0.16. So now I'm confused.

Immediately after upgrading to 23.10 (business edition) I clicked on check for updates again. This resulted:

Code Select Expand

***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 23.10 at Fri Jan  5 17:47:48 CET 2024
Fetching subscription information, please wait... done
Fetching changelog information, please wait... done
Updating OPNsense repository catalogue...
pkg: Repository OPNsense has a wrong packagesite, need to re-create database
Fetching meta.conf: . done
Fetching packagesite.pkg: .......... done
pkg: sqlite error while executing CREATE TABLE packages (id INTEGER PRIMARY KEY,origin TEXT,name TEXT NOT NULL,version TEXT NOT NULL,comment TEXT NOT NULL,desc TEXT NOT NULL,osversion TEXT,arch TEXT NOT NULL,maintainer TEXT NOT NULL,www TEXT,prefix TEXT NOT NULL,pkgsize INTEGER NOT NULL,flatsize INTEGER NOT NULL,licenselogic INTEGER NOT NULL,cksum TEXT NOT NULL,path TEXT NOT NULL,pkg_format_version INTEGER,manifestdigest TEXT NULL,olddigest TEXT NULL,dep_formula TEXT NULL,vital INTEGER NOT NULL DEFAULT 0);CREATE TABLE deps (origin TEXT,name TEXT,version TEXT,package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,UNIQUE(package_id, name));CREATE TABLE categories (id INTEGER PRIMARY KEY, name TEXT NOT NULL UNIQUE );CREATE TABLE pkg_categories (package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,category_id INTEGER REFERENCES categories(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, category_id));CREATE TABLE licenses (id INTEGER PRIMARY KEY,name TEXT NOT NULL UNIQUE);CREATE TABLE pkg_licenses (package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,license_id INTEGER REFERENCES licenses(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, license_id));CREATE TABLE option (option_id INTEGER PRIMARY KEY,option TEXT NOT NULL UNIQUE);CREATE TABLE option_desc (option_desc_id INTEGER PRIMARY KEY,option_desc TEXT NOT NULL UNIQUE);CREATE TABLE pkg_option (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,value TEXT NOT NULL,PRIMARY KEY(package_id, option_id));CREATE TABLE pkg_option_desc (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,option_desc_id INTEGER NOT NULL REFERENCES option_desc(option_desc_id) ON DELETE RESTRICT ON UPDATE CASCADE,PRIMARY KEY(package_id, option_id));CREATE TABLE pkg_option_default (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,default_value TEXT NOT NULL,PRIMARY KEY(package_id, option_id));CREATE TABLE shlibs (id INTEGER PRIMARY KEY,name TEXT NOT NULL UNIQUE );CREATE TABLE pkg_shlibs_required (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,shlib_id INTEGER NOT NULL REFERENCES shlibs(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, shlib_id));CREATE TABLE pkg_shlibs_provided (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,shlib_id INTEGER NOT NULL REFERENCES shlibs(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, shlib_id));CREATE TABLE annotation (annotation_id INTEGER PRIMARY KEY,annotation TEXT NOT NULL UNIQUE);CREATE TABLE pkg_annotation (package_id INTEGER REFERENCES packages(id) ON DELETE CASCADE ON UPDATE RESTRICT,tag_id INTEGER NOT NULL REFERENCES annotation(annotation_id) ON DELETE CASCADE ON UPDATE RESTRICT,value_id INTEGER NOT NULL REFERENCES annotation(annotation_id) ON DELETE CASCADE ON UPDATE RESTRICT,UNIQUE (package_id, tag_id));CREATE TABLE pkg_conflicts (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,conflict_id INTEGER NOT NULL,UNIQUE(package_id, conflict_id));CREATE TABLE provides(    id INTEGER PRIMARY KEY,    provide TEXT NOT NULL);CREATE TABLE pkg_provides (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,provide_id INTEGER NOT NULL REFERENCES provides(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, provide_id));CREATE TABLE requires(    id INTEGER PRIMARY KEY,    require TEXT NOT NULL);CREATE TABLE pkg_requires (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,require_id INTEGER NOT NULL REFERENCES requires(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, require_id));PRAGMA user_version=2014; in file pkgdb.c:2333: attempt to write a readonly database
Unable to create repository OPNsense
Unable to update repository OPNsense
Error updating repositories!
pkg: Repository OPNsense cannot be opened. 'pkg update' required
Checking integrity... done (0 conflicting)
Your packages are up to date.
***DONE***

Harmless?

I got ET Open working. I have returned to ET Telemetry and everything is now working. A summary of the things that confused me:

• ET Telemetry comes with empty rule sets for botcc, dhsield, drop, ciarmy, compromised. You need to add an additional ET Open set (specifically for ET Telemetry) in System/Firmware/Plugins. My guess is there are SID conflicts between ET Telemetry and ET Open and hence you need an 'ET Telemetry safe' ET Open set, but that is a guess.
• When using ET Telemetry there is a 'Save' button and a 'Download & Update' button. If you change the enabled set, pressing Save gives a short living warning about having to Download & Update. Normally in UX (also in most parts of OPNsense, 'Save' is the final action if you change something, but here it is not the case. Also, a warning on Save generally means in UX that the Save has not proceeded. In combination with the previous point (empty sets) this gave the impression that 'Save' did not work. The UX here is: You Save your selection and with Download & Update you 'execute' that selection (which is a real 'save' too as this is when the rules actually change). So, I tried to get to a point where 'Save' would not give the warning, but it constantly did and Suricata also warned 'there were no rules'.
  (Note: ET Open does not have that 'Save' button. Probably a good solution would be to use 'Set Token' for the token and 'Apply' for the rest (as it is elsewhere in the UI))
  
• 
  A Policy is needed to change the default 'Alert' that is in almost all downloaded rulesets to 'Block'. Two things are confusing here. In Settings, when you turn on IPS, you may expect to go from 'inform me' to 'really block'.
• Trying the aubse.ch ThreatFox ruleset doesn't work together with setting a Policy. The Policy 'Apply' button gets a 'working' state and never finishes, not even after many hours.

So, the three things mixed here: ET Telemetry comes with a bunch of empty rule sets, turning IPS on doesn't block anything until you change 'alert' to 'block' in a Policy/Policies, and 'Save' is not (as it n romally is' the definitive action to get something working.

This happened when I load abuse.ch ThreatFox. It does not happen with my current set which excludes ThreatFox

One thing I ran into that confused me is that the policy has a 'from' action (at the top) and a 'to' action (at the end). The top action is by default set to 'Drop'. So, if you change the bottom one to 'Drop' you are in effect saying: if the action is 'Drop' change it to 'Drop'.

Make sure that you create a policy to change (Action, top) 'Alert' to (New Action, bottom)'Drop'.