Messages

While looking deeper in the two networks, I've still no explanation why the one I installed via USB is working out-of-the-box, and the other one, as an inplace-upgrade performed via GUI, made it impossible for win10 workstations to connect to the ad server (nethserver 7).

Walking through all KEA options, disabling "Auto collect option data", configuring DNS and domain search properly, also adding an override in unbound for the local AD server, everything is working now. Maybe there was an interference with the ISC server. IDK.

Anyway, I changed the config also for the USB installed OPNsense.

Interesting - while trying to connect the win10 worstations, I couldn't find any hint in any log. Not in the FW logs, not in the KEA logs, not in the unbound logs, nor in the general logs.

In the end the help came from AI. Also interesting.

Question: while reading the docs it's not quite clear to me if I do have to define a DNS server to the clients, in my case unbound, if using KEA? IMVHO if using KEA and unbound this should be passed automatically to the clients if the field is left empty. Wrong?

I migrated to the NEW firewall rules.

I also migrated from ISC to KEA.

As I reported, this are two pretty much similar installations, with the same provider germany vodafone, static IP, very much in the same way configured. One is working, the other I'm not able to connect to a windows 10 workstation.

There's a difference in the upgrade process. For the working location I installed 26.1 via USB drive and restored the config. The other one was an inplace upgrade. There are a lot of differences while comparing both backups with meld (of course not the specifically interfaces, domains, IP ranges, etc.).

Don't know if this really matters.

As I'm not able to reboot the windows workstations remote, let's wait until tomorrow. I'll report.

Wait - on a second location same scenario seems to work. I'll take a closer look in the configs of both locations and will report.

Thank's for hopping on.

No. Didn't change anything on the workstations. Worked before flawless. I didn't reboot the workstations after migrating to KEA. Can't do this right now. Maybe a reboot helps. IDK. Will try this tomorrow. Maybe it's still the old lease from ISC?

BTW connecting to the workstations via windows remote doesn't work either...

Versions
OPNsense 26.1.6-amd64
FreeBSD 14.3-RELEASE-p10
OpenSSL 3.0.20
up-to-date

Did an upgrade to the latest OPNsense version. Migrated to NEW firewall rules. Migrated from ISC to KEA. Left all other settings as before.

System is up and running. As far as I can see, no errors occur.

Wireguard tunnel is up and running. I'm able to connect from remote to any service in my LAN.

BUT - I'm not able to connect from remote via RealVNC (port is via alias allowed in the LAN) viewer to a windows 10 workstation in my LAN. Which worked flawless before upgrading. I'm able to ping any machine (server, printer, ...) from remote, but not to any windows workstation.

At this point I'm lost. Anybody with any similar problem? Any hint?

Thank's for any help.

O.k. making sense. Didn't see/try this yet. Will try.

Thx so far, will report.

It's not in the way:

1. upgrade to 25.1.12
2. install the plugins
3. import the config.xml

?

I think I remember while trying to install plugins to an outdated version, that'll not work?

Thank you for hopping on.

To adjust ifaces on a new hardware that's clear to me.

But:

updated from 25.1 to 25.1.12

How to update from 25.1 to 25.1.12

request the syncing of the declared plugins, which will need to be downloaded and setup according to that config

Where/how do I download and install (from cli) the plugins.

Assuming I want to build/migrate a new HW FW based on the live OPNsense which is in use. Right now the FW looks like this:

OPNsense 25.1.12-amd64
FreeBSD 14.2-RELEASE-p4
OpenSSL 3.0.17

and has a couple of plugins installed:

os-acme-client (installed)   4.9   789KiB   3   OPNsense   ACME Client   
os-caddy (installed)   2.0.2   246KiB   3   OPNsense   Modern Reverse Proxy with Automatic HTTPS, Dynamic DNS and Layer4 Routing   
os-crowdsec (installed)   1.0.10   62.7KiB   3   OPNsense   Lightweight and collaborative security engine   
os-etpro-telemetry (installed)   1.7_5   50.3KiB   2   OPNsense   ET Pro Telemetry Edition   
os-realtek-re (installed)   1.0   409B   3   OPNsense   Realtek re(4) vendor driver   
os-sensei (installed)   2.0.5   248MiB   2   SunnyValley   Enterprise Security Extensions for OPNsense (ZENARMOR)   
os-sensei-agent (installed)   2.0.5   117MiB   2   SunnyValley   ZENARMOR Connectivity Agent for Cloud Central Management   
os-sensei-updater (installed)   1.18   4.09KiB   2   SunnyValley   OPNsense ZENARMOR Plugin Updater   
os-smart (installed)   2.3_1   22.8KiB   3   OPNsense   SMART tools   
os-sunnyvalley (installed)   1.5   2.44KiB   2   OPNsense   Vendor Repository for Zenarmor (Enterprise Security Modules - NGFW, SSE, SASE, f.k.a Sensei)   
os-wol (installed)   2.5_1   22.7KiB   3   OPNsense   Wake on LAN Service

I understand building a live usb-stick from vga iso, booting in this, is able to import the config.xml. I assume, the config will not work proper without beeing updated from 25.1 to 25.1.12, and without the configured plugins installed.

What would be the correct approach, to have a working migration (or a proper live usb-stick) in case of a breakdown?

Thank's for showing the light and pointing in the right direction.

regrads,
stefan

Workaround - but no explaination:

Adding this specific domain to DoT (1.1.1.1) works.

Would like to know why unbound does not resolve. I don't get it.

Anyone?

System is

OPNsense 25.1.12-amd64
FreeBSD 14.2-RELEASE-p4
OpenSSL 3.0.17
Unbound is running on port 53

up-to-date.

Under interfaces - diagnostics - DNS Lookup i.e. Hostname: google.com brings up:

Response
Type    Answer    Server    Query time
A   google.com. 300 IN A 142.250.184.238   127.0.0.1   29 msec
AAAA   google.com. 300 IN AAAA 2a00:1450:4001:831::200e   127.0.0.1   29 msec
MX   google.com. 300 IN MX 10 smtp.google.com.   127.0.0.1   27 msec

or t-online.de:

Response
Type    Answer    Server    Query time
A   t-online.de. 300 IN A 52.209.116.123
t-online.de. 300 IN A 34.246.241.220
t-online.de. 300 IN A 54.217.253.146   127.0.0.1   104 msec

MX   
t-online.de. 7200 IN MX 10 mx03.t-online.de.
t-online.de. 7200 IN MX 10 mx01.t-online.de.
t-online.de. 7200 IN MX 10 mx02.t-online.de.
t-online.de. 7200 IN MX 10 mx00.t-online.de.   127.0.0.1   27 msec

So far so good.


While trying to lookup for zeppelin.com I get a network error:

Query failure
Error: error sending query: Could not send or receive, because of network error

From CLI with drill, dig or traceroute same problem.

#:dig zeppelin.com MX +trace

; <<>> DiG 9.20.10 <<>> zeppelin.com MX +trace
;; global options: +cmd
.         84892   IN   NS   d.root-servers.net.
.         84892   IN   NS   f.root-servers.net.
.         84892   IN   NS   e.root-servers.net.
.         84892   IN   NS   i.root-servers.net.
.         84892   IN   NS   m.root-servers.net.
.         84892   IN   NS   b.root-servers.net.
.         84892   IN   NS   a.root-servers.net.
.         84892   IN   NS   c.root-servers.net.
.         84892   IN   NS   l.root-servers.net.
.         84892   IN   NS   k.root-servers.net.
.         84892   IN   NS   j.root-servers.net.
.         84892   IN   NS   g.root-servers.net.
.         84892   IN   NS   h.root-servers.net.
.         84892   IN   RRSIG   NS 8 0 518400 20250906170000 20250824160000 46441 . XrZ9CBBLm4nziYVEaK3h4ZM05XT6zde0Gqlt5+VrRXb+nP2QZPfp64Wg eaZy55K4eMLJ1IoHhC8QZXGoei/a7xUkGGWtwQul4hLxaTRUcfeI/mAd DlQNTSY8oi8tFM+78UKnGCqPHFDkaupe64Qi73Do0UfxZ2a7aYjj3paY fNc5+1vmo7TUwdUtb2NM7qcVXR82kLj33DT8BwJ90LSnHJqXcF8Z8wQN ydfVx6M+Wd2wV+TFuUHvxWpWmgF3qkvI6sMUeajvVudPuBFrNh8SQX2A XqUaGbxeBr/W0scm2jfugMx/Nq7w1jYO6WarEtUx17PD/ke7fpekjLeL g7ysEA==
;; Received 1097 bytes from 127.0.0.1#53(127.0.0.1) in 0 ms

com.         172800   IN   NS   h.gtld-servers.net.
com.         172800   IN   NS   k.gtld-servers.net.
com.         172800   IN   NS   f.gtld-servers.net.
com.         172800   IN   NS   b.gtld-servers.net.
com.         172800   IN   NS   d.gtld-servers.net.
com.         172800   IN   NS   g.gtld-servers.net.
com.         172800   IN   NS   c.gtld-servers.net.
com.         172800   IN   NS   e.gtld-servers.net.
com.         172800   IN   NS   a.gtld-servers.net.
com.         172800   IN   NS   i.gtld-servers.net.
com.         172800   IN   NS   j.gtld-servers.net.
com.         172800   IN   NS   m.gtld-servers.net.
com.         172800   IN   NS   l.gtld-servers.net.
com.         86400   IN   DS   19718 13 2 8ACBB0CD28F41250A80A491389424D341522D946B0DA0C0291F2D3D7 71D7805A
com.         86400   IN   RRSIG   DS 8 1 86400 20250906170000 20250824160000 46441 . j180qM7z00ikoZnkPmiABdQCoSJMeUc1cQ+FYLsnu4qIQRJjDUyHXfiO VRJFM00EpNfikriYKvdOjCtRLVWz8zZA03lMkePBxAwwdY7NW8hhWYmR iA0xJuhIYilGGtIhf//P2bbechgVwvAWsDTyrMZxme8IkJaPV0sA2W1f 963s+7WJTlFy5xL3irw5KIYcJgIOEIkBfeGDSdXqvSiNK/JizVZ4iaYX oHJ457UqPS0/V/aE0fEJg6Xu5mkU4UqMYUty6e6aHKtNYyR7ITt6/a8k 7ALIQTGWM93zJcJd4Q5K/Xap7CYmPM3V3NHbWbRQdxVl+PEAb/JFozU9 ZXUALQ==
;; Received 1200 bytes from 192.33.4.12#53(c.root-servers.net) in 23 ms

zeppelin.com.      172800   IN   NS   ns1.arcor-ip.de.
zeppelin.com.      172800   IN   NS   ns2.arcor-ip.de.
zeppelin.com.      172800   IN   NS   ns3.arcor-ip.de.
CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 900 IN NSEC3 1 1 0 - CK0Q3UDG8CEKKAE7RUKPGCT1DVSSH8LL NS SOA RRSIG DNSKEY NSEC3PARAM
CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 900 IN RRSIG NSEC3 13 2 900 20250831015158 20250824004158 20545 com. KwEWamEC8pX2daXBOa2BY/AGCUCb+3Khm5Ao6bpOsD8Aj1En1mb2hO00 CtpTsH5JQu5HQD8QFWyb6ss6/vz3Mg==
C0UGRKKSAS0GF6FFKTOPVKI97J1HPQGU.com. 900 IN NSEC3 1 1 0 - C0UH524PN2G0H9955GVG6V4VHIU6SG6Q NS DS RRSIG
C0UGRKKSAS0GF6FFKTOPVKI97J1HPQGU.com. 900 IN RRSIG NSEC3 13 2 900 20250831021716 20250824010716 20545 com. D5OzZM+00WbYpUrjSd2QRhQhypdYRzljSKs+oSUBXnmiqYqYWZ4C6UiK 232bYXEGFzIIGP0vd5qexHdyCuhA9g==
couldn't get address for 'ns1.arcor-ip.de': not found
couldn't get address for 'ns2.arcor-ip.de': not found
couldn't get address for 'ns3.arcor-ip.de': not found
dig: couldn't get address for 'ns1.arcor-ip.de': no more

and:

#drill zeppelin.com @127.0.0.1 -p53
;; ->>HEADER<<- opcode: QUERY, rcode: NXDOMAIN, id: 27301
;; flags: qr rd ra ; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0
;; QUESTION SECTION:
;; -p53.   IN   A

;; ANSWER SECTION:

;; AUTHORITY SECTION:
.   3600   IN   SOA   a.root-servers.net. nstld.verisign-grs.com. 2025082401 1800 900 604800 86400

;; ADDITIONAL SECTION:

;; Query time: 52 msec
;; SERVER: 127.0.0.1
;; WHEN: Mon Aug 25 00:43:19 2025
;; MSG SIZE  rcvd: 97


#:drill zeppelin.com @1.1.1.1
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 27740
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; zeppelin.com.   IN   A

;; ANSWER SECTION:
zeppelin.com.   60   IN   A   194.49.74.122

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 37 msec
;; SERVER: 1.1.1.1
;; WHEN: Mon Aug 25 00:44:48 2025
;; MSG SIZE  rcvd: 46



For a test I changed the setup like this:

Hostname: zeppelin.com
Server: 1.1.1.1

brings up:

Response

Type    Answer    Server    Query time
A   zeppelin.com. 60 IN A 194.49.74.122   1.1.1.1   22 msec
MX   zeppelin.com. 600 IN MX 10 mxb-00702901.gslb.pphosted.com.
zeppelin.com. 600 IN MX 10 mxa-00702901.gslb.pphosted.com.   1.1.1.1   27 msec


From another company/location in another city (same ISP - Germany vodafone business, static IP), same system setup, the domain zeppelin.com is reachable, lookup is working flawless, dig, drill etc. everything works as expected.

During the last week there was a disturbance with vodafone in this area. Internet was down/extremly slow. They told us, they're not 100% back on track they have to do some more investigations/repairing. Since Friday the speed is normal, remote work is no problem. But how can it be, that only this single domain is not reachable? Could it be possible at all?

I have no clue at all what's going on here. Any help/hint would be greatly appreciated.

Actually I tried to split port 443 in HAProxy. I couldn't find a working solution for my setup. While struggling around Cedrik gave me the hint to try Caddy. The idea behind HAProxy was to restrict access to the LAN and to present all certs to any clients or applications in the LAN. Connections from outside are only allowed through VPN. In this setup there's only a minimum of ports at the WAN interface open.

The main reason for port-sharing is that more and more wifi's in hotels or airports have only two ports open. As long as there's no deep packet inspection, one could use port 443 for openvpn. In other enviroments wireguard maybe a good choice.

Here are the steps sharing the port 443 between openvpn and a web application running on https, which are working for me. As pre-requisites there are (up-to-date)

OPNsense 24.7.9_1-amd64
FreeBSD 14.1-RELEASE-p6
OpenSSL 3.0.15

- all DNS records setup at the ISP/DNS registrar
- all (let's encrypt) certificates are stored at the correct local places and up-to-date
- there's a user created for openvpn
- local certificates have been created for the vpn-server and the vpn-client (user)
- there's a VPN instance up and running bound to 127.0.0.1 on port 1194

a) in Caddy - general settings - enable caddy and layer4 proxy. Advanced, Log, DNS, etc. are left on default.
b) in reverse proxy - http access - create your acl. I allow access only to LAN and VPN. HTTP response code for me is 403, the message is "HTTP 403 - Forbidden"
c) in reverse proxy - Domains - create your web-application on port 443 (https). Don't forget the corresponding certificate and the access list to this application.
d) in reverse proxy - http handlers - create the web-application which belongs to step c). Handler is "handle", leave path to "any", directive is "reverse_proxy", leave http version on default, protocol is "https", define your upstream domain/IP on the upstream port 443. Leave upstream path empty. Change the TLS server name that matches the SAN "Subject Alternative Name" of the offered upstream certificate.
e) in layer4 proxy - leave/change routing type "listener_wrappers", protocol is TCP, local port leave empty, matchers is "openvpn", mode and key is "any", upstream domain is "127.0.0.1", upstream port is 1194. Leave the rest empty/on default.

Connect your roadwarrior through port 443 to the openvpn instance. I used for client export "file only".

That's it. Working at least for me. If there are questions with this setup, I'll try to help. I had to start over for a second try. The first approach didn't work as expected. While re-installing (I removed every leftover from caddy via cli) it worked in the way I described. This time I was better prepared and didn't change or alter any setting while configuring caddy. Be sure to have all pre-requisites working as they should. Then start configuring caddy.

I can't push the DNS through the linux client (not working with WIN-clients), access to the LAN apps works only with IP's. Or connecting via vnc to a machine in the LAN. I can live with that. Or maybe someone is able to rule this out.

regards,
stefan

P.S. thank's to cedrik - all credits to him

Getting closer while following this: https://forum.opnsense.org/index.php?topic=18538.0

For the https_passthrough I set the Type to SSL / HTTPS (TCP mode). The tcp-request inspect delay is set to 10s. (maybe to high?)

For testing I disabled all servers, backends and frontends from the tutorial. For the moment HAProxy passes through to Openvpn on port 443. I'm able to connect my roadwarrior through HAProxy to openvpn on port 443.

HAProxy config looks like this:

global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbthread                    2
    hard-stop-after             60s
    no strict-limits
    maxconn                     50
    tune.ssl.ocsp-update.mindelay 300
    tune.ssl.ocsp-update.maxdelay 3600
    httpclient.resolvers.prefer   ipv4
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.bufsize                16384
    tune.lua.maxmem             0
    log                         /var/run/log local0 info
    lua-prepend-path            /tmp/haproxy/lua/?.lua

defaults
    log     global
    option redispatch -1
    maxconn 10
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc
    default-server maxconn 10

# autogenerated entries for ACLs


# autogenerated entries for config in backends/frontends

# autogenerated entries for stats




# Frontend (DISABLED): 0_SNI_frontend (Listening on 0.0.0.0:80, 0.0.0.0:443)

# Frontend (DISABLED): 1_HTTP_frontend (Listening on 127.0.0.1:80)

# Frontend (DISABLED): 2_HTTPS_frontend (Listening on 127.0.0.1:443)

# Frontend: https_passthrough ()
frontend https_passthrough
    bind 0.0.0.0:443 name 0.0.0.0:443
    mode tcp

    # logging options
    # ACL: traffic_ssl
    acl acl_6741bff6e05423.95774512 req_ssl_hello_type 1
    # ACL: sni_mydomain_condition
    acl acl_6740b3e1c59208.20735171 req.ssl_sni -i abc.mydomain.de

    # ACTION: request_inspect_delay
    # NOTE: actions with no ACLs/conditions will always match
    tcp-request inspect-delay 10s
    # ACTION: request_content_accept_ssl
    tcp-request content accept if acl_6741bff6e05423.95774512
    # ACTION: myservice_sni
    use_backend VPN_backend if acl_6740b3e1c59208.20735171

# Backend (DISABLED): SERVER_backend ()

# Backend (DISABLED): SSL_backend ()

# Backend (DISABLED): MTA_STS_backend ()

# Backend: VPN_backend ()
backend VPN_backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    server VPN_server my_ip

Now the next tricky part: how to merge/integrate this config into the 0_SNI_frontend of the tutorial?

Here's another solution:

https://www.reddit.com/r/selfhosted/comments/i0iq4g/guide_haproxy_with_openvpn/

"global
   log /dev/log   local0
   log /dev/log   local1 notice
   tune.ssl.default-dh-param 2048
   chroot /var/lib/haproxy
   stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
   stats timeout 30s
   user haproxy
   group haproxy
   daemon
   ca-base /etc/ssl/certs
   crt-base /etc/ssl/private
   ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
   ssl-default-bind-options no-sslv3

defaults
   log      global
   mode   http
   option   httplog
   option   dontlognull
   timeout connect 5s
   timeout client   50s
   timeout server   50s

frontend http
   bind :80
   mode http
   option http-keep-alive
   option forwardfor
   timeout client 30s
   # Redirect everything to https
   redirect scheme https code 301 if !{ ssl_fc }

frontend TLS_passthrough
   bind :443
   mode tcp
   option tcplog
   tcp-request inspect-delay 5s
   tcp-request content accept if { req_ssl_hello_type 1 } or !{ req_ssl_hello_type 1 }
   # Change this to your domain
   use_backend tcp_to_https if { req_ssl_sni -m end .yourdomain.tld }
   default_backend openvpn
      acl http req.ssl_ver gt 0

backend tcp_to_https
   mode tcp
   timeout connect 30s
   timeout server 30s
   server https 127.0.0.1:8443

frontend https
   bind :8443 ssl crt-list /etc/ssl/haproxy.certlist
   mode http
   option http-keep-alive
   option forwardfor
   timeout client 30s

   acl acl_guacamole hdr_beg(host) -i guacamole
   acl acl_plex hdr_beg(host) -i plex

   use_backend guacamole if acl_guacamole
   use_backend plex if acl_plex

backend openvpn
   mode tcp
   timeout connect 30s
   timeout server 30s
   retries 3
   server openvpn 192.168.100.218:443

backend plex
   mode http
   balance source
   stick-table type ip size 50k expire 30m
   stick on src
   timeout connect 30s
   timeout server 30s
   http-reuse never
   server plex 192.168.100.212:32400

backend guacamole
   mode http
   balance source
   stick-table type ip size 50k expire 30m
   stick on src
   timeout connect 30s
   timeout server 30s
   http-reuse never
   server guacamole 192.168.100.201:8084
"

How to translate this to OPNsense??? I tried for a few days, but it's beyond my knowledge.