I've spent far longer than I would like to admit implementing the below arrangement. It's all working fine except one issue that I've spent days trying to resolve. I'm at a point where either I fundamentally misunderstood something or I've missed something obvious. Either way I need help!
The crux of the issue is that internal web gui's from a domain/hostname will not load (ip:port is fine and will load).
Setup is pretty straightforward:
- OPnsense VM (25.7.10) on proxmox hypervisor (9.1)
- OPnsense operates on the WAN/public face, and manages LAN.
- wireless AP behind opnsense/proxmox
- various LXC/VM's on proxmox
OPnsense configured with:
- vlans created and tagged
- Dnsmasq for DHCP4/6
- Unbound as DNS using TLS
- Caddy plugin as reverse proxy, https/cert handler and DDNS
Network topology:
--all internal network no public access. Example.com is used in place of my actual domain.
- OPnsense on 192.168.1.1/24 (opnsense.example.com)
- Proxmox on 192.168.1.10/24 (pve.example.com)
- wireless AP on 192.168.1.3/24 (dhcp handled by opnsense)
- proxmox backup server VM on 192.168.10.10/24 (pbs.example.com) (one example, there are others but presumably all have the same issue).
Long story short:
- opnsense.example.com loads fine
- pve.example.com does not load, but if you press the back button on the browser, and then forward, it will load. It just never loads on first attempt or refresh.
- pbs.example.com does not load at all.
- access through IPs/ports works for everything
- all hostnames/domains, DNS servers and search domains etc. have been set correctly including certificate renewals.
- DNS over TLS appears to work fine for everything based on leak tests
- DNS resolution, tracing, pings appear to work fine for all servers/apps based on terminal commands from any location
- Caddy handshakes and certs work fine, but logs show they are not called in for pve/pbs.example.com (they are for opnsense)
- Caddy reverse proxy works and logs for opnsense resolution but not pve/pbs
- Proxmox firewall does not block any relevant traffic
- I used the opnsense doc guides + homenetwork guy/various other vids to set up the system
Everything seems to point to opnsense firewall/port forwarding, but at this point I have no idea. Screenshots of the rules are attached for reference.