Messages

Code Select Expand

# DO NOT EDIT THIS FILE -- OPNsense auto-generated file


# caddy_user=root

# Global Options
{
    log {
        output net unixgram//var/run/caddy/log.sock {
        }
        format json {
            time_format rfc3339
        }
        level DEBUG
    }

    servers {
        protocols h1 h2

        log_credentials
    }

    email chris@mydomain.net
    grace_period 10s
    skip_install_trust
    import /usr/local/etc/caddy/caddy.d/*.global
}

# Reverse Proxy Configuration


h.mydomain.net {
    @xxxxxxxxxxxxxxxx_hmydomainnet {
        not remote_ip 192.168.0.0/16 172.16.0.0/12 10.0.0.0/8 172.16.0.0/16
    }
    handle @xxxxxxxxxxxxxxxx_hmydomainnet {
        abort
    }

    handle {
        reverse_proxy 192.168.1.124:8123 {
            transport http {
            }
        }
    }
}


import /usr/local/etc/caddy/caddy.d/*.conf

The A zone record is 192.168.1.1, which is the router and caddy proxy. Attempting to go to h.mydomain.net on the LAN won't take me to 192.168.1.124:8123, instead to 192.168.1.1. Does Caddy not work on the LAN side?

That won't work for what I need (how it is setup now). I need to use an internal IP for the VPN as the VPN is only used to access internal IPs. If I use the public IP it won't go through the VPN and I would have to leave the server open to the WAN. 

What I want to do, but I am not sure it is even possible, is have home.mydomain.net point to the router/caddy at 192.168.1.1. Then proxy to the server at 192.168.1.124:8123 with a usable certificate.

I am having a bit of an issue getting Caddy to work like I want it to for certificates and reverse proxy. I have caddy setup per this guide: https://docs.opnsense.org/manual/how-tos/caddy.html

Caddy is on my router at 192.168.1.1. I have a dynamic dns address that points towards my public IP. For example, home.mydomain.net points to my public IP. In Caddy I have a domain setup for home.mydomain.net as well a a handler that proxies home.mydomain.net to 192.168.1.124:8123. This works fine for accessing the site internally and externally (access lists in caddy prevent external access though).

However, for most things I do not need any kind of external access but I still would like working certificates. So, instead of using dynamic DNS to create a zone record for home.mydomain.com to my external IP I created an A zone record for home.mydomain.net to 192.168.1.1. Same setting in caddy as above with external IP. Trying homemydomain.net EXTERNALLY gets the expected results, it just tries to go to 192.168.1.1 on whatever network you are on at that time. However, accessing home.mydomain.net INTERNALLY seems to be bypassing caddy and just attempts to go to 192.168.1.1 directly.

To clarify, first working example:  home.mydomain.net --> wan IP --> caddy (on router @ 192.168.1.1) --> reverse proxy to 192.168.1.124:8123

What I am tring to do: home.mydomain.net --> 192.168.1.1 --> caddy --> reverse proxy to 192.168.1.124:8123

I do have a reason for attempting to do it this odd way. Using a VPN on my phone only for local addresses only.

What am I doing wrong?

I'm not really trying to do that much with my switches. Just 3 vlans. The main vlan10, the guest vlan40 and an IOT vlan. I've used Cisco before and those were pretty easy for me to setup. Mikrotik is just a different creature with a lot more options. So, I'm  bit stumbling around on them.

The multiple IPs were definitely causing issues. I did have one static DHCP setup previously (when it was running as a basic switch), but something happened to it when I started setting up the vlans. Then I created the mess with multiple IPs and completely forgot about the DHCP client. It appears those IPs were the issue with the guest vlan getting internet access as well as the proxmox server attached to the second SFP+ Mikrotik switch not accessing the management sites. I only really needed the Proxmox to get to the management site for a widget in the Homepage dashboard. Also, the switches themselves couldn't get to the internet for updates, cloud backups, etc.

I finally remembered about the DHCP Client config and setup it up. I then disabled the other IP addresses, leaving only the static mapped DHCP address enabled. Everything appears to be working again. I'm sure I have more fine tuning to do, though.

After I recovered access to the switch the guest vlan(40) still wasn't working 100%. I thought what I did had fixed it, but I realized that using android and zorin linux it worked fine. But when I tried my windows 10 notebook again it did not work. No dns, no internet access. I had created another vlan for IOT devices. I hadn't used that vlan yet so I didn't configure much except the DHCP server and the allow all firewall rules. After fighting with the vlan40 guest I switched the vlan on the WAP for the guest network to the unused IOT vlan. The internet finally worked on the windows 10 notebook. After removing config items from the guest vlan to get it to match and trying after each change I narrowed the error down to the guest vlan having an IP on Mikrotik switch. Once I disabled that the guest vlan was working properly. See attached image.

The only thing I don't understand is why linux and android worked fine but win10 did not.

Thank you for your help.

I setup vlan10 for the main lan and vlan40 for the guest lan. SFP+ 1 (to opnsense) set to trunk vlan10 and vlan40 tagged only, ether4 (Ruckus WAP) to vlan10 access/PVID10 and vlan40  tagged, the remaining ports to access 10, PVID 10. I then set OPNsense to vlan10 for the lan. Turned Vlan filter on on the bridge and rebooted the switch. the guest network now has internet and dns. OPNsense looks to be running 100%. I haven't noticed and issues so the network is working 100% for now.

However, not all sunshine and rainbows. I locked my self out of the switch. I thought I could access it by MAC address, but it looks like that isn't an option. I have a back up before I applied the vlan filter on the bridge. The vlans did have IP addresses assigned to them. So, before I reset the switch and apply the backup, how do I make the management interface accessible from vlan 10. I believe I may have made an error in the attached images, highlighted areas.

From the mikrotik you need to set the "wire" going from it to OPN as a trunk. That means ALL traffic is tagged.
Then on the OPN side of it, you need to have all VLANs as tagged devices. The "parent" device does not need to be assigned.
See mine. igc1 is the "parent" and not assigned. The two VLANs hanging from it are. igc1 is the "wire" from the mikrotik switch as trunk.

I was in the middle of replying asking if I should convert the untagged traffic on opnsense to tagged traffic when you posted that.

How do you want to setup your VLAN DNS? Normally you run on OPN either dnsmasq or Unbound. In Unbound you set it to listen to all interfaces, which means will start listening on your VLAN IP. Then you simply copy the DNS allow rule from your LAN.
DHCP obviously needs also setting up on the VLAN.

I am using unbound which is set to all interfaces already and have the allow DNS rule on the VLAN 40 FW "    IPv4+6 TCP/UDP    GVLAN40 net    *    GVLAN40 address    53 (DNS)    *    *       Allow access to LAN DNS server ". DHCP is setup on vlan40 as well and the client is grabbing the correct servers.

I'll convert the LAN to a vlan, setup the mikrotik port to opnsense as an only tagged trunk and go from there.

And just to clear up, on the mikrotik:  sfp port to open sense: tagged only, vlan 10(converted from LAN) and vlan 40. Port ether4 w/ruckus, access port vlan 10, tagged vlan 40. All other ports, access port vlan 10.

Yes, 192.168.40.0/24  is the vlan subnet.

The 127.0.0.1 rules were taken from: https://homenetworkguy.com/how-to/redirect-all-dns-requests-to-local-dns-resolver/ to force the use of unbound and the ntp server on opnsense. I'll disable those for now for trouble shooting, but I do have the same rules on the LAN and they appear to work.

I'll got through and rearrange the rules, putting the two allow all rules at the bottom.

As for connecting to VLAN 40, I have a Mikrotik switch with ports tagged for Vlan 40 (opnsense port and wap port), connected to a Ruckus WAP with a wireless network assigned to VLAN 40. How is the client correctly able to obtain an IP using the vlan DHCP server, but not send traffic over the vlan? I did see the LAN interface noted in the auto block rule, but I assumed that that was correct because the vlan's parent is the LAN

I'm trying to setup a guest vlan names GVLAN40, Vlan tag of 40. Client can get a DHCP address, reach the management interface, ping the gateway and DNS server. However, I can not get to the internet and can not resolve domain names. You will have to excuse me as I am new to vlans, so I anot 100% sure if I am setting it up correctly. The client received the correct DNS and Gateway. It looks like the traffic is being blocked by an automatic rule. I setup the firewall rules as I think they need to be done. I think I have attached most of the relevant settings in the images. I ope someone can offer some insight into my stumbling around.

Thanks,
Chris

Your job to find out :)

HA can also ping this 10.10.10.37. It is in the Wireguard pool, but the WG server is 10.10.10.1

What's in "AllowedIPs" in the WG config on that Android client?

192.168.1.0/24 I also tried adding 192.168.1.124/32, but it didn't help.

I can see packets from the tunnel to the lan and from the lan to HA

The second hop on traceroute to 10.10.10.2 from HA goes to 10.10.10.37... I have no idea what that is

I just realized that traceroute on OPnsense to 10.10.10.2 times out as does ping. On the wireguard client on android I do see a "IPV4 packet with disallowed source address from peer.." while pinging from opnsense, but nothing when trying traceroute.

I can ping 10.10.10.2 from 192.168.1.9 (other machine on local net).

No firewall on the HA machine. It is not a virtual install. Previously I was using Asuswrt-Merlin with wireguard and/or openvpn and had no issue. I'll have to figure out how to do a packet trace from android.

It does. It is using DHCP with a reservation. I'll attach the network setting of HA and client config of wireguard here. 192.168.1.1 is the opnsense machine, 192.168.1.124 is HomeAssistant. I have no problem connecting to anything else over wireguard like 192.168.1.120 or 192.168.1.9.

Hello. I recently installed an opnsense server and and still configuring everything. I have wireguard server working using opnsenses' instructions. My mobile device can ping and connect to almost everything machine on the LAN with the exception of one LAN client. 192.168.1.124, which is my Home Assistant machine. It does have a static mapping. Every other machine on the LAN can ping this one machine not visible over wireguard.
Wireguard IP: 10.10.10.1/24
Lan: 192.168.1.0/24

Any help would be appreciated and let me know if anything is needed.

Thanks,
Chris