Caddy http access issues

[NeoDragon]

Hi,

I've been using caddy plugin for a little while. Recently, http access started acting up and not allowing ip's included in the addresses, specifically opnvpn subnet.

VPN subnet is on 192.168.50.1/24
Local Subnet is on 192.168.0.0/16

Tried adding 192.168.0.0/16 and the 192.168.50.2 ip (the vpn user address) and it still doesn't work.

As soon as I remove access restriction, everything starts to work again.
------

As i was typing this, I figured it out looking to share the caddyfile.
Invert option might be "inverted", as in it denies access to ip address listed instead of allowing.

Inverted not checked : not client_ip 192.168.0.0/16 192.168.50.2
Inverted checked : client_ip 192.168.0.0/16 192.168.50.2

[Monviech (Cedrik)]

Invert not checked should:

"If the client ip /is not/ 192.168.0.0/24, then abort the connection."

Invert checked should:

"If the client ip /is/ 192.168.0.0/24, then abort the connection."

After that match, all IPs that fall through and are not aborted are allowed.

Also keep in mind its the actual client IP on Layer 7 in the header the client sends in the HTTP or HTTPS request. It can be a different one than expected. You have to inspect the HTTP access log.

Maybe that connection doesnt take the route through the VPN but around it and comes with an external IP.

[NeoDragon]

You were right, even tho VPN traffic is routed and i can access local ip's and all, caddy is still getting a WAN ip from http log instead of local id.

Any way to change that?

[Monviech (Cedrik)]

Well yeah that can be changed but that has nothing to do with caddy.

Read on the difference between a Split Tunnel and Full Tunnel VPN.

Another option is Split DNS while the tunnel is connected.

[Tried0748]

I currently struggle a bit with the Caddy http access list, I followed the Docs and everything works like a dream but sadly as soon as I enable the "private_ipv4" Access List I can't access my stuff.domain.net neither external nor internal.

I am using my phone which is on cellular to test external, and my pc to test internal

here is my caddyfile:

Code Select Expand

# DO NOT EDIT THIS FILE -- OPNsense auto-generated file


# caddy_user=root

# Global Options
{
    log {
        output net unixgram//var/run/caddy/log.sock {
        }
        format json {
            time_format rfc3339
        }
        level DEBUG
    }

    servers {
        protocols h1 h2 h3
    }

    dynamic_dns {
        provider ovh {
        #    redacted
        }
        domains {
            domain.net *
            domain.net haus
        }
    }

    email ###
    grace_period 10s
    import /usr/local/etc/caddy/caddy.d/*.global
}

# Reverse Proxy Configuration


# Reverse Proxy Domain: "922091ee-1d99-4c1a-a59f-7f3bf9087262"
*.domain.net {
    log {
        output file /var/log/caddy/access/922091ee-1d99-4c1a-a59f-7f3bf9087262.log {
            roll_keep_for 10d
        }
    }
    tls {
        issuer acme {
            dns ovh {
            }
        }
    }

    @4210526e-5403-4ee8-a97b-6c77cbcb371f {
        host haus.domain.net
    }
    handle @4210526e-5403-4ee8-a97b-6c77cbcb371f {
        handle {
            reverse_proxy haus.internal:8123 {
            }
        }
    }
    @e0347484-55ea-4dd0-b50c-f6b74efd16cd {
        host pve01.domain.net
    }
    handle @e0347484-55ea-4dd0-b50c-f6b74efd16cd {
        @cff47bd5-bb15-4551-89bc-9eef1b952b06_pve01domainnet {
            not client_ip 192.168.0.0/16 172.16.0.0/12 10.0.0.0/8
        }
        handle @cff47bd5-bb15-4551-89bc-9eef1b952b06_pve01domainnet {
            abort
        }

        handle {
            reverse_proxy https://pve01.internal:8006 {
                transport http {
                    tls_insecure_skip_verify
                }
            }
        }
    }
    @fb9e6f2f-4d02-4b5b-944d-8a1bbc520c27 {
        host page.domain.net
    }
    handle @fb9e6f2f-4d02-4b5b-944d-8a1bbc520c27 {
        @cff47bd5-bb15-4551-89bc-9eef1b952b06_pagedomainnet {
            not client_ip 192.168.0.0/16 172.16.0.0/12 10.0.0.0/8
        }
        handle @cff47bd5-bb15-4551-89bc-9eef1b952b06_pagedomainnet {
            abort
        }

        handle {
            reverse_proxy lxc-docker-host.internal:3000 {
            }
        }
    }
    @b1aa60b9-e9cc-493e-ab5b-db3b3cb8ad5f {
        host amp.domain.net
    }
    handle @b1aa60b9-e9cc-493e-ab5b-db3b3cb8ad5f {
        @cff47bd5-bb15-4551-89bc-9eef1b952b06_ampdomainnet {
            not client_ip 192.168.0.0/16 172.16.0.0/12 10.0.0.0/8
        }
        handle @cff47bd5-bb15-4551-89bc-9eef1b952b06_ampdomainnet {
            abort
        }

        handle {
            reverse_proxy vm-pterodactyl.internal:8080 {
                transport http {
                }
            }
        }
    }
    @271f3871-f4b0-4e56-a1aa-b2ac0e36f2d8 {
        host mc.domain.net
    }
    handle @271f3871-f4b0-4e56-a1aa-b2ac0e36f2d8 {
        handle {
            reverse_proxy vm-pterodactyl.internal {
                transport http {
                }
            }
        }
    }
}

import /usr/local/etc/caddy/caddy.d/*.conf





Runing OPNsense 25.1-amd64 with the newest caddy version

Ah and sorry for hijacking this post, I can open a new one if that would be better!
Thanks for this awesome plugin @Monviech

[Monviech (Cedrik)]

Well you didnt do anything wrong in your Caddyfile, that means something in your infrastructure is wrong.

Maybe your WAN interface has a private IP?

Activate Debug logs and check HTTP Access log in one domain. Access it without the access list in place and check the logs which client_ip and remote_ip your external and internal clients have. Troubleshoot from there.

[Tried0748]

Well, I tested it on my work laptop which has IPv6 disabled, and it works... I disabled IPv6 on my desktop PC, and it also works now :)

Now I have to figure out how to set up my network to properly support IPv6.

[Monviech (Cedrik)]

Since your IPv6 Prefix is most likely dynamic, you will not be able to use Access Lists with IPv6.

I have a static prefix and added my /56 GUA to the access list.

[Tried0748]

Hi Cedrik,

sadly, I habe a dynamic IPv6 prefix and my ISP (1&1) is not willing to change that.

I looked through the forum and found you mentioning that it might be possible to use the API via a PHP script to add the dynamic prefix to the access list.

I chatgpted my way through the API documentation, but I couldn't figure out how to update the access list. My problem is that I could not find a unique UUID for the access list. When I looked through the caddy file, I noticed that the UUID is always attached to the subdomain, for example "@cff47bd5-bb15-4551-89bc-9eef1b952b06_pagedomainnet"

My first guess is that I use the script to update the caddy config, but I think that's a stupid idea because then I would no longer be able to use the GUI if my understanding is correct.

Here is the script "I" created with the help of ChatGPT, sadly I am not a programmer, but I work in IT, so I am a little bit familiar with scripting (PowerShell).

Github Link

Code Select Expand

#!/usr/local/bin/php

<?php// === Konfiguration ===$interface        = "pppoe0";                                 // Name des Netzwerkinterfaces$prefixLength     = 64;                                      // Prefix-Länge (z. B. 56)$accessListUUID   = "cff47bd5-bb15-4551-89bc-9eef1b952b06"; // UUID des Access-List-Eintrags// API-Zugangsdaten$api_base_url     = "https://opnsense.domain.net/api/caddy/"; // Basis-URL$api_key         = "xxx";$api_secret      = "xxx";// Statische IPv4-Bereiche$ipv4_allowed = [    "192.168.0.0/16",    "172.16.0.0/12",    "10.0.0.0/8"];// === Funktionen ===function callApi($endpoint, $method = "GET", $data = null) {    global $api_base_url, $api_key, $api_secret;    $url = $api_base_url . $endpoint;    $ch = curl_init($url);    if ($ch === false) {        echo "Fehler bei curl_init(): " . curl_error($ch) . "\n";        return null;    }    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $method);    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);    curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);    $headers = [        "Content-Type: application/json",        "X-API-Key: $api_key",        "X-API-Secret: $api_secret"    ];    curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);    if ($data !== null) {        $jsonData = json_encode($data, JSON_PRETTY_PRINT);        curl_setopt($ch, CURLOPT_POSTFIELDS, $jsonData);    }    $response = curl_exec($ch);    if ($response === false) {        echo "Fehler beim Curl-Request: " . curl_error($ch) . "\n";        curl_close($ch);        return null;    }    $httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);    if ($httpCode != 200) {        echo "HTTP-Fehler: " . $httpCode . "\n";        echo "Antwort: " . $response . "\n";        curl_close($ch);        return null;    }    curl_close($ch);    $decodedResponse = json_decode($response, true);    if (json_last_error() !== JSON_ERROR_NONE) {        echo "Ungültige JSON-Antwort: " . json_last_error_msg() . "\n";        return null;    }    return $decodedResponse;}function getIPv6Address($interface) {    $cmd = "ifconfig " . escapeshellarg($interface) . " | awk '/inet6 / && !/fe80/ && /prefixlen/ {print $2; exit}'";    $ipv6 = trim(shell_exec($cmd));    if (empty($ipv6)) {        echo "Fehler: Keine IPv6-Adresse gefunden oder Befehl fehlgeschlagen.\n";        return false;    }    return $ipv6;}function calculateIPv6Prefix($ipv6, $prefixLength) {    $groups = explode(":", $ipv6);    if (count($groups) < 4) {        return false;    }    $g1 = $groups[0];    $g2 = $groups[1];    $g3 = $groups[2];    $g4 = $groups[3];    $g4 = str_pad($g4, 4, "0", STR_PAD_LEFT);    $prefixPart = substr($g4, 0, 2) . "00";    $networkPrefix = "$g1:$g2:$g3:$prefixPart::";    return $networkPrefix;}// === Hauptlogik ===$ipv6_address = getIPv6Address($interface);if ($ipv6_address === false) {    echo "Fehler: Keine IPv6-Adresse auf Interface $interface gefunden.\n";    exit(1);}echo "Gefundene IPv6-Adresse: $ipv6_address\n";$dynamicPrefix = calculateIPv6Prefix($ipv6_address, $prefixLength);if ($dynamicPrefix === false) {    echo "Fehler beim Berechnen des IPv6-Netzwerk-Prefix aus $ipv6_address\n";    exit(1);}$dynamicPrefixWithCIDR = $dynamicPrefix . "/" . $prefixLength;echo "Berechneter dynamischer IPv6-Prefix: $dynamicPrefixWithCIDR\n";$allowedNetworks = array_merge($ipv4_allowed, [$dynamicPrefixWithCIDR]);$updateData = [    "uuid" => $accessListUUID,    "client_ip" => $allowedNetworks];$jsonData = json_encode($updateData, JSON_PRETTY_PRINT);var_dump($jsonData);echo "Sende Update an die Access-Liste (UUID: $accessListUUID)...\n";$updateResponse = callApi("reverseproxy/setAccessList/" . $accessListUUID, "POST", $updateData);if ($updateResponse === null) {    echo "Fehler beim Aktualisieren der Access-Liste.\n";    exit(1);}// Überprüfe die API-Antwort auf Erfolgif (isset($updateResponse['status']) && $updateResponse['status'] !== 'success') {    echo "Fehlerhafte API-Antwort: " . $updateResponse['message'] . "\n";    var_dump($updateResponse);    exit(1);}echo "Access-Liste erfolgreich aktualisiert.\n";echo "Löse Neukonfiguration von Caddy aus...\n";$reconfigResponse = callApi("reverseproxy/service/reconfigure", "POST", []);if ($reconfigResponse === null) {    echo "Fehler beim Auslösen der Neukonfiguration.\n";    exit(1);}echo "Caddy wurde erfolgreich neu konfiguriert.\n";exit(0);?>

[Monviech (Cedrik)]

It's a better idea not to script around the issue. Rather use authentication for your services instead, its independent of IP address changes.