Recent posts

#81

Hardware and Performance / Re: Suggestion for Bufferbloat...

Last post by cookiemonster - December 02, 2025, 06:14:28 PM

Hey. I've been using a windows laptop for testing the bufferbloat so far. Normally I use linux but had a need to stay booted on Win last few days. This one is connected via a Wi-Fi 6 (802.11ax) Wifi network using a Intel(R) Wi-Fi 6E AX210 160MHz adapter. Depending on location I can get as little as 480/721 (Mbps) agregated link speed (rec/tran) so I have a bottleneck there at times. Wired connections are only one for a PC but I can't get to it most of the time.
For OPN's CPU I'm using an AMD Ryzen 5 5600U on Proxmox with two vCPUs. Just did a ubench run on it and gives: Ubench Single CPU:   910759 (0.41s). So I think that is Ok.
I've now reset the shaper to docs defaults. This time also the upload side. I need to reboot (had limit and flows on the pipe), I'll update the post.

#82

General Discussion / Re: TUI for viewing and analys...

Last post by allddd - December 02, 2025, 06:11:25 PM

Hi allddd,

Nice work on this!  If you want we can work on including this in a future release as an optional binary package and see how it goes from there?


Cheers,
Franco

Hi Franco,

Thanks! I'd be honored, just let me know how I can help :) Would you need any changes to the Makefile/build process, maybe an install target? A man page would also be nice.

[dnsmasq leases]

#83

Development and Code Review / Re: How to update client names...

Last post by cyruz - December 02, 2025, 05:59:31 PM

Hello guys,

I didn't want to open a new post because what I want to add feels like a continuation to this very same topic.

The problem that op solved with his solution, can be tackled also in a different way, with a script in OPNsense, using Omada Open API.

I created this script (disclosing the help of AI here), to read dnsmasq leases, create a local cache and update Omada Client List through its API. The API calls are executed only if there is a change in the cache, so traffic is minimized. There is also a small logging feature to track last syncs.

Omada requirements:

• In Global View - Settings - Platform Integration create a New App with Client mode and Admin role. You will be rewarded with a Client ID and a Client Secret.
  
  
• Get the Controller ID in the browser, opening the link https://omada_ip:8043/api/info. The field we are looking for is omadacId.
  
  
• To find the Site ID, open Omada, open the Developer Tools of your browser, go to the Network section and open a page of your Omada Site. Search (CTRL+F) for the prefix /sites/ among the requests. You will see a series of request with the format <controller_id>/api/v2/sites/<site_id> (xxxxxxxxxx/api/v2/sites/xxxxxxxxxx). Get the Site ID from there.
  

Code Select Expand

#!/bin/sh
# dnsmasq_omada_sync.sh
# Sync dnsmasq hostnames -> Omada Controller.
# Skips Omada login if cache shows no changes.
# To create a cron job for this file in OPNsense:
# 1. Copy this file in /root/scripts
# 2. Create the file /usr/local/opnsense/service/conf/actions.d/actions_omada.conf
# 3. Append the following lines to the file:
#      [start]
#      command:/root/scripts/dnsmasq_omada_sync.sh
#      parameters:
#      type: script
#      message: Synchronize dnsmasq hostnames with Omada
#      description: Synchronize dnsmasq hostnames with Omada
# 4. Restart configd: service configd restart
# 5. Create the cron job in "System - Settings - Cron"
# --------------------------------
# cyruz - https://ciroprincipe.net

set -e

### ==============================
### CONFIGURATION
### ==============================

OMADA_URL="https://omada_ip:8043"
OMADAC_ID=""        # Omada Controller ID
SITE_ID=""          # Omada Site ID

CLIENT_ID=""        # Open API Client ID
CLIENT_SECRET=""    # Open API Client Secret

LEASE_FILE="/var/db/dnsmasq.leases"
LEASE_CACHE="/var/db/dnsmasq_omada.cache"

LOG_FILE="/var/log/dnsmasq_omada_sync.log"
MAX_LOG_SIZE=$((1024 * 1024))

#######################################
# REQUIREMENTS CHECK
#######################################
require_cmd() {
  for cmd in "$@"; do
    command -v "$cmd" >/dev/null 2>&1 || {
      echo "Error: required command missing: $cmd" >&2
      exit 1
    }
  done
}

require_cmd curl jq awk date sort mktemp tr sed wc stat mkdir

#######################################
# LOG ROTATION
#######################################
rotate_logs() {
  log_dir=$(dirname "$LOG_FILE")
  [ -d "$log_dir" ] || mkdir -p "$log_dir"

  if [ -f "$LOG_FILE" ]; then
    size=$(stat -f %z "$LOG_FILE" 2>/dev/null || echo 0)
    if [ "$size" -ge "$MAX_LOG_SIZE" ]; then
      [ -f "${LOG_FILE}.1" ] && rm -f "${LOG_FILE}.1"
      mv "$LOG_FILE" "${LOG_FILE}.1"
    fi
  fi
}

rotate_logs
exec >>"$LOG_FILE" 2>&1

if [ ! -r "$LEASE_FILE" ]; then
  echo "$(date -Iseconds) Error: dnsmasq lease file not readable: $LEASE_FILE"
  exit 1
fi

echo "===== $(date -Iseconds) - dnsmasq_omada_sync start ====="

#######################################
# STEP 1: Parse dnsmasq leases
#######################################
echo "[*] Reading dnsmasq leases..."

NOW_EPOCH=$(date +%s)

TMP_LEASES=$(mktemp -t dnsmasq.XXXXXX)
TMP_NEW_HOST=$(mktemp -t newhost.XXXXXX)

cleanup() {
  rm -f "$TMP_LEASES" "$TMP_NEW_HOST"
}
trap cleanup EXIT

# Keep only valid (non-expired, hostname present) leases, last entry per MAC.
awk -v now="$NOW_EPOCH" '
  now <= $1 && $4 != "" && $4 != "*" {
    line[tolower($2)] = $0
  }
  END {
    for (m in line) print line[m]
  }
' "$LEASE_FILE" > "$TMP_LEASES"

# Build NEW_HOST list: "mac hostname".
# dnsmasq format: expiry mac ip hostname clientid.
while read -r expiry mac ip host cid; do
  [ -z "$mac" ] && continue
  mac_lc=$(printf "%s\n" "$mac" | tr "A-Z" "a-z")
  printf "%s %s\n" "$mac_lc" "$host"
done < "$TMP_LEASES" > "$TMP_NEW_HOST"

#######################################
# STEP 2: Compare with cache BEFORE API
#######################################
echo "[*] Loading previous cache (if any) and comparing..."

CHANGED=0

if [ -f "$LEASE_CACHE" ]; then
  while read -r mac new_host; do
    [ -z "$mac" ] && continue
    old_host=$(awk -v m="$mac" 'tolower($1)==m {print $2; exit}' "$LEASE_CACHE" 2>/dev/null || true)
    if [ -z "$old_host" ] || [ "$old_host" != "$new_host" ]; then
      CHANGED=$((CHANGED + 1))
    fi
  done < "$TMP_NEW_HOST"
else
  # No cache yet: everything is considered "changed".
  CHANGED=$(wc -l < "$TMP_NEW_HOST" | awk '{print $1}')
fi

if [ "$CHANGED" -eq 0 ]; then
  echo "[*] No hostname updates detected — exiting without Omada API calls."
  echo "===== $(date -Iseconds) - dnsmasq_omada_sync end (no changes) ====="
  exit 0
fi

echo "[*] Detected $CHANGED hostname changes — requesting access token..."

#######################################
# STEP 3: Get access token (client_credentials)
#######################################
TOKEN=$(
  curl -sk -X POST \
    "$OMADA_URL/openapi/authorize/token?grant_type=client_credentials" \
    -H "content-type:application/json" \
    -d '{
      "omadacId": "'"$OMADAC_ID"'",
      "client_id": "'"$CLIENT_ID"'",
      "client_secret": "'"$CLIENT_SECRET"'"
    }' \
  | jq -r '.result.accessToken'
)

if [ -z "$TOKEN" ] || [ "$TOKEN" = "null" ]; then
  echo "Error: failed to obtain access token from Omada OpenAPI"
  echo "===== $(date -Iseconds) - dnsmasq_omada_sync end (token error) ====="
  exit 1
fi

#######################################
# STEP 4: Apply hostname updates
#######################################
echo "[*] Applying hostname updates via OpenAPI..."

UPDATED=0
MISSING=0
FAILED=0

while read -r mac new_host; do
  [ -z "$mac" ] && continue

  # Load old host from cache (if any).
  if [ -f "$LEASE_CACHE" ]; then
    old_host=$(awk -v m="$mac" 'tolower($1)==m {print $2; exit}' "$LEASE_CACHE" 2>/dev/null || true)
  else
    old_host=""
  fi

  # Only process entries that changed vs cache.
  if [ -n "$old_host" ] && [ "$old_host" = "$new_host" ]; then
    continue
  fi

  # Build clientMac as required by OpenAPI path: upper-case with dashes.
  mac_id=$(printf "%s\n" "$mac" | tr 'a-f' 'A-F' | tr ':' '-')

  # Sanitize hostname for JSON (escape double quotes).
  safe_host=$(printf "%s" "$new_host" | sed 's/"/\\"/g')

  echo "[*] Setting name for $mac_id -> $safe_host"

  RESP=$(curl -sk -X PATCH \
    "$OMADA_URL/openapi/v1/$OMADAC_ID/sites/$SITE_ID/clients/$mac_id/name" \
    -H "content-type:application/json" \
    -H "Authorization:AccessToken=$TOKEN" \
    -d "{\"name\":\"$safe_host\"}" )

  ERR=$(printf "%s" "$RESP" | jq -r '.errorCode' 2>/dev/null || echo "unknown")

  if [ "$ERR" = "0" ]; then
    UPDATED=$((UPDATED + 1))
  elif [ "$ERR" = "-41011" ]; then
    echo "[!] Client $mac_id does not exist in this site (errorCode -41011)."
    MISSING=$((MISSING + 1))
  else
    echo "[!] Failed to update $mac_id (errorCode $ERR)."
    FAILED=$((FAILED + 1))
  fi

done < "$TMP_NEW_HOST"

echo "[*] Updated clients: $UPDATED"
echo "[*] Missing in Omada: $MISSING"
echo "[*] Failed updates:  $FAILED"

#######################################
# STEP 5: Rewrite cache
#######################################
echo "[*] Writing updated cache..."

sort "$TMP_NEW_HOST" > "$LEASE_CACHE.tmp"
mv "$LEASE_CACHE.tmp" "$LEASE_CACHE"

echo "===== $(date -Iseconds) - dnsmasq_omada_sync end ====="

#84

German - Deutsch / Re: IT Security Experte Floria...

Last post by Bob.Dig - December 02, 2025, 05:53:32 PM

War doch offensichtlich erfolgreich. Der Zweck heiligt die Mittel.

#85

25.7, 25.10 Series / Re: 25.7.8 update, lost intern...

Last post by meyergru - December 02, 2025, 05:25:37 PM

Reminds me of https://forum.opnsense.org/index.php?topic=47099.0. In there, is a subnet calculator, too.

#86

General Discussion / Re: Where should I put the mai...

Last post by Greg_E - December 02, 2025, 05:16:10 PM

And after that, how many ports do you have on your firewall hardware?

In theory you would need the following:

LAN
WAN
Management

ports if you want to do things the easy way. You can manage the firewall from the LAN (the default configuration). Many of us just make a maintenance network if we have an extra port and use it kind of as backup if the LAN port gives us a problem. And I've had to use mine when I did something that knocked out all my other ports, makes a good use of those onboard Realtek ports that a lot of us have, works good enough to get in and fix your mistake.

#87

General Discussion / Re: Problems with NRPE

Last post by iYx3Zp8Q08hrNVZCHTYt - December 02, 2025, 05:11:16 PM

Same problem with check_procs here (other commands work) and sudo did not help. I suspect a relation to "unbound: safe command execution changes" (from the release notes of 25.7.8)?!

#88

General Discussion / Re: Where should I put the mai...

Last post by Patrick M. Hausen - December 02, 2025, 04:45:51 PM

If your home LAN is 192.168.75.0/24 then the LAN interface of OPNsense must also have an IP address in that network. Picking an address from 192.168.2.0/24 for OPNsense won't work.

#89

Hardware and Performance / Re: Suggestion for Bufferbloat...

Last post by Seimus - December 02, 2025, 04:29:27 PM

I would advice to run the test over a cable. If you don't have at least WiFi6 + all the BW available in the channel + no noise or overlap of the channel testing via WiFi is not advised. AS any of those 3 things can introduce you Wireless specific latency.

Also, rookie question but I'll ask. Do zenarmor / crowdsec interfere when running the bufferbloat tests?

Not directly and not by intent. This goes around to the CPU bottleneck, if your CPU can not keep up, you will see a latency introduced by the CPU processing of the packets. For example I have ZA on N100, and there is no problem to handle 500+ throughput on WAN with shaping enabled.

And to clarify. Can I/should I reset as per docs on my 25.1.12 version ? Suggested testing method ?

Docs are valid for any OPNsense version.
What you should focus on its the configuration + the (basic) tuning via BW parameter. Configuration for FQ_C as well BW tuning methodology is the the docs.
The advanced tuning is not needed mostly, and its really just if you want to deep dive and squeeze it.

Regards,
S.

#90

Hardware and Performance / Re: Suggestion for Bufferbloat...

Last post by cookiemonster - December 02, 2025, 03:42:31 PM

Very good information. Thank you @OPNethu  your observation of the BW is interesting.
@Seimus very thankful to you for the advice. I'll need to digest it a bit and go back to resetting all the way as per docs BUT I am on OPN 25.1.12 and worry about upgrading to latest for what other changes it might bring, unrelated to the shaper. And yes setting the BW right seems to be the hardest part. I just tested and got an A. I am closer to the AP for the test so it seems my testing methodology is something I need to be more conscious of. And the BW measured was 151 Mbps for this A result. Makes me suspect the results a little.

Also, rookie question but I'll ask. Do zenarmor / crowdsec interfere when running the bufferbloat tests?
And to clarify. Can I/should I reset as per docs on my 25.1.12 version ? Suggested testing method ?