Topics

when using fq_codel and PIE on a queue, are any of these parameters in this image tunable?

just curious what max_burst 150ms is and what it impacts... is there a related property?

I'm interested in adding a few (one or more) 'watch' systems that allow me to grade long distance route metrics using something simple like we have for gateway monitoring.

Is there a way to leverage existing gateway setup / RRD graphing for this?  I don't want to add bogus gateways to do this though, but if there's an option there that permits adding some that will never be used for this purpose is it possible?

question regrading update for | firewall: use tables in the shaper to avoid breaking ipfw with too many addresses

I previously had to break rules down due to too many CIDR addresses in a single rule, will this allow me to consolidate those rules and if so, what system parameters for tables should I keep an eye on or prepare to expand to handle large CIDR sets?

is it possible to update ntopng directly from their repo in CLI?  I have no experience on how to do so, but very hesitant because I fear it will break... a lot.

current version in their repo:
ntopng-4.3.210331.txz

current version as of OPNsense 21.1.3_3-amd64:
4.2.210309 (0) - Community Edition

so, this is more of an experiment on my part i suppose... been using cellular services for a few months now, which is using 464XLAT.  I've somewhat got things working with OPNSense now, but falling short I believe.  This is my first attempt to become more involved with IPv6...

I configured WAN interface to use SLAAC, and get appear to get a valid address with /64 (prefix?)
I configured LAN interface to use "track interface" on the WAN with a 0x0 prefix ID and "allow manual adjustment"

I get what seems like a valid IPv6 address on the LAN, but no gateway?  However, WAN_SLAAC does show up in the gateways list on the dashboard.

I enabled DHCPv6 server on the LAN interface with a range of :: - ::ffff  (? not sure that is totally correct)

I set Router Advertisements on the LAN as "stateless"  (windows 10 supports SLAAC ?) with RA Interface as LAN(dynamic) & router priority to normal.

I can ping ipv6.google.com from WAN interface in opnsense:

Code Select Expand


--- ipv6.l.google.com ping6 statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 29.330/32.563/34.813/2.344 ms


However, LAN interface fails

Code Select Expand


--- ipv6.l.google.com ping6 statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss


clients on LAN network are indeed now getting IPv6 addresses... but are unable to also ping ipv6.google.com
at least unbound seems to be resolving to IPv6.. so i guess that's good right?

what am i missing?

more specifically.. i'm looking for a plugin that might provide some visualizations for the web proxy.  cache hits/misses and other statics perhaps?

If there isn't one, I've seen grafana (?) and other types of visualizers...  would anyone recommend any guides.  In that scenario, i would prefer to offload storage / visualization to another server perhaps?

I tried setting up rules to be applied for a particular gateway, but this didn't do what I thought it would... route that traffic over the interface.

Only seem to be able to setup routing based on network address.  Is there a way to setup routes based on aliases?

First off.. this is not a which is better thread and don't want it to be one.  I've been using OPNSense for quite some time now, and have really enjoyed it and contributed/donated, plus plan on continuing to do so.

I've setup the traffic shaper and overall it has done well.  I have little issues with the implementation and it performs simply and easily.

I spun up a box with x86 OpenWRT just to see and experiment, configured SQM and applied CAKE with their piece_of_cake script.. ran a series of tests, and not really finding (at this time) much difference in network performance.

Other than being able to define my properties, rules, queues etc via the GUI in OPNsense whereas OpenWRT is script oriented.  They both are a bit tedious to a degree, but OpenWRT moreso (IMO).

These are two completely different systems I'm running, so I know it's an Apples to Oranges scenario, but in general.. I'm really curious what at the core are the implementation differences?

I really like to "keep it simple sorta" :)

I'm trying to figure out which option to select from the drop down list to only restore the TrafficShaper pipes,queues, and rules values from backup.

WHich option do i choose?

I had a low sequence rule defined for a particular device that only streams over http to limit its' bandwidth usage and after enabling proxy cache the rule is not respected and all of the device traffic just gets passed to a higher sequence rule that caches all download traffic using http. (atleast that is the behaviour I noticed)

Is there a way in proxy cache to exclude an IP address/range or specify a shaping rule that gets applied?

One of the last rules I have setup for shaping is dealing with somewhat of a "catch" all which immediate follows any HTTP and HTTPS traffic that i have missed in preceding rules.

I try to keep ntopng nearby, but often i 'miss' activity this way and would really like to be able to drill into some better metrics by pipe, queue, rule....

thoughts?

I did a reboot and noticed this during:

Code Select Expand

Waiting for PIDS: 53115
.

but it sat for a very long time.. not usual.

checked PIDs:

Code Select Expand


PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
53115 ntopng 20 0 248M 200M uwait 0 4:34 0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.conf -U ntopng -G /var/run/ntopng/ntopng.pid -1 /usr/local/share/ntopng/httpdocs -2 /usr/local/share/ntopng/scripts -3 /usr/local/share/ntopng/scripts/callbacks -e{ntopng}



and noticed i had the a browser open to a flow page checking a machine...  after closing, reboot proceeded as normal....

I thought that was really strange.

https://bsd-hardware.info/?probe=563c0c9894

pretty cool.. and i appreciate the mac masking ;)

seeing several calls for ntopng doing this with constant above 10% utilization on all cores...  didn't see this behaviour in 20.7.4

/usr/local/bin/ntopng /usr/local/etc/ntopng.conf -U ntopng -G /var/run/ntopng/ntopng.pid -1 /usr/local/share/ntopng/httpdocs -2 /usr/local/share/ntopng/scripts -3 /usr/local/share/ntopng/scripts/callbacks -e{ntopng}


-----------

Code Select Expand


last pid: 68325;  load averages:  0.82,  0.61,  0.47                                    up 0+00:29:03  23:00:21
89 threads:    1 running, 88 sleeping
CPU 0: 22.5% user,  0.0% nice,  0.0% system,  1.4% interrupt, 76.1% idle
CPU 1: 16.9% user,  0.0% nice,  0.0% system,  0.0% interrupt, 83.1% idle
Mem: 234M Active, 65M Inact, 463M Wired, 312M Buf, 3006M Free
Swap: 8192M Total, 8192M Free

  PID USERNAME    PRI NICE   SIZE    RES STATE    C   TIME     CPU COMMAND
22660 ntopng       22    0   201M   160M uwait    1   0:55  12.86% /usr/local/bin/ntopng /usr/local/etc/ntopng.
22660 ntopng       24    0   201M   160M uwait    1   0:54  11.61% /usr/local/bin/ntopng /usr/local/etc/ntopng.
22660 ntopng       22    0   201M   160M uwait    0   0:52  11.60% /usr/local/bin/ntopng /usr/local/etc/ntopng.
6938 root         20    0  1044M  4316K CPU0     0   0:00   0.22% top -aPCH
22660 ntopng       20    0   201M   160M bpf      1   0:01   0.06% /usr/local/bin/ntopng /usr/local/etc/ntopng.
22031 root         20    0  1048M  8040K select   1   0:00   0.04% sshd: root@pts/0 (sshd)
5378 redis        20    0    15M  5260K kqread   1   0:01   0.04% redis-server: /usr/local/bin/redis-server 12
13740 root         20    0    27M    17M select   0   0:00   0.03% /usr/local/bin/python3 /usr/local/opnsense/s
22660 ntopng       20    0   201M   160M uwait    0   0:01   0.02% /usr/local/bin/ntopng /usr/local/etc/ntopng.
22660 ntopng       20    0   201M   160M select   0   0:00   0.02% /usr/local/bin/ntopng /usr/local/etc/ntopng.
28655 root         20    0    31M    20M select   1   1:24   0.01% /usr/local/bin/python3 /usr/local/opnsense/s
22660 ntopng       20    0   201M   160M nanslp   1   0:00   0.01% /usr/local/bin/ntopng /usr/local/etc/ntopng.
96115 root         20    0    20M    11M select   1   0:00   0.01% /usr/local/bin/python3 /usr/local/opnsense/s
29566 root         20    0    21M    11M select   1   0:00   0.01% /usr/local/bin/python3 /usr/local/opnsense/s
22660 ntopng       20    0   201M   160M nanslp   1   0:00   0.01% /usr/local/bin/ntopng /usr/local/etc/ntopng.
22660 ntopng       20    0   201M   160M nanslp   0   0:00   0.01% /usr/local/bin/ntopng /usr/local/etc/ntopng.
22660 ntopng       20    0   201M   160M nanslp   1   0:00   0.01% /usr/local/bin/ntopng /usr/local/etc/ntopng.
55710 root         20    0    11M  2584K nanslp   1   0:00   0.01% /usr/local/bin/dpinger -f -S -r 0 -i WAN_DHC
45071 dhcpd        20    0    23M    11M select   0   0:00   0.01% /usr/local/sbin/dhcpd -user dhcpd -group dhc
55710 root         20    0    11M  2584K sbwait   1   0:00   0.00% /usr/local/bin/dpinger -f -S -r 0 -i WAN_DHC
22660 ntopng       20    0   201M   160M nanslp   1   0:00   0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.
22660 ntopng       20    0   201M   160M nanslp   1   0:00   0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.
17928 root         20    0  1037M  3396K bpf      0   0:00   0.00% /usr/local/sbin/filterlog -i pflog0 -p /var/
22660 ntopng       20    0   201M   160M nanslp   1   0:00   0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.
55710 root         20    0    11M  2584K nanslp   1   0:00   0.00% /usr/local/bin/dpinger -f -S -r 0 -i WAN_DHC
22660 ntopng       20    0   201M   160M nanslp   1   0:00   0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.
22660 ntopng       22    0   201M   160M uwait    0   0:57   0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.
22660 ntopng       24    0   201M   160M uwait    1   0:49   0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.
22660 ntopng       25    0   201M   160M uwait    0   0:10   0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.
22660 ntopng       22    0   201M   160M uwait    0   0:08   0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.
22660 ntopng       29    0   201M   160M uwait    1   0:02   0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.
67753 root         52    0    43M    25M accept   0   0:02   0.00% /usr/local/bin/php-cgi
32667 root         52    0    43M    25M accept   0   0:01   0.00% /usr/local/bin/php-cgi
11248 root         20    0    40M    22M accept   0   0:01   0.00% /usr/local/bin/php-cgi
22660 ntopng       20    0   201M   160M nanslp   1   0:01   0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.
22660 ntopng       24    0   201M   160M uwait    0   0:01   0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.
22660 ntopng       20    0   201M   160M nanslp   0   0:01   0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.
99859 root         52    0    43M    25M accept   0   0:01   0.00% /usr/local/bin/php-cgi
22660 ntopng       20    0   201M   160M nanslp   1   0:01   0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.
22660 ntopng       20    0   201M   160M uwait    1   0:01   0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.
68724 root         52    0    52M    32M accept   0   0:01   0.00% /usr/local/bin/python3 /usr/local/opnsense/s
22660 ntopng       20    0   201M   160M uwait    0   0:01   0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.
94820 root         20    0    18M  7776K kqread   0   0:00   0.00% /usr/local/sbin/lighttpd -f /var/etc/lighty-
22660 ntopng       20    0   201M   160M uwait    1   0:00   0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.
54041 root         52    0    31M    19M wait     1   0:00   0.00% /usr/local/bin/python3 /usr/local/opnsense/s
43469 root         20    0  1036M  3396K select   0   0:00   0.00% /usr/local/sbin/syslogd -s -c -c -P /var/run
85198 root         52    0  1043M  3512K wait     0   0:00   0.00% /bin/sh /var/db/rrd/updaterrd.sh
22660 ntopng       20    0   201M   160M uwait    0   0:00   0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.
22660 ntopng       20    0   201M   160M uwait    1   0:00   0.00% /usr/local/bin/ntopng /usr/local/etc/ntopng.



Fetching base-20.7.8-amd64.txz: ................................ failed, no signature found
***DONE***

hasn't rebooted yet.. should i be concerned?

I was wondering if I can control certain flows using a target and interval of higher values to perhaps adjust some prioritization and improve other queue flows with lower target and interval values for games?

I have two pipes (DL/UL) with FlowQueue-Codel enabled + ECN.

Queues with high weights (with Rules for games using ports and such to identify the traffic) and queues with very low weights for CIDR ranges from known streaming services like NetFlix, Twitch, Google.. etc.

However, it seems that weighting alone is not sufficient to prioritize particular flows (like video games).

or am I completely mis-understanding how FQ-Codel works by adjusting target and interval values on the queues?

Is it possible to somehow export these configurations for another lab computer?

per title... are there any examples available on how to update shaper rules, queues, pipes using the API?  or is this feature even available...

I'm really struggling to maintain this stuff by hand.

This is frustrating beyond belief.  Trying to get lists of IP ranges into a traffic shaper rule and though they are all valid.. copy/paste from text file alway results in the "invalid IP address" message in the UI.

Is there a better way to get a list of address ranges into a shaper rule?

Title pretty much say it all, but I'm looking for something to help with the classification of traffic and Sensei looks like a near fit, but I need to be able to control flow not just block...  does Sensei integration assist with this?