Recent posts

#1

Hardware and Performance / Re: Suggestion for Bufferbloat...

Last post by OPNenthu - Today at 02:59:36 AM

Rule of thumb;
Limit > bellow 10Gbit/s should be around 1000 (usable since 25.7.8)
Flow > If possible set to max 65535

Seimus,

If we have 4 pipes (up/down for control plane, up/down for data) then what is the recommendation for flow value?  Still 65535 for each pipe?

#2

German - Deutsch / Re: Frage bzgl. Unmanaged Swit...

Last post by drosophila - Today at 02:46:36 AM

Vielleicht wäre ein externer PEO Injektor die bessere Wahl für Dich? Klar ist es ein Gerät zusätzlich, aber damit bist Du bei der Wahl des Switches, inklusiver zukünftiger Änderungen, wesentlich freier.

Was das schwache Powerlan angeht, könntest Du die Adapter nochmal resetten und am tatsächlichen Anbringungsort neu verpaaren. Oft werden die nebeneinander gesteckt, gepaart, und dann verteilt. So oder so kann sich an der Verbindung untereinander was ändern (und sei es nur ein unschuldig aussehendes neue Steckernetzteil), was die ganze Signalaushandlung zunichte macht. Da kann dann eine Neuaushandlung Wunder wirken (so gesehen: von 14MBits/s zu über 200MBit/s). Die Verbindungsqualität kann man mit den Powerlan-Managerprogrammen angucken, leider aber keine Neuashandlung anstoßen oder sonstiges Management betreiben.

Generell gilt: das Einzige, was ein festverlegtes LAN-Kabel übertrifft, sind zwei festverlegte LAN-Kabel. Wenn Du also diese Chance hast: nutze sie, und verlege so viele wie möglich, wenn Du einmal dabei bist.

#3

Hardware and Performance / Re: Suggestion for Bufferbloat...

Last post by Seimus - Today at 02:42:26 AM

Cookie,

Looking at your original configuration on the very 1st post, it looks to be misaligned with the docs.

Please align the configuration exactly as is in the official documentation. It was tested on several different configurations (HW + WANs) and its designed to provide a proper baseline with minimal configuration needed. Which usually results B or higher scores, if you at least set the BW properly.

The main point of having properly configured FQ_C is to set properly the BW and to have Pipes and Queues for both Download and UPload. The rest of the parameters should be used for fine tuning.

I admit I can't understand the current way to use the "limit" note of the docs, the reference to the bug.

Prior OPN 25.7.8 there was a BUG that caused a CPU hogging due to excessive logging caused when the limit queue is exceeded. So the advice was to let Limit blank. Franco did FIX this (well at least on OPN side). So now is safe and beneficial to use the Limit queue and set it to 1000 for Speeds under 10Gbit/s.

I did as well update the docs, PR was merged, when Ad will recompile the docs it will be updated
https://github.com/opnsense/docs/pull/811/files

-----------

Alright lets dissect this;

I'd think a simple fair queue with no shaper would be the best option for you. I don't know the best way to accomplish that - perhaps open the pipe beyond 520Mb/s (toward single-station LAN speed).

Your QoS/Shaping should be implemented on the interface that you want to control the bottleneck for. So closer to the source of bufferbloat. A FQ as such doesn't handle in anyway bufferbloat. FQ only shares the BW equally amongst all the flows. To control bufferbloat you need an AQM (FQ_Codel, FQ_Pie) or a SQM (CAKE).
Another point is, you should not set your Pipe to more than you have, this introduces issues. You can not give out what you don't have, in our case BW. By settings BW higher than you have you will end in bufferbloat land, and latency will go high-wire, and you are giving up the control to the ISP.

I haven't looked at the fq-codel implementation in... a while. The one I recall used a flow hash, and you could set the number of bits (up to 16, I believe).

FQ_C creates internal flow queues per 5-tuple using a HASH. There are examples where stochastic nature of hashing, multiple flows may end up being hashed into the same slot. This can be controlled by the flow parameter in FQ_C.

It looks like the ipfw implementation has that limit (65536). I'd think more can't hurt - fewer (potential) collisions. I wouldn't expect any negatives, but you never can tell.

This is a very bad idea if we speak about the "limit parameter". Limit is effectively the Queue size for the internal flows created by FQ_C. If you have a long Queue, but you are not able to process the packets in the Queue in time you create latency. FQ_C because its an AQM, measure sojourn time of each packet in the queue, and if it exceeds it either marks it or drops. But having to big of a queue is still overall bad. We want to TAIL drop packets when we can not handle them and not store them.

limit parameter (max 20480) with flow parameter (max 65535).

Settings the flow parameter higher is not a bad idea, the desired outcome is to have as less possible overlapping flows into the same queue as possible. But this parameter the higher its set takes more memory (in reality its not so much).

Rule of thumb;
Limit > bellow 10Gbit/s should be around (good starting point) 1000 (usable since 25.7.8)
Flow > If possible set to max 65535

PIE just sounds like a RED implementation - I can't see that it'd have much if any effect, as I wouldn't expect your queue depths/times to reach discard levels.

I really don't want to go into PIE to much e.g FQ_PIE, it work similar to FQ_C, but it has different use case, so I will say this:

Code Select Expand

Pie 
- Probabilistic, gradual
- Usage in ISP networks, broadband, general traffic

Codel
- Adaptive, based on packet age
- Low-latency applications, real-time traffic

Regards,
S.

#4

Web Proxy Filtering and Caching / Nginx Unbound Help

Last post by ptmuldoon - Today at 01:38:39 AM

I have Nginx Proxy Manager installed in a Proxmox Container, with ports 80 and 443 passed over.  And I can access my VM's with xxx.mydomain.com, etc.   But now I am having trouble accessing those same VM's while on my lan via the ip addresses of 192.168.x.x   

I think this has to do with setting the correct Unbound or something else, but can't seem to figure it out.

So how do get NPM to work correctly with both allowing use of your xxx.mydomain and the local ip on the lan at the same time?

#5

General Discussion / Re: Degraded printer functiona...

Last post by drosophila - Today at 01:32:48 AM

Also, some devices look for firmware updates, can order consumables in advance a.s.o., let alone collect statistics and push that to the manufacturer without anyone knowing.

There are good reasons to put "smart" devices into a separate VLAN.

Indeed, that's what I was implicitly hinting at, and rather than ordering consumables I'd rather have it send a mail to the admin instead, and use a local NTP server, but I accept that the default configuration might be catering to the person who plops down the device, pops in some cables and then expects it to work until it's decommissioned. Of course, this affords privacy and security only to those select "elites" who at least know what might be happening and therefore might try to prevent them.

#6

Tutorials and FAQs / Re: Packet Flow Diagram

Last post by drosophila - Today at 01:25:24 AM

Does anyone know where scope checking is done WRT this diagram? I'm referring to the part that'll generate a "Destination unreachable: Beyond scope of source address" message when a link-local tries to route to global space? It seems to be done before pf, because it also applies if I set the firewall to disabled (check "Disable Firewall Disable all packet filtering."), which (I think?) would bypass pf entirely (is that assumption correct?). Reason: I'd like to selectively change this behavior using pf rules, so that the exceptions exist only as long as the firewall is enabled, and only for the protocol(s) specified.

I could find precious little documentation on the entire subject of address scoping; almost all results are about the scope ID, which is basicly the %ifX suffix, not the "address scope" I'm referring to, and next to nothing about the handling and manipulation of the link- and site-local scopes themselves. There's more for Linux, but none of that can be applied to BSD.

#7

General Discussion / Re: Degraded printer functiona...

Last post by meyergru - Today at 12:44:21 AM

Depends on what the machine can do. For example, if it can scan/fax, it needs the time, which is also needed for logging. If that is the case, it could use an NTP server. Of course, it could get the NTP sever IP via DHCP, but as it turns out, there are plenty of devices which use arbitrary cloud NTP servers - like AppleTVs, e.g.

Also, some devices look for firmware updates, can order consumables in advance a.s.o., let alone collect statistics and push that to the manufacturer without anyone knowing.

There are good reasons to put "smart" devices into a separate VLAN.

#8

Hardware and Performance / Re: Suggestion for Bufferbloat...

Last post by cookiemonster - Today at 12:24:45 AM

To make it factual, my just-made 2 test results:
BUFFERBLOAT GRADE
B

LATENCY
Unloaded 26 ms
Download Active +39 ms

Upload Active +0 ms
SPEED ↓ Download 259.5 Mbps
↑ Upload 66.9 Mbps

Second:
BUFFERBLOAT GRADE
B

Your latency increased moderately under load.

LATENCY
Unloaded 21 ms
Download Active +42 ms
Upload Active +0 ms
SPEED ↓ Download 262.4 Mbps
↑ Upload 66.8 Mbps
==
So it's giving me Bs at the moment. Is this "good enough" leave-it-alone result? Tomorrow it might give me Cs though. I'll keep checking.

#9

General Discussion / Re: Degraded printer functiona...

Last post by drosophila - Today at 12:22:21 AM

Would be interesting to know if the printer would also work normally if it had no gateway set at all. IMO a printer has no business outside its local network, anyway, so doesn't need / shouldn't have a gateway.

#10

Hardware and Performance / Re: Suggestion for Bufferbloat...

Last post by cookiemonster - Today at 12:14:37 AM

Is a downstream shaper (particularly a single queue) likely to have the effect you want? I used downstream shapers in the past, but my purpose was to control offered load by adding latency, using multiple queues on a CBQ shaper. I didn't bother after my link passed 10Mb; it did help at 6-10Mb.

I'd think a simple fair queue with no shaper would be the best option for you. I don't know the best way to accomplish that - perhaps open the pipe beyond 520Mb/s (toward single-station LAN speed). I haven't looked at the fq-codel implementation in... a while. The one I recall used a flow hash, and you could set the number of bits (up to 16, I believe). It looks like the ipfw implementation has that limit (65536). I'd think more can't hurt - fewer (potential) collisions. I wouldn't expect any negatives, but you never can tell. PIE just sounds like a RED implementation - I can't see that it'd have much if any effect, as I wouldn't expect your queue depths/times to reach discard levels.

Of course, you could have upstream issues, at any point in the path.

You mean set it up as per the docs https://docs.opnsense.org/manual/how-tos/shaper_bufferbloat.html ?
But I can try see if I follow the thinking and put a pipe beyond the 520 Mbps, to see what happens. Thanks for the idea.
Going a little mad with this at the moment.

Thing is, I have a decent (for me) 520 Mbps bandwith. Normally I wouldn't bother with shaping but I seem to have the odd buffering now after this change I made. Frustratingly it is not better ie back to normal after restoring the previous settings.