Pipe limit

[hardek]

Hi All

I installed OPNSense Community Edition on Proxmox VM and I am getting know this solution including each feature. I have a lot of fun, but I have encountered a problem, which I cannot solve - traffic shaper and pipe max bandwidth size limitation.
Does it possible to overwhelm the 4gb/s current limit of each pipe? For now, my internal network bandwidth between five servers is at least 40Gb/s.
My idea was:
1. Create upload pipe with bandwidth 40Gb/s
2. Create download pipe with bandwidth 40Gb/s
3. Create queue for each service group with specific weight and then these queues attach to upload and download pipes to control bandwidth in %
4. Create rules for each service and attach it for specific queue based on source/destination IP addresses

I read about combining pipes into one, but I have doubts whether something like this will work and is supported by OPNSense. I wouldn't lose performance due to this limit (for example I want to achieve 25gb/s for specific service group for all protocols). I am open to any suggestions and will be grateful for any help.

[Monviech (Cedrik)]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194453

Its a hard limit with no way around.

[hardek]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194453

Its a hard limit with no way around.

Yes, I so suspected it might be related. It is interesting, because I heard, that FreeBSD specialize in network (firewall, routers etc), so wondering how it is possible, it can handle very high throughput and control limiting it in larger scale (for example medium Internet provider)?

[Seimus]

Thats correct, however routing, packet handling and shaping are different things.

Shaper e.g dummynet is a bit different thing. Dummynet is used to create Shapers in BSD. Back in the past the Pipe size was limited to 2G on demand of users it was increased to 4G. But that's its current "technical" maximum, until the new dummynet will not be created we are stucked on this.

Regards,
S.

[hardek]

It's a pity that such a limitation exists on the system and dummynet side. This slightly spoils my idea of using OPNSense for projects larger than home lab. For high speeds this causes a significant limitation.
I am wondering whether the same limitation also exists on the Linux side.
If any alternative comes up I would be grateful if someone could share it. I myself will continue to looking for a solution or workaround too.

[Seimus]

The limitation is due to historical reason, basically the way the dummynet is coded.

Sadly there is not alternative in regards of this as BSD/OPNsense implementation for Shaper is thru dummynet.

We can only wait for new dummynet implementation.

Regards,
S.

[hardek]

I don't like to give up quickly :) I have one potential idea on how to get around this. The OPNSense instance is located on a Proxmox virtual machine and the traffic goes through two software bridge OVS (vmbr1 - external (Internet), vmbr2 - internal (internal network)), will it work to move the traffic shaping functionality to OVS before the traffic reaches OPNSense? Does it make sense?

[Seimus]

OVS supports egress Shaping or Ingress Policing. So yes thats a possibility too.

https://docs.openvswitch.org/en/latest/faq/qos/