Slow proxy performance

[maekar]

Hi,

I have a big network in a school with ~1000 devices (iPads and computers). The gateway is a Xeon E5-2620v2 with 16gb RAM and Intel PRO/1000 PT Dual Nic. I think the hardware is ok for that network.

The OS is OPNSense 16.7.6 and we use it as a Firewall, DHCP for VLANs and DNS Resolver (to force Google Safesearch only). With that configuration, everything works fine.

The problem comes when I activate the proxy module (I tried with and without transparent mode, only need content filter function): the navigation speed turns very slow in some moments of the day (depending of the load of the network) and I have to turn it off because its impossible to work. Months ago I had the same problem with pfSense, so I think is a Squid related problem. There is any advanced tunning for a big network with lots connections in Squid to solve this performance problems?

Thanks in advance.

[fabian]

Maybe you are using a cache on a really slow HDD which can cause this issue. Try to use only the RAM for the proxy. That's the only idea I have for now. SSL-Intercept costs a lot of CPU power but I am not sure if you have it enabled.

Kind regards

Fabian

[maekar]

Maybe you are using a cache on a really slow HDD which can cause this issue. Try to use only the RAM for the proxy. That's the only idea I have for now. SSL-Intercept costs a lot of CPU power but I am not sure if you have it enabled.

Kind regards

Fabian

Thanks for the reply.

SSL is not enabled and the HDD is a WD Red. In pfSense I had Cache Size value in 0 to disable the caching function and it didn't solve the performance problem. I'll try the RAM option in OPNSense as soon I can (I thought about replacing the HDD with an SSD but the server is in production and is not easy to performing tests on it).

The strange thing when the problem appears is that the server does not seems specially overloaded. The load of the system is not high(<1.00), the CPU usage is really low, there is a lot of free RAM and the throughtput is low too. The only reason I'm sure it's a network load problem related to the proxy is because the navigation become really slow when the classes starts in the morning (and the students begin to use their iPads) and remain unstable along the day. I talk about 10 seconds to load any website, timeouts erros, many refresh attempts... When the students finish the school (and only a part of computers and iPads still working), the navigation speed come back to normal. And if I disable the proxy, there is no problem at all in any moment.

In any case, How can I disable cache function in OPNSense, since I only want it as a web content filter? And if anyone know how tuning the system for increase the Squid performance in a network with a lot of concurrent users, please tell me!

[marshalleq]

I have noted similar issues, with IPFire's implementation of Squid.  I have semi-decent hardware that should cover the tiny performance requirements of a home network.  I have gigabit internet, and enabling squid in any form slows all network transfers to a maximum of 200Mb/s.  And for any naysayers, yes squid does make a difference on a gigabit connection, particularly when I enable the Update accelerator to locally cache all the updates that I have to do for people when fixing their machines.

Another aspect I looked into (which you should also), is to ensure you have a server grade network card, which offloads processing from the CPU.  I've tried two which didn't help this problem, but should help the outcome if this problem is fixed.  There doesn't seem to be much info on running Proxy's on fast networks other than throw lot's of hardware at it which is not really a solution.

[bartjsmit]

It's worth checking if your NIC's are generating large numbers of interrupts. These may not push up CPU load since they'll cause I/O waits. Keep an eye out with top, systat, iostat and vmstat during peak times.

Bart...