Large HTTP Downloads Fail

[GhostyDog]

Hi,

Just installed OPNSense and finding that large downloads are failing with error output being 'network failure' in Google Chrome.

OPNsense 17.7.5-amd64
FreeBSD 11.0-RELEASE-p12
OpenSSL 1.0.2l 25 May 2017

Is this a known issue and is there a quick fix?

Thanks

GD

[phoenix]

What do you mean by "large"? From where are you downloading these large files? Have you tried any other download method (or browser) such as wget?

[GhostyDog]

What do you mean by "large"? From where are you downloading these large files? Have you tried any other download method (or browser) such as wget?

Multiple locations, I've tried with the files available here, using google chrome, it needs to work in Chrome as this is a widely use browser within our org.

https://www.thinkbroadband.com/download

50MB downloads fine every time but can't download the 100MB without it failing, also trying to download PRTG from here fails.

https://www.paessler.com/download/prtg-download

Tried it in IE11 and Edge same thing, it happens at random points in the download as well, sometimes I get 50% sometimes 60-70% but then it stalls.

From the Dashboard visualisation there seems to be a bit of a CPU spike, this might be related.

OPNSense is installed from the latest iso on a Hyper-V 2016 server. Two NICs, using the OPNSense LAN nic IP as the gateway on the client.

Cheers

GD

[phoenix]

Unfortunately I'm not an expert on debugging this type of problem but it works fine for me using wget and the same for a browser, I have a fibre connection:

Code: [Select]

[root@server downloads]# wget http://ipv6.download.thinkbroadband.com/1GB.zip
--2017-11-29 14:27:46--  http://ipv6.download.thinkbroadband.com/1GB.zip
Resolving ipv6.download.thinkbroadband.com (ipv6.download.thinkbroadband.com)... 2a02:68:1:7::1
Connecting to ipv6.download.thinkbroadband.com (ipv6.download.thinkbroadband.com)|2a02:68:1:7::1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1073741824 (1.0G) [application/zip]
Saving to: ‘1GB.zip’

100%[=====================================>] 1,073,741,824 45.0MB/s   in 18s   

2017-11-29 14:28:04 (57.7 MB/s) - ‘1GB.zip’ saved [1073741824/1073741824]
 
[root@server downloads]#
[root@server downloads]# wget http://ipv4.download.thinkbroadband.com/1GB.zip
--2017-11-29 14:28:43--  http://ipv4.download.thinkbroadband.com/1GB.zip
Resolving ipv4.download.thinkbroadband.com (ipv4.download.thinkbroadband.com)... 80.249.99.148
Connecting to ipv4.download.thinkbroadband.com (ipv4.download.thinkbroadband.com)|80.249.99.148|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1073741824 (1.0G) [application/zip]
Saving to: ‘1GB.zip.1’

100%[=====================================>] 1,073,741,824 28.2MB/s   in 38s   

2017-11-29 14:29:21 (26.8 MB/s) - ‘1GB.zip.1’ saved [1073741824/1073741824]

[root@server downloads]#
My OPNsense is installed in ESXi using the vmxnet3 driver (with real Intel i350 NICs. What type of connection do you have; how much RAM; have you disabled all the NIC offload functions; what make of NIC; was the speed any better prior to using  OPNsense? What sort of hardware are you using?

[GhostyDog]

Unfortunately I'm not an expert on debugging this type of problem but it works fine for me using wget and the same for a browser, I have a fibre connection:

Code: [Select]

[root@server downloads]# wget http://ipv6.download.thinkbroadband.com/1GB.zip
--2017-11-29 14:27:46--  http://ipv6.download.thinkbroadband.com/1GB.zip
Resolving ipv6.download.thinkbroadband.com (ipv6.download.thinkbroadband.com)... 2a02:68:1:7::1
Connecting to ipv6.download.thinkbroadband.com (ipv6.download.thinkbroadband.com)|2a02:68:1:7::1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1073741824 (1.0G) [application/zip]
Saving to: ‘1GB.zip’

100%[=====================================>] 1,073,741,824 45.0MB/s   in 18s   

2017-11-29 14:28:04 (57.7 MB/s) - ‘1GB.zip’ saved [1073741824/1073741824]
 
[root@server downloads]#
[root@server downloads]# wget http://ipv4.download.thinkbroadband.com/1GB.zip
--2017-11-29 14:28:43--  http://ipv4.download.thinkbroadband.com/1GB.zip
Resolving ipv4.download.thinkbroadband.com (ipv4.download.thinkbroadband.com)... 80.249.99.148
Connecting to ipv4.download.thinkbroadband.com (ipv4.download.thinkbroadband.com)|80.249.99.148|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1073741824 (1.0G) [application/zip]
Saving to: ‘1GB.zip.1’

100%[=====================================>] 1,073,741,824 28.2MB/s   in 38s   

2017-11-29 14:29:21 (26.8 MB/s) - ‘1GB.zip.1’ saved [1073741824/1073741824]

[root@server downloads]#
My OPNsense is installed in ESXi using the vmxnet3 driver (with real Intel i350 NICs. What type of connection do you have; how much RAM; have you disabled all the NIC offload functions; what make of NIC; was the speed any better prior to using  OPNsense? What sort of hardware are you using?

Hi, to answer your questions.

What type of connection do you have? ADSL2+

How much RAM? 2GB assigned to the guest, 1 vCPU @

Have you disabled all the NIC offload functions? Where? On the physical hardware or in Hyper-V?

What make of NIC? Intel, but they are teamed using Microsoft Multiplexing in windows Server 2016

Was the speed any better prior to using OPNsense? There is no discernible impact on speed at all, but on large downloads it looks like I might be getting a 'connection reset by peer' type error, but no browser will report it as such.

What sort of hardware are you using? HP Gen9 Proliant Server with 32 cores, and 160GB RAM

GD

[phoenix]

I guess those hardware specs are sufficient. I have a very similar configuration for my OPNsense VM and until I recently had FTTH installed I had an ADSL connection and still didn't see any problems with downloads not working correctly. For the 'offload' functions I meant in OPNsense, you'll find them at Interfaces/Settings - I'm not 100% convinced they do anything for me but I haven't tested it yet and some people recommend it for a virtualized firewall. If you haven't modified those settings it may be worth a try.

PS The offload settings for all the physical hardware NICs are still enabled on my systems.

[GhostyDog]

I guess those hardware specs are sufficient. I have a very similar configuration for my OPNsense VM and until I recently had FTTH installed I had an ADSL connection and still didn't see any problems with downloads not working correctly. For the 'offload' functions I meant in OPNsense, you'll find them at Interfaces/Settings - I'm not 100% convinced they do anything for me but I haven't tested it yet and some people recommend it for a virtualized firewall. If you haven't modified those settings it may be worth a try.

PS The offload settings for all the physical hardware NICs are still enabled on my systems.

Thanks for your help so far,

I've checked and hardware offload is disabled across the board, this is the OOTB configuration.

There's a few other things I noticed in testing, we use 3CX for IP telephony and the desktop app crashes on an incoming call resulting in the process needing to be killed and the application restarted.

Also on my personal web based email the refresh function takes about 5-10 seconds to complete, whereas without OPNSense it is instantaneous.

Cheers

GD

[Nnyan]

I actually had this issue happen to me twice while I was on pfSense.  Never was able to diagnose the issue and fresh re-installs fixed the problems both times.

[GhostyDog]

I actually had this issue happen to me twice while I was on pfSense.  Never was able to diagnose the issue and fresh re-installs fixed the problems both times.

Thanks, I'm trialling pfsense as well, and haven't had this problem with that software.

[AC]

Are you using IDS (Suricata)? Then check that log.
I had this issues lately on my pfsense. I tried to update a linux server, the small packages worked, but IDS regulary blocked the bigger packages.