Utorrent and DAP (UDP and TCP) make opnsense stop routing intermittently.

[opnsense_user_1]

Hi, I have a problem where when I start a download on DAP (download accelerator plus) then the download speed shows 360-403mb/s consistently, but when other users (10 in all) start to download with utorrent at the same time, the traffic graph on opnsense shows the speed fall to 0, then it picks up to 200kb/s before going to 0 and then after around 30s it spikes to 64mb/s and then falls again and repeats like this. Sometimes the speed will stay around 60-200kb/s while other times it will remian flat line consistently.

My setup:

Virgin media hub 3.0 cable internet in modem mode connected to wan on opnsense, lan from opnsense connected to dumb switch. 350/35 package; when it is in modem mode the problem occurs, when in router mode problem disappears, in other words, opnsense traffic becomes flat line when hub 3.0 is in modem mode, otherwise in router mode there is high traffic working normally.

OPNsense 20.1-amd64
FreeBSD 11.2-RELEASE-p16-HBSD
OpenSSL 1.1.1d 10 Sep 2019

Intel(R) Core(TM) i5-3470T CPU @ 2.90GHz (4 cores)
16gb ram
Intel pro/1000 PT Quad gigabit adapter

To fix the problem I tried:

*using a different computer to run DAP and utorrent
*reinstalling opnsense and used factory default settings with just port forwarding mentioned here: https://forum.opnsense.org/index.php?topic=8783.0
*used OPNsense-20.1-OpenSSL-vga-amd64.img without the update and once with the update
*used pfsense instead
*used a switch in between the wan and opnsense and another switch in between the lan and computer
*used the same NIC type and brand on the opnsense computer and lan PCs
*messed around with mtu values
*used different interface ports, swapping wan and lan around etc
*used 2 computers, 1 with DAP, the other with utorrent on differet interface ports in bridge mode
*used 2 computers as above, this time 1 on lan and 1 on opt1 with default allow ipv4 rule copied from lan
*set speed/duplex to 100
*used traffic shaper to spread bandwith evenly
*messed around with tunables tcp and udp datagram sizes, multiplying values by 2 seeing what happens, then try other combination like divide values by 2 etc

Problem happens in both set ups:

1. two computers 1 with DAP, the other with 5x utorrent, on switch, connected to opnsense lan port.
2. 1 computer, no switch, directly on opnsense lan port, with 5x utorrent and DAP.

None of the above helped, problem persists.


To recreate the problem (I don't think it has to be this specific, but still..):

DAP v10.0.6.0 (default settings, set internet speed to your own, unlimited no cap under bandwith tab)
utorrent v2.2.1 (default settings with port forwarded manually)
opnsense v20.1 (default factory settings with port forwarding from above forum thread)

1. in DAP paste large test file links like these: http://speedtest.tele2.net/50GB.zip  and  http://ams-speedtest-1.tele2.net/50GB.zip

2. in utorrent, add a famous or popular torrent to download with lots of sources, I used this linux torrent https://linuxtracker.org/download.php?id=0ba973670d943861fb9453eecefd3bf7d3054713&f=kali-linux-2020-2-installer-amd64-iso.torrent&key=6c2d037a      (link probably not work, since it is tracking me)

You can see the effects on the opnsense with just 5x utorrent running on the network after around five minutes but then it goes away and then returns, to make the effects permenent, it takes 10x utorrents, each with just 1 active torrent (could be the same torrent file on each instance, use shortcut with /RECOVER in each folder), they don't have to be transferring at full speed or anything.

Looks like some sort of collison with packets, between TCP and UDP, but no collison/errors/drops/missed/deferred from netstat or sysctl. It could be an issue with the way virgin modem receives data from the opnsense in modem mode, would be interesting to see if any virgin media customers experienced this.

I would appreciate any help/suggetions on this or things to try out.

Any help much appreciated, thanks.

*Attached a picture of 1PC running 5 of the 10 utorrent and DAP, showing the issues.

[opnsense_user_1]

Hi, I did further testing....

I used a single pc linked to a 10/100 switch which is linked to opnsense lan, then from the opnsense wan linked to another 10/100 switch which is linked to virgin modem, the speed was obviously slow around 92mb/s down and 36mb/s up.

But when I started the DAP downloading the large file, then started 5x utorrents, the speed still fell to 0 for a while then spiked then fell to 0 again.

[opnsense_user_1]

Hi, I had a linksys e2000 lying around (https://downloads.linksys.com/downloads/userguide/E2000_UG_USA_V10_NC-WEB.pdf) and set the virgin media router into modem mode and linked it to this e2000, then linked the same pc to it and tried the same test. The total speed is somewhat slow, but after launching DAP and 5x utorrent (just like before), it didn't have that intermittent swings of 100kb/s to 125mb/s issue.

I'm guessing that the reason why the virgin media hub 3 in router mode doesn't show the problem is the same reason why the linksys e2000 didn't show it, because they must be following some sort of standards of something related to packet transfer/processing while opnsense may not follow this so when it sends data to the virgin media router in modem mode, it causes it to have this intermittent problem.

I ran the test for a while and have a picture showing no problems.

Thanks