WAN choking with bridged cable modem

[williamgr]

Hello!

I have been facing a very odd behavior with a cable modem on bridge mode connected to OPNsense which I have no clue how to debug.

First let me give you some background. I have had ADSL WAN link for quite some time with the modem in bridge mode and it has always worked flawlessly.
I recently acquired an additional Cable WAN link (DOCSIS 3.x) and configured it to work in bridge mode and it just doesn't seem to work right and I cant easily explain how but it goes like this: as soon as DHCP is acquired and there is no network activity I can ping IPs (e.g. 8.8.8. and resolve names just fine, but it seems as soon as network activity goes higher (e.g. many internet tabs open or I open up a torrent client) it seems to start to "choke". Suddenly simple ping wont work for some time (first X ICMP requests wont get replies) or TCP connections will take a long time to establish although previously running ping or ongoing connection will keep working.

For example: have torrent client open and then ping 8.8.8.8

Code: [Select]

root@OPNsense:~ # ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=7 ttl=43 time=62.375 ms
64 bytes from 8.8.8.8: icmp_seq=8 ttl=43 time=58.086 ms
64 bytes from 8.8.8.8: icmp_seq=9 ttl=43 time=58.837 ms
64 bytes from 8.8.8.8: icmp_seq=10 ttl=43 time=61.004 msAs you can see it lost first 6 packets and from then on a icmp request is never lost.

Additional comments
- It does seem related to OPNsense/FreeBSD -- I have tried using Linux and Windows also in bridge mode and it works just fine, I dont see the described behavior
- Configured dual WAN with my ADSL and once I failover to ADSL everything starts to work normally
- Also configured with Cable alone (no ADSL), problem persists
- Ran tcpdump on it while presenting the behavior and the only odd thing I noticed is that I get a very high TCP SYN retransmission rate
- netstat -in does not show any errors or dropped packets

I am clueless to what the problem could be. Any clues would be very much appreciated!
Please let me if there is any additional information I can provide.
Thank you!

[bartjsmit]

Which make/model of NIC do you use for the cable connection? FreeBSD has a thing about non-Intel interfaces.

Bart...

[williamgr]

igb1: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0xd000-0xd01f mem 0xc8800000-0xc881ffff,0xc8820000-0xc8823fff irq 17 at device 0.0 on pci2

I have also tried with an external USB ethernet device, same behavior.

[RNHurt]

Wow, this sounds similar to a problem I've been having since the middle of Oct. 2018  (https://forum.opnsense.org/index.php?topic=11052.0).  We have a cable modem that seems to choke several times throughout the day for no reason (that I can see).  However, we are not bridged in any way, its just a simple one WAN setup.

We have 500 devices (iPad, Chromebook, etc.) on the network and everything will be working fine.  Then suddenly packets will start to be dropped; I have several pings going all time (8.8.8.8, 192.168.0.1, etc.) and when the Google DNS ping starts dropping packets I know that we are getting ready to have problems.

It doesn't seem to be bandwidth related as our numbers never get near our 300/25 connection limit.  Most of the time we are only running 100Mb down and 10Mb up, so that doesn't seem to be the problem.  Also, the problem doesn't happen at the same time every day.

Our ISP has changed modems several times and insists that their connection is good.  They even came on-site, traced their drop, replaced some connections, and it didn't help at all.

Environment:
* Arris Touchstone DG1670A
* Dell PowerEdge R210
* OPNsense 18.7.8-amd64