Messages

Hello,

I am using a custom appliance (Qotom) /w Intel NICs (2.5G) using an Intel Denverton chipset - Bare Metal install. Never had issues with upload or download packets getting dropped until recently. Just noticed as I started to get worse scores on the bufferbloat test over at Waveform.

Now when I do this:

ping google.com -f -l 1472

I get back 4 responses and usually 1-2 of those 4 don't make it back. The two that make it back are what they should be, about less than 20 ms.

When I play Marvel Rivals or Fortnite and use both of their network diag tools, I can see that only the upload is being affected. My download is rock solid on both. I see levels as low as 1-3%, yet up to 50% a few times.

If I use a software VPN and encapsulate all of the traffic on the PC running those games, then I get almost no packet loss on my uploads.

My first upgrade to v26 was a disaster. Worked for less than 24 hrs then woke up to a failing to boot router. Rebuilt it using the old config, upgraded to v26 and migrated to the new rules.

I'm using this appliance bridged /w x5 2.5G ports, one being WAN plus 4 SPF+ ports of which x2 I use for a 10G/SR Fiber connection back to my Mini PC running Proxmox for a homelab. The router is an Intel 8C/8T CPU, so it has plenty of resources on a 32GB Optane SSD /w 32GB of DDR4 RAM. I ordered 5x 16GB sticks of Optane SSD sticks and this appliance has the ability to RAID x2 of them, so I am planning on a RAID 1 rebuild soon.

I also recently replaced my ISP's cable modem /w a Netgear CM2000 which meets and exceeds the specs of the ISP's modem.

I appreciate any help pointing me in the right direction of what to check for.

TIA

EDIT:

I was tired posting this and tried one more thing before going to bed, I added CoDel to the upload queue (and not just the shaper). Rebooted and it seems to be working now. I played some matches of Fortnite today and didn't see any upload packets drop, whereas yesterday I saw up to 50% get tossed. Calling this a win for now. Saving config and rebuilding this tonight or tomorrow to take advantage of a RAID setup vs the one SSD in this appliance now.

I did a webUI upgrade yesterday and exported/imported my rules over. Ran it for a few hours with the Dashboard open and saw no issues, even after a few reboots.

Went to bed 3am, woke up at 9am and the router stopped routing. Using the VGA port for display, I logged in locally and tried to reboot. Got the option menu to reboot or restore, etc. Selected reboot, started to and hung.

Never rebooted, so I had to hard shutdown the router. Upon coming back up, it couldn't find a library and asked for a location and I choose default. Stopped right there. Worst upgrade of Opnsense I ever experienced in the last year of using Opnsense.

Wiped everything this morning and restored to my config backup of 25.7.11

This is the hardware in question:

https://www.qotom.net/product/MiniPC_Q20300S9_S10_Series.html

/w 32GB Intel Optane SSD and 32GB of DDR4 RAM

Sorry I don't have more complete info or logs. I had a migraine this morning and just wanted to brute force fix it asap.

I solved the problem by moving the cable over to the one other free port on the device then assigned that port in assignments and it worked. I think there's a bug in Opnsense since all 3 of those subnets were the same minus the actual subnet. I'm going to try and reset to defaults and set it all up again.

Reset to defaults and now all the ports with something plugged in are green and after a reconfig it's leasing out the correct IP ranges.

[LAN's is .10.1 /24 (living room switch)OPT1's is .20.1 /24 (TP Deco AP)OPT2's is: .30.1 /24 (side office switch)]

Hello and thanks for reading...

1st off: I'm new to opnsense, but I have worked in IT front and backend services for decades.

I'm running the most current build of Opnsense you can get right now 24.1.1

I setup 3 subnets one for each interface (LAN, OPT1, OPT2)

LAN's is        .10.1 /24 (living room switch)
OPT1's is      .20.1 /24 (TP Deco AP)
OPT2's is:     .30.1 /24 (side office switch)

I actually don't need to subnet it this much for such a small network and would have prefered everything under one, but I wasn't sure how to using Opnsense, so I went with this solution. One of those goes to my living room, one goes to TP-LINK Deco's AP and one goes to a Netgear switch in the extra room we use for a home office. That switch has another TP-LINK Deco plugged in, plus other office gear (8 ports). That switch is the one that should be getting a .30 address. It's the Nighthawk S8000 and it is somewhat manageable, but when I log into it, it's a .20 as well.

All the rules are default except for one, a floating rule that ensure all LAN traffic can pass between LAN, OPT and OPT1

I only have two subnets working. The one going into that side office (.30) won't come up, but instead, the .20 subnet is being leased out to it and the .10 is working and is contained to it's interface (which was the one I used to install and initial setup)

Under dashboard, the .30 (OPT2) shows "no carrier" and is red. Yet, I can get a .20 address from it and everything works. I can ping a .30.1 address. I plugged a laptop into the port and still only get a .20 vs .30 so it has nothing to do with that segment of my network.

I checked the DHCP4 lease ranges and they are all correct, giving out their correct subnet from .101-.200 on each.

Why is OP2 "down", yet not really and is being given the wrong subnet of IP's? Bonus points if someone can also point me to or tell me how to use one subnet for my entire network. Either way, after I iron this out, I'm planning on creating VLANs to help increase my security footprint.

Regards,

Sisko