Messages

While this isn't an answer to why this happened, is it possible for you to check the box to Enable DNS blocklists and then pick a list from the drop down menu. Apply those settings and let the list download. Then go back, de-select the blocklist and uncheck the "enable" box for the block lists?

This might 'reset' whatever odd config is causing these to run?

Can you post screenshots of your LAN and WAN interface settings from the GUI? I take it the LAN is already set to "track interface" for the IPv6 Configuration type?

For the LAN side delegation, are you using dhcpv6 or just Router Advertisements?

While I don't claim to be the expert on this, I would use the same config (3). I think matching it to the thread count makes the most sense.

That said, it doesn't hurt to try 2 and 3 and see if you notice a difference. I would guess anything under 1gbps and it probably won't be noticeable?

In my experience most modern online games don't work with out of the box settings on OPN/pfS due to the way they re-write source ports for NAT traffic. I'm not sure why this continues to be the default config. Many years ago it was a security feature but now it just breaks stuff more than it helps. As far as I'm aware only OPN/pfS do this, literally every other implementation of a router/firewall will not.

There are 3 work arounds.

1. You can install a Upnp plugin and attempt to use that to allow devices to open ports.
2. If you have many LAN devices all trying to join the same online game lobby (multiple Xbox consoles for example), you will need to set DHCP reservations for each one and manually set outbound traffic rules for each one and also leverage Upnp. Many threads available for this, see here for reference: https://forum.opnsense.org/index.php?topic=8812.0
3. You can set static port outbound for your entire /24 LAN and get most games working without additional effort.

I'm going to focus on option #3, its what I've used for over a decade now. This will get you "moderate" NAT and allow the vast majority of games and game consoles to join online and stay connected.

You'll need to manually switch the firewall to Hybrid outbound NAT and manually create a single outbound NAT rule with the "static" port option selected. This will prevent your source ports from being rewritten by the firewall during NAT traversal. It's pretty straight forward and I've attached a screenshot for reference. Make a rule like you've see in the screenshot and save/apply the settings. Then retry your games and see if this helps.

I can't tell from the post how RSS was implemented. Are you saying that you turned it on with sysctl via CLI, or did you add tunables to enable it?

The tunables should be permanent and survive reboots. The sysctl options usually will not survive a reboot if just enabling a feature to test. Screenshots attached as an example of my config.

If you configure it per the screenshot, when you run a 'netstat -Q' you'll see the following:

Code Select Expand

netstat -Q
Configuration:
Setting                        Current        Limit
Thread count                         4            4
Default queue limit                256        10240
Dispatch policy                 direct          n/a
Threads bound to CPUs          enabled          n/a

Protocols:
Name   Proto QLimit Policy Dispatch Flags
ip         1   1000    cpu   hybrid   C--
igmp       2    256 source  default   ---
rtsock     3    256 source  default   ---
arp        4    256 source  default   ---
ether      5    256    cpu   direct   C--
ip6        6   1000    cpu   hybrid   C--
ip_direct     9    256    cpu   hybrid   C--
ip6_direct    10    256    cpu   hybrid   C--


It would also appear this user is seeing a similar issue on 24.7.7: https://forum.opnsense.org/index.php?topic=43661.0

I ended up finally reverting to this version to get my graphs back

Code Select Expand

opnsense-revert -r 24.7.6 opnsense

All my old graphs were there, but everything after the day of the 24.7.7 upgrade has a gap as no graph data was written. Still, I didn't have to reset RRD data or lose any info.

I'll stay on 24.7.6 until I have a better understanding of what is happening to graph data. I'm also a graph nerd and have years of trend data that I don't want to lose. ;)

Interestingly, ALL of my RRD graphs have stopped updating since I upgraded from 24.7.6 to 24.7.7.

If the normal optimization isn't interfering with anything else on the network, I would suggest leaving that in place and see how things go? I'm still not sure why the conservative optimization was causing the issue.

While I don't know the answer I'm curious if changing the firweall optimization back to default has any impact on the success of the first call?

If we change the optimization back to default, I would expect the first call to drop even earlier if that was the component that was causing the issue? Can you try this and see if this has any impact? That might give more clues as to what is causing the issue.

I'm on a bare metal install and also seeing the ICMP issue on traceroutes. Screenshot of my liveview log attempt to run mtr to one of quad9's IPV6 servers.

Each ISP is different however, I've found the default gateway WAN_DHCP6 monitoring to be really flaky. In my case it just uses an fe80 address and the latency and packet loss are all over the place, to the point of making the interface appear "down" due to packet loss and causing issues.

For my WAN_DHCP6 I've always had to use a separate, external monitor IP. That stays super consistent and has essentially zero packet loss. In my case, I'm using Quad9 for my WAN_DHCP6 monitor (2620:fe::9).

For my WAN_DHCP monitor I'm just using the default value that OPNsense discovers when it requests an IP. Again in my case, its worked well for years.

I have the byte value for both of my monitors at 3.

What IP is OPNsense selecting for the WAN gateway monitor? Is it directly on the ATT provided gateway, or is this an IP further upstream on their network?

If it's an IP upstream, can you duplicate the higher response times when you manually ping the same IP from a client on your LAN?

This may be a case of ATT starting to de-prioritize repetitive ping traffic over a period of time from the same device on their network.

Two other things I would try:
Go to System/Gateways/Configuration and click the edit button for your WAN DHCP gateway.

In there set your data length to a small number, usually 3 to 5. This will assign a byte value to your ICMP packets and some ISPs will handle them differently after you do this and therefore, you may see more consistent gateway monitoring.

On the same configuration page, you can also try to change the monitor IP to a known stable WAN IP (4.2.2.2, 8.8.8.8, 1.1.1.1, 9.9.9.9, etc.). This will be useful if your ISP is slowly throttling your pings to their own internal gateway over a period of time. Using an external monitor IP will also tell you if the ping replies continue to increase, the issue lies somewhere within the ISP side and is impacting all your traffic, not just the 1st hop pings.

Thanks, that's good feedback. I do see the CWWK units being considered slightly less "janky" than some of the other clones.

Still, after I buy my own RAM, SSD, and 12v external power supply the value of these units gets lower.

My 2c right now is I'd rather still build my own and get solid firmware updates that are more trustworthy. I realize this won't work for everyone but for me space isn't an issue so re-using an old mATX or ATX case and having an N100 board isn't a big deal. I already have a PicoPSU that I used for my first low power router build some years back, so I could just re-use that for the N100 machine.

My current system is an Asrock J3455M. I can trust the bios updates available on Asrock's website. And the CPU never goes above 40c with the stock heatsink (the case is fanless).

The only downsize to my approach is that a dual port or quad port 2.5G NIC isn't readily available. If 2.5G WAN speeds are a requirement, using these Chinese devices may be the best option with their multiple integrated i226 ports. However, even those NICs seem quite flaky and there's many reports of the NICs randomly failing or interfaces being randomly re-assigned if a NIC goes bad (igb2 becomes igb1, etc. etc.).

No problem. :) That seems to be the vast majority of why these Chinesium N100 devices are unstable.

Please report back after a while and let us know how it turns out. I've also been interested in an N100 based device but between the microcode and the generic Chinese devices being a train wreck on their firmware I've stayed away from them.

For the extended dhcp6c logging, you can enable that under Interfaces/Settings/IPv6 DHCP and select "debug" level logging from the dropdown menu. You can also set a DHCPv6 UID there, which can be handy for some ISPs.

The next step would be to verify the settings that worked for years. I presume you're using WAN DHCPv6 and LAN is set to track interface? Do you use any manual LAN DHCPv6 assignments or prefix IDs?