OPNsense Forum

English Forums => 25.1, 25.4 Production Series => Topic started by: TomT on February 02, 2025, 10:42:25 PM

Title: Config tuning after server move.
Post by: TomT on February 02, 2025, 10:42:25 PM
Hi.
I've moved my config from an AMD GX-420CA system to an Intel N100 system, apart from updating the temperature sensors and adding the following tuneables, is there anything else I need to update or change to get the max performance out of this ?

net.isr.dispatch: deferred
net.isr.bindthreads: 1
net.isr.maxthreads: -1
net.inet.rss.bits: 2
net.inet.rss.enabled: 1

Speedtests seem good, may be 20/30Mbps slower than the ISP router.
Any tips for tweaking the last bit out of the connection ?

I have IPSec and WireGuard VPNs configured, do I need to do anything else for them ?

In plugins I have two entries in red.
os-realtek-re (missing)
os-udpbroadcastrelay (missing)

I don't need either of those, how do I clear the red entries ?

Thanks
Title: Re: Config tuning after server move.
Post by: TomT on February 03, 2025, 12:35:49 AM
I've still got powerd enabled, should it be for an Intel N100 ?
If I have to disable it, is there anything else I need to enable or configure?
Thanks
Title: Re: Config tuning after server move.
Post by: newsense on February 03, 2025, 03:58:40 AM
This is a must for N100 and later CPUs, reboot after installation.

pkg install os-cpu-microcode-intel-1.1
You can fix the missing plugins in the Firmware section.
Title: Re: Config tuning after server move.
Post by: pouakai on February 03, 2025, 06:40:50 AM
I ended up disabling PowerD on an N100 and used this post:

Do I need to enable PowerD (https://forum.opnsense.org/index.php?topic=42817.0)

Note that the thread mixes up SpeedStep and Speed Shift in a few places!  I think this is the right way around:

QuoteSpeedStep is the "old" method - needs PowerD.
Speed Shift is the new method and its available since 6th Gen of Intel CPUs (I think) - doesn't need PowerD.
Title: Re: Config tuning after server move.
Post by: TomT on February 03, 2025, 09:49:46 AM
Quote from: newsense on February 03, 2025, 03:58:40 AMThis is a must for N100 and later CPUs, reboot after installation.

pkg install os-cpu-microcode-intel-1.1
You can fix the missing plugins in the Firmware section.

Thanks I'll install that and reboot.

I ran the plugin conflict checker which install the kissing plugins. I then uninstalled them as they aren't needed.
Title: Re: Config tuning after server move.
Post by: TomT on February 03, 2025, 09:51:24 AM
Quote from: pouakai on February 03, 2025, 06:40:50 AMI ended up disabling PowerD on an N100 and used this post:

Do I need to enable PowerD (https://forum.opnsense.org/index.php?topic=42817.0)

Note that the thread mixes up SpeedStep and Speed Shift in a few places!  I think this is the right way around:

QuoteSpeedStep is the "old" method - needs PowerD.
Speed Shift is the new method and its available since 6th Gen of Intel CPUs (I think) - doesn't need PowerD.

Thanks is this as simple as disabling PowerD or do tuneables ned setting?
Thanks
Title: Re: Config tuning after server move.
Post by: Seimus on February 03, 2025, 10:57:59 AM
Disable PowerD.

The N100 has on by default all enabled on FreeBSD, you can check your LOG if is really the case. If yes PowerD does not control the CPU anymore.

Regards,
S.
Title: Re: Config tuning after server move.
Post by: TomT on February 03, 2025, 05:16:52 PM
Thanks,

I've disabled PowerD is there anything specific I need to check in the logs ?

Does System: Settings: Miscellaneous:  Hardware acceleration need enabling ? if it does, is that Intel Quick Assist ?

Thanks
Title: Re: Config tuning after server move.
Post by: TomT on February 03, 2025, 11:35:37 PM
Everything seems to be running ok.
Reading another blog there are suggestions for the following tuneables.

hw.ibrs_disable = 1
net.inet.ip.intr_queue_maxlen = 3000
hw.igc.max_interrupt_rate = 20000
hw.igc.rx_process_limit = 5000

Are these worth doing, will they help me get similar speeds to my ISPs router ?
Thanks
Title: Re: Config tuning after server move.
Post by: meyergru on February 04, 2025, 12:20:05 AM
Since you use PPPoE: Did you optimize your MTU? 20-30 Mbps less with 900 MBps theoretical limit means 2% lesser throughput. If your MTU is not optimized to yield 1500 bytes after PPPoE and VLAN overhead, speed may decrease even (or especially) if you clamp your MSS to counter other problems.

See point 9 here (https://forum.opnsense.org/index.php?topic=42985.msg213524#msg213524).

Also, you can try traffic shaping (see official docs).
Title: Re: Config tuning after server move.
Post by: TomT on February 04, 2025, 06:46:47 PM
Quote from: meyergru on February 04, 2025, 12:20:05 AMSince you use PPPoE: Did you optimize your MTU? 20-30 Mbps less with 900 MBps theoretical limit means 2% lesser throughput. If your MTU is not optimized to yield 1500 bytes after PPPoE and VLAN overhead, speed may decrease even (or especially) if you clamp your MSS to counter other problems.

See point 9 here (https://forum.opnsense.org/index.php?topic=42985.msg213524#msg213524).

Also, you can try traffic shaping (see official docs).

Thanks.
I've run the mtu.sh script and it recommended 1484
Looking in the GUI the MTU was set to 1492 ( Calculated PPP MTU: 1484 )

Does that mean I need to change it ?

Thanks
Title: Re: Config tuning after server move.
Post by: meyergru on February 04, 2025, 07:21:20 PM
1. You have to set the MTU to a value <= your possible MTU, or you will have problems with sites without PMTU discovery.
2. To optimize that, it should be = your possible MTU.
3. With some hardware setups and ISPs, you can use even more than 1500 bytes on the underlying physical link, thus giving you again 1500 bytes even with PPPoE and / or VLANs in use, which overheads would normally have to be deducted from the physical link's MTU (often 1500).

To reiterate on #3, I gave detailed instructions here: https://forum.opnsense.org/index.php?topic=44271.msg220936#msg220936, this was linked in my instructions under point #9 already.
Title: Re: Config tuning after server move.
Post by: TomT on February 05, 2025, 09:53:51 AM
Quote from: meyergru on February 04, 2025, 07:21:20 PM1. You have to set the MTU to a value <= your possible MTU, or you will have problems with sites without PMTU discovery.
2. To optimize that, it should be = your possible MTU.
3. With some hardware setups and ISPs, you can use even more than 1500 bytes on the underlying physical link, thus giving you again 1500 bytes even with PPPoE and / or VLANs in use, which overheads would normally have to be deducted from the physical link's MTU (often 1500).

To reiterate on #3, I gave detailed instructions here: https://forum.opnsense.org/index.php?topic=44271.msg220936#msg220936, this was linked in my instructions under point #9 already.


Please correct me on this.. Should my MTU be set to 1500, so the calculated will then be 1492 ?
I've never understood how this is meant to work.

Thanks
Title: Re: Config tuning after server move.
Post by: meyergru on February 05, 2025, 10:36:33 AM
I don't really know what to tell you beyond what I already said or linked any more.

You have to set your WAN MTU at a maximum of what actually is possible (mtu.sh can tell you what that is) - otherwise you will see real problems sooner or later. Probably, you can make something more than 14xx Bytes available by enlarging the MTU (to, say, 1512 bytes) of the underlying layers of your internet connection in order to optimize the payload/gap ratio (that depends on if your ISP can do it and your particular hardware). You may be able to get a net 1500 bytes MTU over the WAN interface.

Detailed instructions on how to do that have been given. If you want to get the last 2% speed, you will have to experiment with the MTUs (yes - there are multiple of them, on different layers) yourself.
Title: Re: Config tuning after server move.
Post by: TomT on February 05, 2025, 04:41:56 PM
Thanks again.

My confusion is related to what I set vs what is computed.
If I enter the value from the MTU script, that makes the computed a lower value.

Should the WAN MTU be the script value, or should the computed value be the script value ?

Thanks
Title: Re: Config tuning after server move.
Post by: meyergru on February 05, 2025, 05:51:07 PM
There is nothing that is "computed".

The mtu.sh script will tell you what really gets through in the upmost layer. That is what you can set the MTU of the upmost layer (=WAN) to.

What actually can get through, though, depends on what the lower layers can accomplish. This is dependent on what the hardware can do, but also limited by what MTU you set there. Also, each layer deduct from its predecessors, because the packets payload is augmented by the overhead of the lower layers, like VLAN and PPPoE.

Thus, if you want 1500 MTU on the WAN layer (and you do!), you must have more than that on the the PPPoE and VLAN layers, respectively. Normally, to have 1500 bytes MTU as a payload on the WAN layer, you need to have 1512 MTU on the physical layer (which is ethernet and usually has a default MTU of 1500), such that VLAN header, PPPoE header and 1500 bytes of payload will fit in. Thus, you will have to set MTUs for every single layer like I depicted (even with the unexpected idosyncrasies included) in my instructions. Each layer adds another few bytes of overhead to the payload.

IDK what works in your specific case, you will have to find out for yourself and test the result with mtu.sh.

P.S.: Obviously, this calls for a HowTo (https://forum.opnsense.org/index.php?topic=45658.msg228347#msg228347).
Title: Re: Config tuning after server move.
Post by: TomT on February 06, 2025, 05:09:15 PM
I've updated my MTU and restarted my server.
So far things are looking ok.

Thanks for your help and advise and the How To.