OPNsense Forum
Archive => 21.7 Legacy Series => Topic started by: cookiemonster on July 30, 2021, 05:34:37 pm
-
I am looking forward to performance improvements on these. One can hope.
With the release notes stating the release includes Intel contributions to the drivers, I'd like to know if there are any "old" tunables that aren't needed anymore.
Has anybody upgraded an APU4D4?, any observations?
With only 4 GB of RAM, do we need to use zfs tunables to prevent use of ARC?
What igb tunables are you using?
-
Hello,
The apu3A2 successfully migrated to 21.7 after patch 21.1.8_2.
21.1.8 opnsense environment, custom ports, clamav, freshclam, maltrail, ntopng, suricata, no sensei.
The apu4C4 is picking up the passage in 21.7 before the patch 21.1.8_2.
21.1.8 opnsense environment, custom ports, lamav, freshclam, maltrail, ntopng, suricata, with sensei.
https://forum.opnsense.org/index.php?topic=24136.0
Cordially,
-
Good to hear. Are you leaving tunables on 21.7, can you share?
Also are you using ZFS?
-
OPNsense ZFS defaults work fine on APUs with 4GB ram, installed on multiple APU4s and APU3s and restored the config, no need for additional tunables so far.
-
Same here, just did a test install on my apu4d4 before moving on to my main firewall later today.
-
Thank you both.
It sounds like using zfs instead of ufs on these is not a problem with only 4GB of ram. That's one worry crossed out.
Is any of you on an ISP package of 500 Mb or more, and tested throughput? The reason to ask is that we know the throughput -going by threads here on 21.1- there were tunables used to squeeze igb to reach near 1 Gbps with NAT.
-
One more question, I'm itching to upgrade for ZFS.
Is ARC consuming most of the memory as expected?
I've experienced memory starvation on ZFS systems and I had to tune ARC, hence I ask if we need to do so on the APUs.
-
Could anyone in the know explain the benefits of ZFS over UFS with regards to OPNSense?
I did do some reading, but it looked to me like ZFS was more applicable within a virtualised environment...?
What are the benefits on a bare metal install?
-
Could anyone in the know explain the benefits of ZFS over UFS with regards to OPNSense?
I did do some reading, but it looked to me like ZFS was more applicable within a virtualised environment...?
What are the benefits on a bare metal install?
I'm familiar wiht ZFS for storage systems, not for firewalls. Therefore I can't answer this question.
On the other hand, I have now upgraded my APU4 to 21.7 with a fresh install & import. I did experience some niggles mostly to do with plugins, special configs and an external repo. I did get an error that seem to have not reappeared after a reboot but I'll look around for it in the system to be sure. All solved now except that one errot and I'm just monitoring for memory as I said, so that the ARC doesn't give me trouble.
All said a very nice update. It shows how much effort has gone into it.
-
Could anyone in the know explain the benefits of ZFS over UFS with regards to OPNSense?
Pull the power plug of your firewall at any random time. With ZFS not much will happen. With UFS the system might become unbootable. Simple as that.
https://www.truenas.com/docs/references/zfsprimer/
-
Everyone without an UPS will silently agree even if they don't know they need it yet. ;)
What's been reported a number of times now is that after you safely unpack a new kernel and reboot cleanly UFS could have been corrupted anyway and the next boot will not work anymore.
Cheers,
Franco
-
Indeed Franco. In my opinion all things being equal ZFS is superior to UFS. That part for me is beyond debate.
The reason I've been asking is because ZFS benefits have a cost in the ARC memory consumption. With these APUs that are very memory-constrained, we need to see if ZFS is suitable or not.
Happily so far with 1 day uptime, my memory consumption has gone up to 54% but that's not a problem. I think it used to hover around 35% on 20.1 on UFS, the same services.
What I have been looking for is zfs tuning hints on loader.conf to limit the arc consumption so I don't end up with an out of memory system. So far none appear there.
May I ask if 21.7 is using freebsd zfs and arc defaults.
-
I think I have my answer listing the sysctl values.
One more question please. Do we have a boot environment utility included like deadm, or is the user expected to get it in order to use boot environments?
I'm loving this release!
-
bectl is in the base system in 12.x ...
-
Much obliged @pmhausen
-
@cookiemonster: Any changes regarding throughput? And did you keep those tunables?
My APU2 delivers ~400 Mbit/s iperf3 NAT throughput with a single connection and ~938 Mbit/s with 4 concurrent streams. I could swear I had that almost-gigabit throughput on 21.1.9 with one connection but I've got no data to prove it.
-
Hi lynxix. My fresh install naturally wiped out my tunables in rc.conf.local and only kept the ones from the gui.
There I only have
hw.igb.rx_process_limit = -1
hw.igb.tx_process_limit = -1
legal.intel_igb.license_ack = 1
I've left those and monitoring but unfortunately I haven't had the opportunity to plug a machine into the WAN side to test NAT throughput. It's in my to-do list.
I've done iperf tests to public iperf servers and the throughput seems unchanged from 21.7 with tunables for igb.
The problem for me is that these I can't tell if they're limited on the public servers. I really need to do the test internally by myself.
My ISP service is just over 500 mbps down. I'd love to know how close or far from it I am.
Also I need to research what it means for the APU with the change to em that I think 21.7 changed to.
How are your tests going?
-
Most of the tunables have changed with 21.7. Only one of the three you have mentioned is still there but under a different name:
hw.igb.rx_process_limit --> hw.em.rx_process_limit
Credit goes to Gary7 in this thread (https://forum.opnsense.org/index.php?topic=24247.0) for clarifying this. For me that tunable doesn't seem to have an effect at all anymore.
With my APU2 I'm reaching ~300 Mbit/s NAT throughput when using a single connection and ~938 Mbit/s with four connections. An important thing to keep in mind is that if you run 'iperf3' on the box itself it will be CPU-bottlenecked, so always measure throughput between two machines with the APU in between.
My ISP gives me ~600 Mbit/s downstream so I really would like to reach that with a single connection. But I have to admit I'm not absolutely sure I had that with 21.1.9. Maybe I just never noticed.