1
24.7 Production Series / Re: Updated to 24.10 broken business subsricption?
« on: October 18, 2024, 07:59:58 pm »
Thank you. That plus deleting all libfetch_crl. files in /tmp did the trick.
All working now.
All working now.
Show Posts
This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.
Pages: [1] 2
2
24.7 Production Series / Re: Updated to 24.10 broken business subsricption?
« on: October 18, 2024, 07:26:41 pm »
I am also running into this issue:
Error logs when running the update:
I can however access the ULR https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/latest/packagesite.txz with my browser (using my subscription ID instead of the variable).
Using CURL on the router itself to download the file also works and I can see when using update via the SSH menu that the correct URL is being used. Still gets an authentication error:
Error logs when running the update:
Code: [Select]
***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 24.10 at Fri Oct 18 10:20:45 PDT 2024
Fetching subscription information, please wait... Could not load CRL file /tmp/libfetch_crl.24101810
fetch: https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/subscription: Authentication error
Fetching changelog information, please wait... Could not load CRL file /tmp/libfetch_crl.24101810
fetch: https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/sets/changelog.txz: Authentication error
Updating OPNsense repository catalogue...
Could not load CRL file /tmp/libfetch_crl.24101810
Could not load CRL file /tmp/libfetch_crl.24101810
Could not load CRL file /tmp/libfetch_crl.24101810
Could not load CRL file /tmp/libfetch_crl.24101810
Could not load CRL file /tmp/libfetch_crl.24101810
Could not load CRL file /tmp/libfetch_crl.24101810
pkg: https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/latest/meta.txz: Authentication error
repository OPNsense has no meta file, using default settings
Could not load CRL file /tmp/libfetch_crl.24101810
Could not load CRL file /tmp/libfetch_crl.24101810
Could not load CRL file /tmp/libfetch_crl.24101810
pkg: https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/latest/packagesite.pkg: Authentication error
Could not load CRL file /tmp/libfetch_crl.24101810
Could not load CRL file /tmp/libfetch_crl.24101810
Could not load CRL file /tmp/libfetch_crl.24101810
pkg: https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/latest/packagesite.txz: Authentication error
Unable to update repository OPNsense
Error updating repositories!
Checking integrity... done (0 conflicting)
Your packages are up to date.
***DONE***
I can however access the ULR https://opnsense-update.deciso.com/${SUBSCRIPTION}/FreeBSD:14:amd64/24.10/latest/packagesite.txz with my browser (using my subscription ID instead of the variable).
Using CURL on the router itself to download the file also works and I can see when using update via the SSH menu that the correct URL is being used. Still gets an authentication error:
Code: [Select]
Fetching change log information, please wait... Could not load CRL file /tmp/libfetch_crl.24101810
fetch: https://opnsense-update.deciso.com/<removed>/FreeBSD:14:amd64/24.10/sets/changelog.txz: Authentication error
done
3
Hardware and Performance / Re: DEC 4040 / iperf3 10G single threat question
« on: October 04, 2024, 01:21:00 am »
What you are seeing is pretty much expected when running iperf with a single stream/connection.
Can you run some tests using parallel streams? The -P option is what you need to add.
E.g. -P 10 would be 10 connections.
I still do see a performance issue with 24.7 that is not in 21.4 when it comes to 10gig performance but let's see what results you get with a more realistic test setup.
Can you run some tests using parallel streams? The -P option is what you need to add.
E.g. -P 10 would be 10 connections.
I still do see a performance issue with 24.7 that is not in 21.4 when it comes to 10gig performance but let's see what results you get with a more realistic test setup.
4
24.7 Production Series / Re: Performance issue on DEC850 after upgrade to 24.7
« on: September 16, 2024, 10:42:23 pm »
I just went to 24.4 to 24.7 again for a test of the performance in a real production environment as I was no longer seeing performance issues on 24.7 with RSS enabled in my lab.
Unfortunately took less than 24h for the performance to drop from 9.4gig (expected and normal using 24.4) down to less than 8G down and 6G upload.
I really cannot put my finger on what is causing the issue.
Take the exact same config file on 24.4 and no performance issues. A reboot of 24.7 restores the performance for a few hours before it degrades again.
I understand that there are likely very few people in this forum that run 24.7 in a 10gig WAN environment and on a DEC850 or any other A20 based (axgbe NIC) based system but would like to hear if somebody solved this or sees no performance issues.
Let's figure this out before a potential bug makes it into the business edition. This is likely a FreeBSD related issue but I hope that a few of you are willing to work with me on figuring this out.
EDIT: A few more observations and relevant details:
1. My 10Gig WAN has a fix IPv4 that is used with NAT and a fixed /56 IPv6 subnet not used with NAT. Essentially a direct BIDI connection that goes straight into my ISP’s back-end router.
2. The issues happens with both IPv4 and IPv6 traffic ruling out issues with NAT.
Thanks,
Sven
Unfortunately took less than 24h for the performance to drop from 9.4gig (expected and normal using 24.4) down to less than 8G down and 6G upload.
I really cannot put my finger on what is causing the issue.
Take the exact same config file on 24.4 and no performance issues. A reboot of 24.7 restores the performance for a few hours before it degrades again.
I understand that there are likely very few people in this forum that run 24.7 in a 10gig WAN environment and on a DEC850 or any other A20 based (axgbe NIC) based system but would like to hear if somebody solved this or sees no performance issues.
Let's figure this out before a potential bug makes it into the business edition. This is likely a FreeBSD related issue but I hope that a few of you are willing to work with me on figuring this out.
EDIT: A few more observations and relevant details:
1. My 10Gig WAN has a fix IPv4 that is used with NAT and a fixed /56 IPv6 subnet not used with NAT. Essentially a direct BIDI connection that goes straight into my ISP’s back-end router.
2. The issues happens with both IPv4 and IPv6 traffic ruling out issues with NAT.
Thanks,
Sven
5
24.7 Production Series / Re: Performance issue on DEC850 after upgrade to 24.7
« on: August 12, 2024, 05:59:20 pm »
I debugged this some more. There seems to be a change in how FreeBSD handles network load scheduling across cores. Can you try to set "net.inet.rss.enabled" to "1" in tunables and test again? I saw a noticeable improvement (back to line speed) after setting it.
6
24.7 Production Series / Re: Failing widgets after upgrade to 24.7
« on: July 25, 2024, 08:59:59 pm »
I do run into the issue on Edge.
7
24.7 Production Series / HELP NEEDED: Performance issue on DEC850 after upgrade to 24.7
« on: July 25, 2024, 08:59:14 pm »
I see a noticeable performance impact on 10G routing after upgrading my DEC850 to 24.7. Going back to 24.1 solves the issue.
Setup:
I am on a 10G internet connection and use a DEC850. AX0 is the WAN port and AX1 is used as LAN. AX1 has several v-lans. No other ports are in use.
The recommended tuning parameters to get 10G performance are set
Issue:
On prior release (including 24.1) is was able to get "line" performance while having relatively low CPU load on the 8 cores the DEC850 has. Since upgrading to 24.7 the network performance is inconsistent and no longer line performance. CPU core spike at 100% during transfer tests.
Test to repro
Using a device connected to the LAN, run a moderate network load to a WAN target that can provide consistent performance close to 10G line speed. For my example I used "iperf3 -c speedtest.sea11.us.leaseweb.net -p 5201-5210 -P10" which normally results in about 9.4 to 9.3 Gbits/sec and a well distributed load across CPU cores. Screenshot attached (24.1.jpg)
Running the same test on 24.7 results in only 8.8 Gbits/sec down and ~5 Gbits/sec up while maxing out most CPU cores. Screenshot attached (24.7.jpg).
I have two DEC850 units so that I was able to do a clean install on both for the test and apply the exact same configuration.
Is anybody else seeing this with 10G or higher speed WAN connections?
Setup:
I am on a 10G internet connection and use a DEC850. AX0 is the WAN port and AX1 is used as LAN. AX1 has several v-lans. No other ports are in use.
The recommended tuning parameters to get 10G performance are set
Code: [Select]
dev.ax.0.iflib.override_nrxds = 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048
dev.ax.0.iflib.override_ntxds = 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048
dev.ax.0.rss_enabled = 1Same settings for ax.1 as well.Issue:
On prior release (including 24.1) is was able to get "line" performance while having relatively low CPU load on the 8 cores the DEC850 has. Since upgrading to 24.7 the network performance is inconsistent and no longer line performance. CPU core spike at 100% during transfer tests.
Test to repro
Using a device connected to the LAN, run a moderate network load to a WAN target that can provide consistent performance close to 10G line speed. For my example I used "iperf3 -c speedtest.sea11.us.leaseweb.net -p 5201-5210 -P10" which normally results in about 9.4 to 9.3 Gbits/sec and a well distributed load across CPU cores. Screenshot attached (24.1.jpg)
Running the same test on 24.7 results in only 8.8 Gbits/sec down and ~5 Gbits/sec up while maxing out most CPU cores. Screenshot attached (24.7.jpg).
I have two DEC850 units so that I was able to do a clean install on both for the test and apply the exact same configuration.
Is anybody else seeing this with 10G or higher speed WAN connections?
8
24.7 Production Series / Re: Failing widgets after upgrade to 24.7
« on: July 25, 2024, 08:25:02 pm »
Running into the same issue on a DEC850 which should have enough CPU resources.
There must be something else going on that is not caused by low performance systems.
There must be something else going on that is not caused by low performance systems.
9
Hardware and Performance / Re: BIOS and BMC firmware for Supermicro A2SDi-4C-HLN4F
« on: May 20, 2024, 10:13:42 pm »
I applied it earlier today to two of my servers and had no issues.
All working as expected.
Did the update via UFI using the built in console which is fast and easy.
The BMC update is a separate update that I applied via the web interface.
Take note of any BIOS settings as they will all be reset to default by the update.
The BMC update gives you the option to retain all BMC related settings and that worked out fine.
All working as expected.
Did the update via UFI using the built in console which is fast and easy.
The BMC update is a separate update that I applied via the web interface.
Take note of any BIOS settings as they will all be reset to default by the update.
The BMC update gives you the option to retain all BMC related settings and that worked out fine.
10
23.7 Legacy Series / Re: 23.7.3 Issues with IPv6 on wireguard
« on: August 31, 2023, 02:47:47 am »
I don't have IPv6 issues (tested and all working) but do not see anything in the logs.
Running os-wireguard. Is that expected?
Running os-wireguard. Is that expected?
11
23.7 Legacy Series / Re: Bad 10 Gbit routing performance
« on: August 04, 2023, 12:36:07 am »
Here you go:
Relevant Tunables:
I tested different SFP+ modules and it made no difference. Currently running
LAN:10Gtek SFP+ DAC Twinax Cable
WAN: Flexoptix BIDI module from my ISP
Are you getting 10G performance if you are just checking from test PC to test PC with no router in the middle?
Also, do you have flow control disabled on you switches?
Relevant Tunables:
Code: [Select]
dev.ax.0.iflib.override_nrxds = 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048
dev.ax.0.iflib.override_ntxds = 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048
dev.ax.0.rss_enabled = 1
dev.ax.1.iflib.override_nrxds = 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048
dev.ax.1.iflib.override_ntxds = 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048
dev.ax.1.rss_enabled = 1
hw.ibrs_disable = 1
net.isr.bindthreads = 1
net.isr.maxthreads = -1
vm.pmap.pti = 0I tested different SFP+ modules and it made no difference. Currently running
LAN:10Gtek SFP+ DAC Twinax Cable
WAN: Flexoptix BIDI module from my ISP
Are you getting 10G performance if you are just checking from test PC to test PC with no router in the middle?
Also, do you have flow control disabled on you switches?
12
23.7 Legacy Series / Re: Services: DHCPv6: Leases are not showing.
« on: August 03, 2023, 08:18:54 pm »
No need to mess with PHP files.
Just run the following via an SSH connection (or serial):
opnsense-patch 23318015b 4ec97df4c
Just run the following via an SSH connection (or serial):
opnsense-patch 23318015b 4ec97df4c
13
23.7 Legacy Series / Re: Bad 10 Gbit routing performance
« on: August 03, 2023, 08:12:16 pm »
I am running on a DEC850 and see line performance on 10G.
https://www.speedtest.net/result/c/59052567-48a8-470c-8dd0-9ead1e3f4034
This is with RSS and all hardware offloading enabled.
https://www.speedtest.net/result/c/59052567-48a8-470c-8dd0-9ead1e3f4034
This is with RSS and all hardware offloading enabled.
14
23.7 Legacy Series / Re: Services: DHCPv6: Leases are not showing.
« on: August 01, 2023, 07:55:22 pm »
Interfaces do show for me.
15
23.7 Legacy Series / Re: Services: DHCPv6: Leases are not showing.
« on: August 01, 2023, 07:06:59 pm »
Same issue with most clients showing offline.
Debugging this a bit more, I see that all online client have static IPs and no client with a dynamic IP shows online.
Debugging this a bit more, I see that all online client have static IPs and no client with a dynamic IP shows online.
Pages: [1] 2