Messages

Great! At least we know what to do now.
Let's hope reassigning the interfaces will do after changing the NIC.
Else it's going to be a long night.

Might also just end up using proxmox and opnsense within. we've had no issues with those nics there (virtualized, NOT pass through).

Thank you for the assistance and information franco!

We updated to 24.7.1
No issues during updates except CrowdSec. But we solved that and everything is up to date.

We had another crash of said firewall (or at least the bnxt0 and bnxt1) yesterday though. It only came back after several reboots

We can't pin the issue down. The first error messages we can find are those:

2024-08-12T18:45:33   Notice   kernel   bnxt0: Timeout sending HWRM_PORT_PHY_QCFG: (timeout: 2000) seq: 44051   
2024-08-12T18:45:33   Notice   kernel   bnxt0: Timeout sending HWRM_PORT_QSTATS: (timeout: 2000) seq: 44050   
2024-08-12T18:45:33   Notice   kernel   bnxt1: Timeout sending HWRM_PORT_QSTATS: (timeout: 2000) seq: 24278   
2024-08-12T18:36:00   Error   configctl   error in configd communication Traceback (most recent call last): File "/usr/local/sbin/configctl", line 65, in exec_config_cmd line = sock.recv(65536).decode() ^^^^^^^^^^^^^^^^ TimeoutError: timed out   
2024-08-12T18:26:00   Error   configctl   error in configd communication Traceback (most recent call last): File "/usr/local/sbin/configctl", line 65, in exec_config_cmd line = sock.recv(65536).decode() ^^^^^^^^^^^^^^^^ TimeoutError: timed out   
2024-08-12T18:16:00   Error   configctl   error in configd communication Traceback (most recent call last): File "/usr/local/sbin/configctl", line 65, in exec_config_cmd line = sock.recv(65536).decode() ^^^^^^^^^^^^^^^^ TimeoutError: timed out

followed by a lot more kernel notices about bnxt0 and bnxt1

Seems to be similar to this post but not identical: https://forum.opnsense.org/index.php?topic=38434.0

We wonder if it's a hardware issue.. or maybe suricata/sensei?

Any ideas anyone?

Thank you for the swift response franco.

Will do tonight!

Dear Community

Two days ago we've had our OPNsense suddenly stop working (pic1).
It's been working great for months and we have rebooted it many times before without issues.
But this time when we booted.. (pic2). bnxt couldn't load.
Yes we do have a custom tunable in place. only this one (pic3)

After another reboot the OPNsense is working smooth again.
Has anybody ever had a similar event happening with the bnxt driver?
Is there a way to have a deeper look in what actually happened except the Logs in the Web GUI?
We are since a bit afraid to touch it at all.

Would you suggest switching to a natively supported Network card?
It's an idea we've had for a while.
Is the error shown in pic1 the cause for the crash even?

Thank you for your thoughts and ideas :-)

Dear Community

I have read many threads regarding similar issues. But not quite exactly ours.

FW: DEC2685 OPNsense.
WAN: PPPoE
Telephony: Mitel MiVoice Office 400

So far normal calls work without any specific Rules or NAT settings.

The only Problem we have is if people want to redirect calls automatically to their cell phones. The cell phone rings and the connection is stable. BUT we just can't hear each other.

The technician of the Mitel Box told us to disable SIP-ALG, which, as found in other threads, isn't available in OPNsense anyway.

We have tried Port-Forwardings.. Outbound with static port. nothing seems to fix it.
Do you guys think the issue is even Firewall related?
Or is it more likely to be an issue with the Mitel installation?
After all, regular calls in/out do work properly.

PS: In the Mitel GUI, where the SIP and RTP Ports and configured, I could also define a value for "public NAT-Gateway-Adress".. Is it possible I have to use the OPNsense as value? right now the field is NULL. (See attachment)

Kind regards!
Ben

Oh right well that's so obvious.. haha
my colleague will set it up for me.

Thank you for the idea :)

Dear OPNsense users

I have a OPNsense Firewall, off-site. When I installed it, the WebGUI worked.
Few reboots later and I can't access the WebGUI anymore.

Everything else works, really, I believe it's just the webserver

As secure as I am I have disabled SSH. So.. no way to start the webserver from there.

I have installed the business subscription so I believe it's Version 23.x, NOT 24.

Anything else I can try before traveling there for a day?

The Person on site restarted it several times already. Kinda given up that it might start the webserver one time.

Cheers

I have Suricata enabled. I get the Alerts and everything works as expected.

As soon i turn on IPS mode, the complete OPNsense machine freezes.
Has anyone experienced that before? What was your solution?

Hardware CRC, TSO and LRO disabled. NO vlan hardware filtering.

I have tried different Pattern matchers and promiscuous mode.

I tried to delete custom tunables aswell:
net.isr.maxthreads=-1
net.isr.dispatch=deferred
net.isr.bindthreads=1

with no luck.
Ryzen 7700 with a bnxt card and driver.

I wonder if it has anything to do with the driver.. I load it with a tuneable "if_bnxt_load=YES"

I see no errors in the log files in the frontend. Guess the logging freezes too.

I can mount the zfs pool and edit the config.xml file to disable IPS mode and everything works as expected, again! But I'd really like to use suricata in IPS mode, obviously so..

Any help or ideas appreciated!

ah welp. Thanks! should i remove any of the downloaded opnsense-code? Or is there no concern except disk space?

My CPU used to sit pretty much idle.
Constantly at 20% now.

Decided to have a look with htop.
Tried to install with https://forum.opnsense.org/index.php?topic=7796.0

Code Select Expand


root@xxxxx:/usr/ports/sysutils/htop # make install
===>   htop-3.2.2_1 depends on package: pkgconf>=1.3.0_1 - found
===>   htop-3.2.2_1 depends on file: /usr/local/bin/python3.9 - found
===>   htop-3.2.2_1 depends on package: autoconf>=2.71 - not found
===>   autoconf-2.71 depends on executable: gm4 - not found
===>   m4-1.4.19,1 depends on package: libiconv>=1.14_11 - found
===>   m4-1.4.19,1 depends on file: /usr/local/bin/makeinfo - not found
===>   texinfo-7.1,1 depends on executable: help2man - found
===>   texinfo-7.1,1 depends on package: p5-Locale-libintl>=0 - not found
===>   p5-Locale-libintl-1.33 depends on package: perl5>=5.36<5.37 - found
===>   p5-Locale-libintl-1.33 depends on shared library: libiconv.so - found (/usr/local/lib/libiconv.so)
===>  Configuring for p5-Locale-libintl-1.33
env: /usr/local/bin/perl5.36.3: No such file or directory
*** Error code 127

Stop.
make[4]: stopped in /usr/ports/devel/p5-Locale-libintl
*** Error code 1

Stop.
make[3]: stopped in /usr/ports/print/texinfo
*** Error code 1

Stop.
make[2]: stopped in /usr/ports/devel/m4
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/devel/autoconf
*** Error code 1

Stop.
make: stopped in /usr/ports/sysutils/htop


Any help appreciated

Thank you for your recommendations

It works!

Thanks for the clarification!

But when I choose "any" I can access my other vlan subnets.
Does that mean I have to deny DMZ on all other networks?

Or what's the best approach?

Heya.

I can't believe I have to ask this but here we are.
I have read through https://docs.opnsense.org/manual/firewall.html
I have a DMZ vlan and interface.
When I configure following rule:

Interface: DMZ
Direction: in
Source: DMZ net
Destination: any

DMZ can access the internet.
Now when I change Destination to "WAN net", The traffic is blocked by a default rule.
What could be the cause for this issue?
I want clients in the DMZ net to be able to access the internet.
Is this possible with a single rule?

I sit in a /26 subnet from my provider.
Does my rule limit access to said subnet?
What's the best approach? Allow DMZ to any and block all other nets specifically?

I've been digging in the wrong direction I guess. Apparently Hardware Offloading causes too much trouble for the negligible perfomance gains.

The holy grail for "most" modern hardware and 10Gbit/s seems to be following tunables

net.isr.dispatch="deferred"
- Defines which Interrupt Service Routine (ISR) to use. Controls how the CPU handles network interrupts.
(Note that all traffic is a network interrupt message to the CPU)
- "Deferred" Let's the CPU use it's prefered routine. Adds minimalistic delay to queue interrupts before distributing the workload across cores.

net.isr.bindthreads="1"
- Binds ISR threads to specific CPU cores.
- Each thread runs exclusively on one core.

net.isr.maxthreads="-1"
- Sets the maximum threads for network interrupts.
- "-1" allows dynamic adjustment based on workload and available cores.

Most information gathered with following Link and Links in the article:
https://binaryimpulse.com/2022/11/opnsense-performance-tuning-for-multi-gigabit-internet/

Closed

[not]

10Gbit/s Fiber Uplink.
bare metal on a Ryzen 7700 and 64GB of ECC.

The system is not in production yet.
I want to experiment with IPS but would like to have as many things in place as possible.

https://docs.opnsense.org/manual/interfaces_settings.html
suggests to disable pretty much all Hardware capabilities.

https://forum.opnsense.org/index.php?topic=10839.0
suggests to enable VLAN Hardware Filtering.

NIC:

Code Select Expand

options=48500b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,NOMAP>
capabilities=4f507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>

Hardware CRC: Probably.. Disabled?

Hardware TSO: Disabled. I will enable IPS

Hardware LRO: I guess I can leave this disabled as long the CPU can handle all the traffic. right?

VLAN Hardware Filtering: Like stated, the recommendations are mixed.

I haven't found much on the topic so here I am.
Do you guys use Hardware LRO and VLAN Hardware Filtering?
Did you ever have any stability issues when using those?
I appreciate any experience you share!

Best regards