Messages

Very strange...  I was about to flash back from coreboot to the standard AMI BIOS as a (very) out-side possibility, decided to test a few other things first.

1) I'm using a firewall group for "Inside Networks"
2) I have the main LAN interface assigned as a fail-over LAG (POS Netgear smart-switch can't do LACP and doesn't behave properly with static or LB modes - wish I would have thought to look for EOS / EOL cisco switches on amazon / e-bay)

I dropped the firewall group after moving the rules to the individual interfaces - and the problem largely disappeared.  Used the same testing methodolgy as before (enable/disable rule-set, download and update rules, restart service, go to http://www.dslreports.com/speedtest) and definitely getting different results without the firewall group.

Rulesets for botcc.portgroupd, ciarmy, emerging-malware, and emerging-mobile_malware now have no impact on throughput.  Attempting to use emerging-web_client still tanks throughput though - guessing that is a ruleset issue.  I don't want to tear-down lagg0 just to check this rule-set as it is a PITA to reassign everything, wish there was a simple way to "reassign" an interface (but get why there isn't).

This doesn't make much sense to me, other than a weird possibility that ipfw has some sort of "translation" issue talking to pf if a group is in use.  Can't sort out in my head how that would happen though...

Between the Unbound DNS SBLs, a firewall drop alias for https://sslbl.abuse.ch/blacklist, http://rules.emergingthreats.net/blockrules/compromised-ips.txt, http://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt, and the above IPS rulesets that do work I'm pretty happy with the additional level of protection.

Thank you to the devs for an excellent product.

LOTRouter, out of curiousity - is your unit using CoreBios?

Sorry, I know there are a lof of threads related to this.  I'm new to opnsense, but have done a fair bit of research and looking through the forum threads as well.
I have a Protectli FW6B with an Intel  i3-7100U (2-core / HT, 2.4GHz) with 16GB of RAM and (6) Intel 82583V (I210) NICs.  Internet pipe is 300Mbps fiber-to-the-home.  Opnsense initial install was 20.7, now at 21.1.2.

With IPS enabled and only using a single-ruleset (emerging web client, 859 rules) online speedtests show ~190Mbps down / ~290Mbps up.  I thought it might be related to the Intel NICs, as there are a lot of comments about them, so I went through all the standard "tunables" and made certain the em and dev.em.X.iflib settings werre the "recommended" for 1Gb Intel NICs.  This made no difference what-so-ever.

Disabling IPS, online speedtests immediately change to ~580Mbps down / ~290MBps up (fyi, this is a little better than the results I got with my old consumer router - which I would expect to be the case).  This is fully repeatable.  There also doesn't appear to be ANY change in CPU utilization / Memory consumption during the tests with IPS on or off...  I've also tried with different rulesets (malware, ciarmy, botcc.portgroup) and get identical results.

To me this doesn't seem like expected behavior and even though it's only an I3 it "should" be able to keep up (based on experience with commericial firewalls that have UTM).  It seems very strange to me as well that the CPU utilization doesn't appear to change with IPS enabled.  But again, I am new to opnsense, and it has been quite a while since I've played with *nix firewalls / routers of any type.  Are my expectations off?  Or am I missing something?

em0 is the WAN port, em1 (LAN) configured identically.  Main references for settings were various forum posts and https://calomel.org/freebsd_network_tuning.html
ifconfig em0
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=802008<VLAN_MTU,WOL_MAGIC>
        media: Ethernet autoselect (1000baseT <full-duplex>)

sysctl hw.em
hw.em.max_interrupt_rate: 8000
hw.em.eee_setting: 1 (per man this is the disabled value)
hw.em.rx_process_limit: 1000 (default is 100, without MSIX hesitant to try "-1")
hw.em.smart_pwr_down: 0

sysctl dev.em.0
dev.em.0.eee_control: 1 (per man this is the disabled value)
dev.em.0.fc: 0
dev.em.0.iflib.disable_msix: 1 (this appears to be default, several reports of traffic failure with MSIX enabled)

dmesg | grep em0
em0: <Intel(R) PRO/1000 Network Connection> port 0x2000-0x201f mem 0x7e400000-0x7e41ffff,0x7e420000-0x7e423fff irq 16 at device 0.0 on pci1
em0: Using 1024 TX descriptors and 1024 RX descriptors
em0: Using an MSI interrupt
em0: Ethernet address: 00:e0:67:21:c4:36
em0: netmap queues/slots: TX 1/1024, RX 1/1024

sysctl kern.ipc
kern.ipc.maxsockbuf: 16777216
kern.ipc.nmbclusters: 492680

sysctl net.inet.tcp.tso
net.inet.tcp.tso: 0

Edit: forgot to include the changed entropy pool (shouldn't matter at less than 10Gbps anyway)
sysctl kern.random
kern.random.harvest.mask_symbolic: PURE_RDRAND,[UMA],[FS_ATIME],SWI,[INTERRUPT],NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED
kern.random.harvest.mask: 65887

Perhaps if I spelled "tunables" correctly I would have seen this before posting:
https://forum.opnsense.org/index.php?topic=21451.msg100910#msg100910
Known bug, will be fixed in 21.1.2

Running OPNsense 21.1.1-amd64 (updated from initial install of 20.7.8), the edit doesn't appear to be working correctly in System -> Settings -> Tuneable - it brings up a _different_ setting than the one the edit button is clicked for.
Example:
The "target URL" for net.inet.icmp.drop_redirect shows /system_advanced_sysctl.php?act=edit&id=7, clicking the button brings up the setting for net.inet.icmp.log_redirect.
The edit button brings up the wrong "target" for every tuneable, but is "consistent" in doing so (i.e., net.inet.icmp.drop_redirect brings up net.inet.icmp.log_redirect every time).  This happens in every browser I've tested with.