Messages

1

General Discussion / [ solved ] syncookies block http(s) traffic for some not all sites

« on: June 13, 2024, 08:42:27 am »

Because I've spent extraordinary time on finding out since I don't understand the context (yet)

Enabling syncookies blocks some site but not all, this is particulary so for sites hosted by sucuri.net such as linuxmint.com and reportedly also facebook sites. Which makes one wonder what's the common factor here.

Solution is two-fold. Disable syn cookies are set them to adaptive. When setting to adaptive use a value of start of >= 50% but <100%

Hope this helps.

br,

Joris

2

24.1 Legacy Series / Re: First experience with 24.x release

« on: May 26, 2024, 09:37:38 am »

Missing the point goes two ways so it seems. Yeah, i'm also sleep deprived and known to be grumpy, yet helpful I can only hope.

The distinct impression the platform (opnsense obviously since it is the title topic) is faster was/is a compliment. Pages loading faster is indeed a generic statement since both the UI and surfing is perceived to be generally (far) more responsive.

"Lack of consitent behaviour" is the observation a working config does not mean it works reliably for the next few hours, in part it is sure to be me. I'm also not being very specific to OPNSense but point at the dumpster fire Ubuntu in combination with OPNSense administration. For OPNSense it is an impression that just does not go away. It seems states are not purged instantly on commit (apply changes) and linger for some time until the new config is actually in effect.

Paired with the absolute madness Ubuntu is projecting with the latest LTS when it comes to networking on the desktop this past impression does not go away easily. Having worked with Juniper, CheckPoint, Cisco, Meraki, WatchGuard and other odd number of firewalls I hope I'm not simply making weird. I'm stating an observation which is not easy to pinpoint either.

The Suricata issue's with every set-up that does not stick to MTU=default (1500) I think to have observed this is actually from 1500-2048 MTU size. I've shared my fix with support, posted the fix here, nothing happened with that information. As if everyone seems to know already. Yet, forums are plenty with people observing the start/stop of Suricata.

The fix is literally one value in tunables, adding this as a default would save people loads of time.
Fix = Tunable: dev.netmap.bufsize = <highest MTUsize here>
 
People like me who among other things manage firewalls and don't have time to make a dedicated life out of firewall administration for one product.

Regarding GeoIP. I meant, what use is adding this if it is not displayed anywhere in the logs. Right?

OH now I read the business edition has that GeoIPdb on-board already, so the logs ARE void of GeoIP info and no way to add a column to look this up. TLDR on that one, could have known. The 'full help' function in OPNSense could be useful to inform.

For example: checking the Alias i created for Belgium i don't see IP registered for Belgium, such as 193.191.245.121
When i do a lookup reference it tells me it is Belgium but the list of subnets listed only shows 0 values.

By sharing the GEOIP link the collateral finding of 'URL tables' which I'm sure to explore, thanks for that, a new feature to test.

Either way, OPNSense is affordable and good enough for my purpose.
I'm sure people who can dedicate a considerable amount of their time will master to build larger and complex networks with it.

3

24.1 Legacy Series / First experience with 24.x release

« on: May 25, 2024, 09:17:12 pm »

a bad week, karma, or something entirely different, who will tell

things to know for working with 24.x

foremost: know KEA DHCP requires for the GATEWAYIP/MASK to be set, not the subnet/mask as the GUI help suggests. Though Kea is interesting it seems sub-par in features compared to ISC and also seems less configurable. Curious for what will come of this.

There is a distinct impression the platform is just faster, pages load in an instant

Unbound is not behaving as well as can be again, enabling DNSSEC is not recommended at first. Just leave it off. My assumption is an update may fix whatever is going on. If not, let me know what I did wrong.

Pairing Unbound with DNScrypt can be a headache. Just point only the "query forwarding" to the DNSCrypt service, don't combine this with "DNS over TLS" from unbound, stuff breaks here :-D Also, Unbound has its own visual dashboard.

Writing rules 'feels different', could be me paired with a lack of sleep.

I've also had the weirdest experience with getting the network to actually work properly. This not in small part due to Ubuntu 24 LTS, it is not recommended to upgrade just yet. Something seems alive in there and it is cheeky and mischievous.

Somehow "automatic outbound NAT rules" gave up on me a few times. I had to switch to hybrid mode. I mean, is there a ghost in the machine or what ?!?  This makes a requirement to hide each network separately behind the WAN address. Including the WAN network it seems.

There quite a few small and notable changes and improvements. I'm actually getting curious about opnsense again. Though I do think there's too much awkwardness for casual use it's growing on me. I'm sure to try out the central management features.

There something different about how gateways are managed, not sure what, seems too easy now ?

There's something odd about "Dynamic gateway policy", do i need it enabled or not, the change does not seem to propagate or act consistently over time.

Lack of consistent behavior seems to be growing trend with open source software lately, it is quite concering. Settings were saved but were not, flows seemed to pass until they did not. The mess I've seen Ubuntu make of simple things is just disengious. OPNSense seems to suffer from "ghost states" and can sometimes use a reboot. Not recommendable to accept rebooting as a standard practice.

What I miss profoundly are a way to add exceptions to the bogon filtering. Now it has to be disabled because it matches DHCP and disrupts that. There appears to be some things missing here, maybe a feature deprecated ?

Suricata borks again, I hope I find back the post on how to keep it up and not have it randomly crash. It is stupid this is not documented or fixed in the build. Yes, all hardware offloading is disabled. Oh wait, yes, Suricata stops when the MTU is not set consistently for all interfaces. If you change the MTU updated it here. At least that used to work. Now it reports there is an "<Error> -- opening devname netmap:vtnet1/R failed: Invalid argument" I don't see why and did not find why yet. Oh no it does bork for all interfaces, with the same invalid argument argument. Let's disable IPS mode that worked last time, until i remember the fix.

Just in case, here is how i fixed it last time https://forum.opnsense.org/index.php?topic=38140.0

Why is GeoIP under Firewall Aliases ? Why it is documented so vaguely what URL to point at it?
Works though. I think, can i actually see the GeoIP info anywhere ?


In all, it's okay to work with. It is a building block rather than a one stop toolkit.
Reminds me I have to get Elasticsearch up and running again, pair it with Grafana and stuff.




















Setting up DNScrypt requires little but you should know what.

4

Intrusion Detection and Prevention / Re: How to subscribe ETPRO telemetry edition

« on: January 27, 2024, 10:10:45 pm »

is there an error message visible ?

5

Intrusion Detection and Prevention / Re: Suricata slows outbound traffic by a factor of 10 on KVM/virtio

« on: January 27, 2024, 10:10:18 pm »

Having run Suricata on OPNSense with virtio since many years I do not have such issue.
The internet line is 100Mbps but the lan is set to 1Gbps.

Here's my best guess.

Don't try and "tweak" network drivers, this is overwritten in many cases due to limitations in the driver(s) and such.
Yes, especially not with e1000 and related cards.

I've had long time issues with Suricata uptime until I finally had some time and fixed the MTU by setting dev.netmap.bufsize to the relevant MTU value.

Also consider evaluating the MTU for the bridge interfaces. To my understanding it is best to have large MTU for Gbps networks.

With some Linux you (at least this used to be so) may need to tweak sysctl settings to allow for large transfers.
I assume you've checked that ?

Also, are there any console message visible ?

Wrong MTU size for example will throw an error like: netmap_buf_size_validate error: large MTU (8192) needed but igb1 does not support NS_MOREFRAG

6

23.1 Legacy Series / Re: Suricata can`t start in IPS mode

« on: January 16, 2024, 06:05:23 pm »

check my post here (both IPS and IDS are working now)
if need be I can help out debugging this issue[/size]https://forum.opnsense.org/index.php?topic=38140.0the main issue of Suricata failing or not failing are MTU inconsistenciesThere's a typical overhead (8 bytes for Windows / 22 bytes for Linux) to consider but bridges and ppp also add overhead.So, if you start with the default MTU of 1500 (1518) or have  jumbo frames (<=9000 MTU) this will have great effect.I can say with confidence this approach works. Suricata is now up 100% of the time since 24 hours.[/size]

7

Intrusion Detection and Prevention / Re: IPS mode freezing OPNsense

« on: January 16, 2024, 06:01:25 pm »

don't try tuning network cards


unless somehow the driver is very broken and the system deadlocks there should be output in /var/log for causing the freeze

did you try inserting a different network card ?

bnxtload suggest this is a server network card with multiple nic ?

since you mention the ZFS pool, is this by any chance opnsense running in a VM ?

8

Intrusion Detection and Prevention / Re: Suricata in Wan does not work with ppoe

« on: January 16, 2024, 05:58:28 pm »

check my post here (both IPS and IDS are working now)

https://forum.opnsense.org/index.php?topic=38140.0

the main issue of Suricata failing or not failing are MTU inconsistencies

There's a typical overhead (8 bytes for Windows / 22 bytes for Linux) to consider but bridges and ppp also add overhead.

So, if you start with the default MTU of 1500 (1518) or have  jumbo frames (<=9000 MTU) this will have great effect.

I can say with confidence this approach works. Suricata is now up 100% of the time since 24 hours.

9

Intrusion Detection and Prevention / [solved][how-to-fix] OPNSense with Suricata IPS service failure crash

« on: January 15, 2024, 11:30:41 pm »

Please like or share a comment if this post is helpful or you have more questions.


This how-to-fix post to inform people on how Suricata crashes with OPNSense on Proxmox (any version) can be remediated. The advisories here may not be suitable for production environments, I trust you know this already.

SHORT FORM specific to Proxmox


set all bridge interfaces used for opnsense to the same MTU
(it may be required to set the bridge-if MTU to the physical inteface MTU-22)

use the opnsense VM intefaces used for suricata only with virtio network adapters
set the network adapter MTU to 1 to adopt the bridge MTU from proxmox
in opnsense, leave the MTU for the interface blank
in opnsense, leave the MTU for suricata blankin opnsense for Suricata keep the MTU blank and disable promiscuous mode
in opnsense for Suricata set the exact network masks configured for each interface, it may help to add remove networks to match the interfaces enable for Suricata
add the tunables:
> set net.devmap.bufsize to the value display for NS.MOREFRAG or to the MTU value of proxmo (trial and error)
> set the net.devmap.ad to 1
> set the ns.morefrag to the same as for net.defmap.bufsize
reboot the VMsuricata should now be stable


Context


VM-hardware has Q35 chipset and uses virtio network interfaces.
The OPNSense host has qemu-guest-agent installed.


Indicator (console output)
Jan 28 12:39:45 opnsense kernel: 385.664273 [2197] netmap_buf_size_validate error: large MTU (8192) needed but igb1 does not support NS_MOREFRAG

Assumption
This indicates MTU inconsistency when MTU is set >1500 on the bridge and this is 'broken' in-between the bridge and the IPS. To my understanding the network interfaces available on Proxmox are well supported by OPNSense.

For non-virtualised systems the issue may be the same. Check the MTU of the network, match the MTU of the network on the physical interfaces. Consider subtracting 22 from the MTU for compatibility.

Recommended is to check if
MTU on the bridge is >1500

configure : within Proxmox

check and set the VM-hardware network-interface(s) to 1 so these adopt the MTU of the connected network.
you can consider decreasing the MTU with 22 (now named PMTU)


configure : within OPNSense


[ for Suricata] under the 'advanced' section of the IPS service : check and/or clear default packet size (MTU) setting
setting the MTU here can affect detection reliability and 'drop' or 'conflate' frames on inspection, consider setting MTU-22


[ for Interfaces ] check and/or clear MTU settings for the monitored interfaces OR recommended is to set the PMTU as value
important know that on non-enterprise network cards there may not be support for 'real' Jumbo frames which permits MTU >1500


Look up the specifications for the network interface cards (NIC) and do not set the MTU higher than the hardware supports, even if the MTU on the connecting switch is set to a much higher value.


[ for SYSTEM: SETTINGS: TUNABLES ] manually create the key dev.netmap.bufsize with value = <PMTUvalue>
this to work around issues with some NIC where MTU is not working well, so hard-set it here with this key


configure : optionally for OPNSense


[ for SYSTEM: SETTINGS: TUNABLES ] manually create the key dev.netmap.admode with value = 1this to avoid flapping between native and emulation state for the network interface


[ for Suricata] you must consider set the MTU-22 as size for stability


Considerations

when the value for the MTU is cleared for an interface this defaults to 1500
consider this may severely impact IPS performance and/or accuracy

Resources

https://docs.opnsense.org/manual/ips.html
https://man.freebsd.org/cgi/man.cgi...eBSD+12.1-RELEASE+and+Ports#SUPPORTED_DEVICES
https://man.freebsd.org/cgi/man.cgi?vtnet

10

Intrusion Detection and Prevention / Re: Suricata crashing

« on: January 15, 2024, 06:09:08 pm »

original comment removed

this requires modifying the suricata.yaml file to include the correct sections for the mentioned App-Layer protocols which are missing, this is a best practice since the behavior will change in the future and the protocols will no longer be auto-enabled


"This behavior will change in Suricata 7, so please update your config"



if you have not tweaked the suricata.yaml file, consider looking for a suricata.yaml from a more recent versions


check if these sections are present as such in suricata.yaml, consider adding them at the appropriate place


 #- dnp3
        - dcerpc
        - ftp
          #- ikev2   
        - krb5
        - nfs
        - rdp
        - rfb
        - sip
        - smb
        - snmp
        - tftp
        - dhcp:
           ......



    # Note: parser depends on Rust support
    ntp:
      enabled: yes


    dhcp:
      enabled: yes


    sip:
      enabled: yes
    http2:
      enabled: yes
    snmp:
      enabled: yes
    rfb:
      enabled: yes
    mqtt:
      enabled: yes
    rdp:
      enabled: yes

11

General Discussion / NTP blocked due to bogon filtering on exiting interface

« on: February 12, 2023, 12:16:30 am »

After much wrestling and worrying the fact NTPd does not sync appears to be due to the bogon filtering flag.

I've noticed other mishaps with bogon filtering in the past, it seems to be there should be some automated excemptions so this flag can be left enabled.

Regards,

JL

12

General Discussion / Compressing resolver (Unbound) logs is not possible ?

« on: January 28, 2023, 08:21:49 pm »

From some reason i ended up with over 60GB of logs for unbound.

These are not compressed. I had hoped opnsense would compress the logs automatically, yet i fail to find an option to configure it do so. Typically on a *nix system there's little in the way for reading compressed text log files so I'd prefer to go ahead.

Is there anything to consider for doing so ?

Documentation seems lacking.

13

22.7 Legacy Series / Re: Failing DNS services

« on: January 03, 2023, 08:13:10 pm »

If DNScrypt is a must, use the latest version in a docker container. The one in OPNsense is quite old, unsure where the issue is there but I wouldn't use it on the internet until it is upgraded to current.

Bind -- zone management on the FW wouldn't be my first choice.

For anything else Unbound is more than fit for the job, and latest version as well.


Removing one or two if possible from the chain would help you narrow down the DNS issues.

Thanks, to me these are DoS issues caused by unknown origin.

It is interesting you mention dnscrypt is outdated, i'll check, thanks. Personally, i stay away from Docker, don't like it for no tangible reason. It is bad IT to me.

I disagree Unbound is adequate, it is not a very stable service.
I disagree running zone mgmt on a firewall should not be first choice, if a firewall cannot stay up or stay intact, little use for it.

14

22.7 Legacy Series / Failing DNS services

« on: December 31, 2022, 03:39:58 pm »

While on older opnsense the 'intrusion detection service' frequently crashed, after the upgrade to 22.7 there are new issues, now it are the DNS services crashing ... which worked fine with older opnsense releases.

There are no apparent log entries indicating the reasons why for the DNS service crashes, using unbound+dnscrypt+bind
To my surprise all three service go down simultaneously. As I've noticed at least one succesful (likely) DNS spoofing attempt I'm not confident these crashes are benign.

15

22.7 Legacy Series / Re: DNS not working from firewall, OK from LAN

« on: December 31, 2022, 03:35:02 pm »

how about

host -v opnsense.org

?

or

host -U -v opnsense.org

?