Messages

Wanted to start with a thank you, the /usr/local/etc/suricata/conf.d/custom.yaml appears persistent and this allows us to further customize Suricata, thank you OPNSense!! We have had difficulty customizing and having it persist until now.

The issue found appears to be that BPF Filtering via netmap in the Suricata config does not appear to work, hosts/networks filtered still show up in the Suricata alerts. The netmap area of suricata.yaml from OPNSense 25.x to OPNSense 26.x appears to have gone through a lot of changes.

The new divert feature being one of the new elements I have not had a chance to explore, I'm using PCAP IPD mode for Suricata.

The documentation for BPF-Filtering within Suricata shows this as an example:

Code Select Expand

not (host IP1 or IP2 or IP3 or net NET/24)
A most desired filter is between a subnet and one or more other subnets, and I've tried this in several methods, here's the most basic I have tried:

Code Select Expand

not ((net NET1/24 and NET2/24) or (net NET2/24 and NET3/24) or (net NET3/24 and NET1/24))
So far none of the BPF filters work with Suricata - does anyone use this and/or have experience?

Edit:
For a while I was only looking at the suricata.yaml file and didn't realize the level of control that went into the command line execution of arguments sent to suricata on service start. For a while I was trying to put netmap in IDS mode thinking that's how the traffic was going and wondering why the BPF filter for netmap wasn't working.

Did the upgrade to 26.1.3, everything so far was working, except a home grown automation to sync stuff gathered from the OPNSense API, Portainer, and kept in a Redis Cache to my BIND setup. This is done by a Python script.

In a most strangest interaction, use case testing Python using requests, http, or httpx and was not able to reach and more correctly connect to OPNSense's API, but Postman/CURL both could - and got valid responses back. Tested the HTTP/1.1 or HTTP/2 nature, always worked for Postman and CURL, never worked for Python.

So I went back to the OPNSense, maybe I could re-reset some stuff, started simple - restarted the System Configuration Daemon and then Web GUI services from the Web GUI, suddenly Python is working again.

Also did a another restart of the OPNSense (it restart initially after the upgrade), and the issue did not present itself again. So this is just a heads up to any that might hit whatever edge case I did, restart services and/or host, should work again.

Not terribly often talked about, but, all FQDNs (even domains) have a period at the end. So, "google.com" is actually "google.com." and we just get to skip the last period because we never had to use it, they have let us skip it since for about forever (in nearly every service, save, Raw/back-end DNS).

At times like this you see the standard show up where you get to see the last period at the end of a name on the network.

You could pursue using Suricata's own rule update/sync mechanism if you wish:

https://www.nova-labs.net/using-suricata-update-on-opnsense/

^ there is only one major deviation, and that is that the custom.yaml file is in a new location and the contents have updated a little... I will update the blog post or make a new one eventually.

The firewall migration worked well for me, and I was able to manually move over ISC to KEA ahead of time and then remove the plug-in that wasn't needed anymore.

To any wanting to 'sync' their KEA IPv4 and IPv6 Reservations by just creating a IPv4 Reservation... here's some Python ^_^
https://github.com/j0nny55555/homelabdnsupdater/blob/main/ipv6reservationupdates.py

Great update, thank you!!

Just wanted to extend OPNSense, the community, FreeBSD, associated projects, and the Open Source community a thank you!!

I wrote a Linux ran multi-threaded Python detail to update several Aliases (two of which have 10k+ IPs) with IP lists via OPNSense API, and on average it is completing now in about 30% less time (that means I block an attacker (CrowdSec enabled Reverse Proxy + Suricata) at Layer 3 faster).

No change to my code, I've already tweaked it for speed in about every way (nice in Linux, pre-compiling Python...) and this speed up is purely because OPNSense continues to become a more lean/mean beast.

Can the OPNSense affected also if hacker got access to LAN?

Internal Firewall rules with separate zones/interfaces for Wifi/Client/DMZ/Core/etc. Would advise using VLANs if you can, otherwise subneting with /24s is a good idea.

From what I've read, you might also want to turn on MAC-Address filters on your WAPs and/or OPNSense's DHCP, good luck!

Ok - so it was the option in Unbound "Register DHCP Static Mappings" which more or less cancelled the forwarding for the local domain, with that disabled, it follows forwarding rules...

So the Overrides work as intended, which is great, and now Aliases asks Unbound (localhost:53) for hosts and my Aliases now update as expected.

Should be whatever is configured in System: Settings: General.

No, I believe it is the localhost's (the OPNSense) port 53 if it is turned on at all - and for me that is Unbound.

The reason I think this is that the Alias wasn't able to get the IPv6 addresses for hosts that are on the LAN and not overridden in Unbound. If Unbound had to forward for the local domain AAAA resolution, it did not work/resolve. It does Upstream correctly, all public (not LAN) AAAA upstream resolve, but Conditionally (locally?) Forwarded AAAA does not happen - and since this is the issue, each affected Alias has only 1 IP, its IPv4 address.

To summarize the first post, Unbound does not Query Forward for a "local domain" AAAA, but does Query Forward both local A and PTR (and PTR for either IPv4 or IPv6). Further, yes it does the TLS resolution for all public stuffs, I'm only having the issue with otherwise local (but technically a /64 public DHCP IPv6 subnet) AAAA Unbound Query Forwarding.

Environment detail:
OPNSense Unbound for DNS Upstream + Overrides (so when external to OPNSense local domain BIND is down critical infra still works), Hosts use PiHoles, and both OPNSense Unbound and PiHoles use local BIND infra for the Local Domain. The Local BIND has the forward and reverse lookup all setup, and populated, and Unbound and the PiHoles are set to forward for the local domain and all /24 IPv4 and /64 IPv6 subnets for reverse DNS lookup. This worked previously - I am considering going back to verify.

Upgrade journey:
Migrated from ISC to KEA, Upgraded, did the firewall migration, removed ISC plugin, most everything works well - most hosts seem to correctly populate their Alias content counts for IPs to Hostnames.

Testing the process:
There is a Python script I wrote that updates forward and reverse records in the local Bind infra for the hostnames via OPNSense(ARP/NDP/Reservations)/Portainer(Docker Hosts) and I can

Code Select Expand

drill fqdn @pihole or

Code Select Expand

drill -x ip @pihole for A and AAAA/IPv4 and IPv6, and together I get 2+ IPs back as expected. In this case the hostname happens to be "plex.localdomain.home" (not really but close enough), and yeah, most/all other hostnames appear to correctly populate their counts (especially those that are overridden via IPv4 and IPv6 entries in Unbound's Override space).

Problem:
The issue is that the OPNSense Firewall Alias for the FQDN in question only has one value for its "content", or just one IP resolved. This FQDN is not overridden in Unbound. OPNSense's Host discovery / Host detect sees all the IPs for the FQDN's associated MAC address, and all of them resolve to the FQDN against PiHoles/BIND, but Alias does not? Seems odd. I'm curious where the configuration/direction for OPNSense's firewall to resolve hosts comes from - which DNS source of truth is it using?

!!! Interesting:
Doing a drill against the OPNSense for that FQDN and AAAA returns nothing, but from either PiHole or BIND, results. Interestingly though, if I do a reverse lookup on the FQDN's IPv6 against the OPNSense it would seem Unbound responds with the IPv6's FQDN, so A (IPv4 forward DNS) and IPv4 and IPv6 PTR (both IPv4 and IPv6 reverse DNS) works, but AAAA (IPv6 forward DNS) does not for Unbound query forward/response?
!!! Further:
After disabling all Unbound Overrides for the local domain, it still has the same issue - AAAA query for local domains fails - and yes, I have the local domain added to the "Private Domains" in Unbound's Advanced settings. Extra, in this, it would seem to only know about the IPv6 addresses for FQDNs that were overridden, and is unable to do a conditionally forwarded AAAA/forward-ipv6 lookup (unless the FQDN in question is IPv6 overridden manually, and then it isn't forwarding/asking, it is merely answering if you will).

(Extra - I'm considering setting up the BIND plugin on the OPNSense just so I can have my existing Primary BIND send updates to what would be OPNSense's Secondary BIND. Want to possibly understand why it doesn't already work, and maybe explore what is necessary to configure the BIND plugin to be a secondary BIND server as a part of my existing infra while keeping the state in OPNSense conf/backup - fix 1 problem w/possibly 2 or more problems lol, but if anyone has pointers on the original issue, pls lmk)

Ok - so it was the "Register DHCP Static Mappings" which more or less cancelled the forwarding for the local domain, with that disabled, it follows forwarding rules...

Currently still on 25.7.11_9 and have transitioned from ISC to KEA, and so far things are working okay.

With ISC, I could find all of my leases for DHCPv4 and DHCPv6, but with KEA, that does not seem accessible. I tried looking into "host discovery" / "host watch", but maybe it isn't built out in 25.7.x yet. Curious what we can expect to use "host discovery" for and if the data will be available via the API?

Are there plans with KEA to allow us to see our DHCPv6 leases via API, both reserved and un-reserved?

What more info is needed? What should I look at?

Logs are being rotated daily, settings say weekly.
More than 4 logs are saved, settings say save 4.

gotta admit, i have mine set at 2 weekly, and i only have 2... i was about to say "that's 4 weeks of logs..." but i only have two files and 2 + weekly... not sure if either of our retention is matching the configured state

i did figure out how to enable manual rotation of an extra suricata log file i have created through the use of suricata's custom.yaml, and this file has stuck around through several upgrades

file name example:
/usr/local/etc/newsyslog.conf.d/suricataxff.conf:

content example:
# logfilename [owner:group] mode count size when flags [/pid_file] [sig_num]
/var/log/suricata/evexff.json      root:wheel      640     1       500000  $W0D23  B       /var/run/suricata.pid   1

There's one isr setting that I believe really enables the rest, see attached image too.

net.isr.dispatch: deferred

Checked the app-layer section of the latest suricata.yaml file against suricata.yaml, this is a strong inspection enablement

Code Select Expand

app-layer:
  # error-policy: ignore
  protocols:
    telnet:
      enabled: yes
    rfb:
      enabled: yes
      detection-ports:
        dp: 5900, 5901, 5902, 5903, 5904, 5905, 5906, 5907, 5908, 5909
    mqtt:
      enabled: yes
      # max-msg-length: 1 MiB
      # subscribe-topic-match-limit: 100
      # unsubscribe-topic-match-limit: 100
      # Maximum number of live MQTT transactions per flow
      # max-tx: 4096
    krb5:
      enabled: yes
    bittorrent-dht:
      enabled: yes
    snmp:
      enabled: yes
    ike:
      enabled: yes
    tls:
      enabled: yes
      detection-ports:
        dp: 443

      # Generate JA3/JA4 fingerprints from client hello. If not specified it
      # will be disabled by default, but enabled if rules require it.
      ja3-fingerprints: auto
      ja4-fingerprints: auto

      # What to do when the encrypted communications start:
      # - track-only: keep tracking TLS session, check for protocol anomalies,
      #            inspect tls_* keywords. Disables inspection of unmodified
      #            'content' signatures. (default)
      # - bypass:  stop processing this flow as much as possible. No further
      #            TLS parsing and inspection. Offload flow bypass to kernel
      #            or hardware if possible.
      # - full:    keep tracking and inspection as normal. Unmodified content
      #            keyword signatures are inspected as well.
      #
      # For best performance, select 'bypass'.
      #
      #encryption-handling: track-only

    pgsql:
      enabled: yes
      # Stream reassembly size for PostgreSQL. By default, track it completely.
      stream-depth: 0
      # Maximum number of live PostgreSQL transactions per flow
      max-tx: 1024
    dcerpc:
      enabled: yes
      # Maximum number of live DCERPC transactions per flow
      # max-tx: 1024
    ftp:
      enabled: yes
      # memcap: 64 MiB
    websocket:
      enabled: yes
      # Maximum used payload size, the rest is skipped
      # Also applies as a maximum for uncompressed data
      max-payload-size: 64 KiB
    rdp:
      #enabled: yes
    ssh:
      enabled: yes
      # hassh: no

      # What to do when the encrypted communications start:
      # - track-only: keep tracking but stop inspection (default)
      # - full:    keep tracking and inspect as normal
      # - bypass:  stop processing this flow as much as possible.
      #            Offload flow bypass to kernel or hardware if possible.
      # For the best performance, select 'bypass'.
      #
      # encryption-handling: track-only
    doh2:
      enabled: yes
    http2:
      enabled: yes
      # Maximum number of live HTTP2 streams in a flow
      #max-streams: 4096
      # Maximum headers table size
      #max-table-size: 65536
      # Maximum reassembly size for header + continuation frames
      #max-reassembly-size: 102400
    smtp:
      enabled: yes
      raw-extraction: no
      # Maximum number of live SMTP transactions per flow
      # max-tx: 256
      # Configure SMTP-MIME Decoder
      mime:
        # Decode MIME messages from SMTP transactions
        # (may be resource intensive)
        # This field supersedes all others because it turns the entire
        # process on or off
        decode-mime: yes

        # Decode MIME entity bodies (ie. Base64, quoted-printable, etc.)
        decode-base64: yes
        decode-quoted-printable: yes

        # Maximum bytes per header data value stored in the data structure
        # (default is 2000)
        header-value-depth: 2000

        # Extract URLs and save in state data structure
        extract-urls: yes
        # Scheme of URLs to extract
        # (default is [http])
        #extract-urls-schemes: [http, https, ftp, mailto]
        # Log the scheme of URLs that are extracted
        # (default is no)
        #log-url-scheme: yes
        # Set to yes to compute the md5 of the mail body. You will then
        # be able to journalize it.
        # Set it to no to disable it.
        # Default is auto: not enabled until a rule needs it
        # body-md5: auto
      # Configure inspected-tracker for file_data keyword
      inspected-tracker:
        content-limit: 100000
        content-inspect-min-size: 32768
        content-inspect-window: 4096
    imap:
      enabled: detection-only
    pop3:
      enabled: yes
      detection-ports:
        dp: 110
      # Stream reassembly size for POP3. By default, track it completely.
      stream-depth: 0
      # Maximum number of live POP3 transactions per flow
      # max-tx: 256
    smb:
      enabled: yes
      detection-ports:
        dp: 139, 445
      # Maximum number of live SMB transactions per flow
      # max-tx: 1024

      # Stream reassembly size for SMB streams. By default track it completely.
      #stream-depth: 0

    nfs:
      enabled: yes
      # max-tx: 1024
    tftp:
      enabled: yes
    dns:
      tcp:
        enabled: yes
        detection-ports:
          dp: 53
      udp:
        enabled: yes
        detection-ports:
          dp: 53
    http:
      enabled: yes

      # Byte Range Containers default settings
      # byterange:
      #   memcap: 100 MiB
      #   timeout: 60

      # memcap:                   Maximum memory capacity for HTTP
      #                           Default is unlimited, values can be 64 MiB, e.g.

      # default-config:           Used when no server-config matches
      #   personality:            List of personalities used by default
      #   request-body-limit:     Limit reassembly of request body for inspection
      #                           by http_client_body & pcre /P option.
      #   response-body-limit:    Limit reassembly of response body for inspection
      #                           by file_data, http_server_body & pcre /Q option.
      #
      #   For advanced options, see the user guide


      # server-config:            List of server configurations to use if address matches
      #   address:                List of IP addresses or networks for this block
      #   personality:            List of personalities used by this block
      #
      #                           Then, all the fields from default-config can be overloaded
      #
      # Currently Available Personalities:
      #   Minimal, Generic, IDS (default), IIS_4_0, IIS_5_0, IIS_5_1, IIS_6_0,
      #   IIS_7_0, IIS_7_5, Apache_2
      libhtp:
         default-config:
           personality: IDS

           # Can be specified in KiB, MiB, GiB.  Just a number indicates
           # it's in bytes.
           request-body-limit: 100 KiB
           response-body-limit: 100 KiB

           # inspection limits
           request-body-minimal-inspect-size: 32 KiB
           request-body-inspect-window: 4 KiB
           response-body-minimal-inspect-size: 40 KiB
           response-body-inspect-window: 16 KiB

           # response body decompression (0 disables)
           response-body-decompress-layer-limit: 2

           # auto will use http-body-inline mode in IPS mode, yes or no set it statically
           http-body-inline: auto

           # Decompress SWF files. Disabled by default.
           # Two types: 'deflate', 'lzma', 'both' will decompress deflate and lzma
           # compress-depth:
           # Specifies the maximum amount of data to decompress,
           # set 0 for unlimited.
           # decompress-depth:
           # Specifies the maximum amount of decompressed data to obtain,
           # set 0 for unlimited.
           swf-decompression:
             enabled: no
             type: both
             compress-depth: 100 KiB
             decompress-depth: 100 KiB

           # Use a random value for inspection sizes around the specified value.
           # This lowers the risk of some evasion techniques but could lead
           # to detection change between runs. It is set to 'yes' by default.
           #randomize-inspection-sizes: yes
           # If "randomize-inspection-sizes" is active, the value of various
           # inspection size will be chosen from the [1 - range%, 1 + range%]
           # range
           # Default value of "randomize-inspection-range" is 10.
           #randomize-inspection-range: 10

           # decoding
           double-decode-path: no
           double-decode-query: no

           # Can enable LZMA decompression
           #lzma-enabled: false
           # Memory limit usage for LZMA decompression dictionary
           # Data is decompressed until dictionary reaches this size
           #lzma-memlimit: 1 MiB
           # Maximum decompressed size with a compression ratio
           # above 2048 (only LZMA can reach this ratio, deflate cannot)
           #compression-bomb-limit: 1 MiB
           # Maximum time spent decompressing a single transaction in usec
           #decompression-time-limit: 100000
           # Maximum number of live transactions per flow
           #max-tx: 512
           # Maximum used number of HTTP1 headers in one request or response
           #headers-limit: 1024

         server-config:

           #- apache:
           #    address: [192.168.1.0/24, 127.0.0.0/8, "::1"]
           #    personality: Apache_2
           #    # Can be specified in KiB, MiB, GiB.  Just a number indicates
           #    # it's in bytes.
           #    request-body-limit: 4096
           #    response-body-limit: 4096
           #    double-decode-path: no
           #    double-decode-query: no

           #- iis7:
           #    address:
           #      - 192.168.0.0/24
           #      - 192.168.10.0/24
           #    personality: IIS_7_0
           #    # Can be specified in KiB, MiB, GiB.  Just a number indicates
           #    # it's in bytes.
           #    request-body-limit: 4096
           #    response-body-limit: 4096
           #    double-decode-path: no
           #    double-decode-query: no

    # Note: Modbus probe parser is minimalist due to the limited usage in the field.
    # Only Modbus message length (greater than Modbus header length)
    # and protocol ID (equal to 0) are checked in probing parser
    # It is important to enable detection port and define Modbus port
    # to avoid false positives
    modbus:
      # How many unanswered Modbus requests are considered a flood.
      # If the limit is reached, the app-layer-event:modbus.flooded; will match.
      #request-flood: 500

      enabled: yes
      detection-ports:
        dp: 502
      # According to MODBUS Messaging on TCP/IP Implementation Guide V1.0b, it
      # is recommended to keep the TCP connection opened with a remote device
      # and not to open and close it for each MODBUS/TCP transaction. In that
      # case, it is important to set the depth of the stream reassembling as
      # unlimited (stream.reassembly.depth: 0)

      # Stream reassembly size for modbus. By default track it completely.
      stream-depth: 0

    # DNP3
    dnp3:
      enabled: yes
      detection-ports:
        dp: 20000

    # SCADA EtherNet/IP and CIP protocol support
    enip:
      enabled: yes
      detection-ports:
        dp: 44818
        sp: 44818

    ntp:
      enabled: yes

    quic:
      enabled: yes

    dhcp:
      enabled: yes

    sip:
      enabled: yes

    ldap:
      tcp:
        enabled: yes
        detection-ports:
          dp: 389, 3268
      udp:
        enabled: yes
        detection-ports:
          dp: 389, 3268
      # Maximum number of live LDAP transactions per flow
      # max-tx: 1024

    mdns:
      enabled: yes

A reference to what I am checking against:
https://github.com/OISF/suricata/blob/main/suricata.yaml.in

You will have to re-register your agents (clients/parsers/blockers) on any CrowdSec Server setup, you cannot migrate. This is true too if you switch DBs for the Server as it stores this in the DB.

That said, if you deploy the CrowdSec Server outside of the OPNSense and just use the CrowdSec Agents (Parser and Blocker) features on the OPNSense, you can more or less have a 'hard coded' deployed to work list of agents as you can seed the agents on Docker Compose deploy via the Environment element of the 'docker-compose.yml' file.

The other option would be to have a script that creates your machines and bouncers (the parser and blocker/bouncer agents) in the Server. Once you have that, just maintain your IaC (Infrastructure as Code) to match your environment and if you have to rebuild, you can have your setup tool-kit.