Rule ID format affecting Security Onion ingest

[Waldhaar_]

BLUF: Built in rules have no dashes ("-") in the Rule ID (e.g., 02f4bab031b57d1e30553ce08e0ec131) and Security Onion (Elastic) seems to parse these events fine. User created rules do have dashes (e.g., 0be7d07b-1e80-47ff-8ff8-8e553095d4e5) and Security Onion (Elastic) seems unable to parse any of these rules.

Have been trying to configure my church's OPNsense to send filter logs to our instance of Security Onion. Church was previously using pfsense and filter logs were working. Elastic uses the same integration for both OPNsense and pfsense, so initially ruled out an issue with the configuration on Security Onion. That said, I was seeing the following error: "Provided Grok expressions do not match field value". Conjured up my best SearchFu but couldn't find anything definitive. The Elastic integration for OPNsense/pfsense seems well maintained so it seemed odd that it couldn't handle OPNsense's input.

So, i pointed my home instance of OPNsense to the Church's Security Onion (networks are connected via Wireguard, and are running exact same hardware and versions of OPNsense) and it seemed like the logs from home worked fine. However, eventually, I did find some from home that weren't being parsed (same error) and some from church that were parsed fine.

I copied the original messages of both good and bad events to compare and could find only one difference; the presence of dashes in the Rule ID. All events with dashes in the Rule ID failed to parse and all events with no dashes parse fine. As far as I can tell, all built-in rules are dash free and all user created rules use dashes. Turns out, for my home instance, I don't have explicit deny rules on each interfaces (using the built in one) whereas the church does have explicit deny rules per interface. This is why, I think, most events from my home instance worked and most from the church did not.

So, anyone else using Security Onion seeing the same thing? I am also curious why the difference in formats for Rule IDs.

[keeka]

Yes, rule id now seem to have differing formats. It messed up my graylog parsing for a while. I am not sure what led to the decision for mixing UUID style. It would be better if they were consistent IMO. In the meantime ;-) are you able to modify the grok expression to cater for both formats?

[franco]

If you add a ticket on GitHub that's something to consider for improvement. I agree that it shouldn't differ but we need to isolate the code bits responsible first to make a meaningful plan forward.


Thanks,
Franco