OPNsense Forum

Archive => 23.1 Legacy Series => Topic started by: Sahbi on April 02, 2023, 01:11:12 pm

Title: [SOLVED] Suricata prevents access to OPN itself
Post by: Sahbi on April 02, 2023, 01:11:12 pm

Since the upgrade to 23.1 I noticed that a few minutes after booting up I lose all connectivity to OPN itself. As in, I can't get in the web GUI or even SSH into it, and existing sessions disconnect with "broken pipe" errors. Traffic still properly passes through OPN though, and oddly enough it does still respond to pings. I finally figured out this was due to Suricata:

• Reboot OPN simply by power cycling
• Continuously try to SSH into OPN and kill Suricata (while true; do ssh root@opn killall -9 suricata; sleep 3; done)
  
• When it succeeds, stop the loop and open a regular SSH session
• This session keeps working indefinitely, unlike before
• Go into OPN web GUI and change Suricata to stop listening on the interface I would access OPN through
• Start Suricata again
• SSH keeps working, but other networks are once again unable to access OPN
• Suricata isn't logging an alert about this, so it's not somehow a rule that's blocking it (I even added a pass rule for my specific IP)
• I also made sure sshlockout wasn't triggered (list is still empty)

I think there may be a problem with Suricata opening the same interface multiple times. I checked the latest.log and these are the only lines that appear right before I lose connectivity:

Code: [Select]

[meta sequenceId="1"] [102930] <Notice> -- opened netmap:igb3/R from igb3: 0x82a915000
[meta sequenceId="2"] [102930] <Notice> -- opened netmap:igb3^ from igb3^: 0x82a915300
[meta sequenceId="3"] [102930] <Notice> -- opened netmap:igb3^ from igb3^: 0x8556f4000
[meta sequenceId="4"] [102930] <Notice> -- opened netmap:igb3/T from igb3: 0x8556f4300

[meta sequenceId="5"] [102930] <Notice> -- opened netmap:igb2/R from igb2: 0x8804f4000
[meta sequenceId="6"] [102930] <Notice> -- opened netmap:igb2^ from igb2^: 0x8804f4300
[meta sequenceId="7"] [102930] <Notice> -- opened netmap:igb2^ from igb2^: 0x8ab2f4000
[meta sequenceId="8"] [102930] <Notice> -- opened netmap:igb2/T from igb2: 0x8ab2f4300

I'm guessing the /R and /T are for receive and transmit, but then what are the 2 lines with ^? I think those being present (either twice, or besides the R/T variants) might cause Suricata to kind of lose track of traffic destined for OPN and it disappears into a black hole.

I'm not entirely sure if this is a problem with Suricata itself, or simply the config OPN generates for it. So I'll try here first. =]

Title: Re: Suricata prevents access to OPN itself
Post by: Sahbi on April 12, 2023, 09:51:51 pm

Little bump after doing a bit more research.

So apparently the igbX^ interfaces are simply there due to IPS mode, to allow traffic to still be seen by OPN and all. Setting Suricata to simple detection mode does resolve the connectivity problem, but then I also don't have any actual prevention. :> I think setting it to IDS only changes one config parameter (possibly one per interface), so I'm pretty sure it's not a config problem and more likely related to Suricata and/or netmap themselves.

I did some other searches around the forum and found a few mentions of the Suricata version I'm on (6.0.9) potentially having problems, so I downgraded it to 6.0.8 as per other people's experiences, but nothing changed for me. I also found some issues related to netmap but not exactly related to mine, e.g. stuff with emulated interfaces/generic mode (which I'm not using). There's a newer version of netmap that's (probably) still in internal testing (https://forum.opnsense.org/index.php?topic=32114.msg161977#msg161977), but I tried it out anyways since Franco did mention another fix. Unfortunately, again nothing changed. I'm doing full reboots every time just to be sure.

Title: Re: Suricata prevents access to OPN itself
Post by: franco on April 12, 2023, 10:03:42 pm

Interfaces: Settings: disable all hardware features that are off by default? ;)


Cheers,
Franco

Title: Re: Suricata prevents access to OPN itself
Post by: Sahbi on April 13, 2023, 06:09:14 pm

I did try that before changing Suricata/netmap versions, but not afterwards. Surprisingly enough that does work now. :> On a certain OPN version I could run Suricata with just VLAN acceleration disabled though, so is the requirement for disabling all acceleration stuff something from newer OPNs/BSDs, or is it perhaps related to the netmap fixes? Or maybe I was just lucky that it used to work at all?