Local vs Remote confusion

[GeoffW]

I've got a new OPNsense installation, and now I'm trying ZenArmor v1.12.3 in Passive mode.  I'm mostly interested in the session details capability, I don't really want all the extra block/filtering smarts so Passive mode seems the best fit.   Installed using MongoDB.  OPNsense is running as a VM inside ESXi v7 using three physical interfaces I called WAN, LAN, DMZ.  ZenArmor set to monitor LAN and DMZ (but insists on using the underlying interface name vmx1 and vmx2).

Anyway the confusion coming in when I see that "Top Local Hosts" reports mostly Internet names/addresses, although there are a couple of LAN address in there too, while "Top Remote Hosts" is showing only LAN addresses.

Much the same on "Top Local Server Ports" vs "Top Remote Ports".  The "local" is showing stuff I know to be destination ports on the Internet, while the "remote" is showing stuff I'm pretty sure is LAN only.

I have set up to use DNS over TLS with LAN and DMZ port-forward rules to enforce this.  So there is port 53 traffic on the LAN and DMZ interfaces but should only be 853 over the WAN.  Could this be confusing ZenArmor.

Or maybe I am the one who is confused and perhaps "local" and "remote" mean different things in this context?

PS: or is this a duplicate of this question: https://forum.opnsense.org/index.php?topic=31674.0

[GeoffW]

Some extra detail...

The "Table of Local Assets" shows IP addresses (untranslated IP addresses) from LAN and DMZ, so this looks right.

The "Table of Remote Hosts" shows only the two default gateway IP addresses (one for the LAN interface and one for the DMZ interface).   Perhaps this is expected, but if so it doesn't seem particularly useful.

I did just realise that using the drilldown and session detail feature from the Top Local Hosts tries to use these as the source address, and since most of these are Internet destination addresses this produces empty results.  (Ditto the Top Remote Hosts searching for destinations using source addresses.)  Which is a relief, it suggests local and remote do mean what I think they mean, and it's ZenArmor that's having an inconceivable moment.


Note that doing a drilldown or session detail from "Top Local Server Ports" searches for a destination port, which makes sense because servers are destinations by definition I would have thought, but of course finds records in which the destination IP is Internet (remote).  "Top Remote Ports" also searches for destination port, which also makes sense, but in my case finds only destination IPs on the LAN or DMZ (for port 53 because of my DoT set up).  But then the filtering from these reports is a bit suspect anyway, since filtering only by destination port is not going to give local vs remote distinction anyway, and that is what I'm seeing when I use this on ports used on both local and remote servers.

[sy]

Hi,

Please exclude the DMZ interface in the report by adding a filter as the interface is not equal "DMZ". Are the Top Local / Remote hosts report still same?

[GeoffW]

Thanks for your reply.

I'm pretty sure I must be doing something wrong, because I cannot get any interface filter to work as expected.

As I noted in the OP, Zenarmor insists on using the nic device name, not the other possibilities (DMZ or opt1), so in this case vmx2.  But just in case I've tried all variations.  And just to be sure, I tried setting up a positive (equals) test using LAN, "LAN", 'LAN', lan, "lan", 'lan', vmx1, "vmx1", 'vmx1'

All the "NOT EQUALS" tests I've tried make no difference to the output.  All the EQUALS tests I've tried produce empty results.

I notice that the filter display, after I hit refresh, changes from showing something like: Interface: LAN

to showing: Interface: Lan

And (so on with quotes or not according to the variation I tested).  Is there any chance this is actually doing a case-sensitive match and so getting it wrong on my all upper or all lowercase interface names?  Or is this just a filter display error?

It really seems like this must be a user error, but I cannot work out what I'm doing wrong.  Removing the interface filter brings everything back to what I've described.

[GeoffW]

I figure I have nothing nothing much to lose on this new installation, so I changed the configuration to remove DMZ (vmx2) from being monitored, then under Reporting & Data in the configuration I chose toe Erase Reporting Data (all of it).

Then I visited a few sites and checked the reports.  I'm still seeing remote names in the "Top Local Hosts" (plus IP of the local device doing the browsing), and seeing LAN IPs in the "Top Remote Hosts".  (And so on for the ports etc.)

There's probably nothing stopping me from doing an uninstall and reinstall of Zenarmor if that might make a difference.

[GeoffW]

So I went back to a previous snapshot and reinstalled, again using Passive Mode, again turned off the Cloud Reputation, but this time selecting only LAN (vmx1).  Same problems.

Then I tried changing the mode (on the above install) from Passive Mode to Routed Mode (native netmap).  Same problems.

Then I did an uninstall and reinstall, but this time I chose Routed Mode (native map) right at install, again turned off the Cloud Reputation and still selecting only LAN (vmx1).  And this time it's looking much better.  Remote hosts are showing under remote hosts and local under local ... the only exception seems to be local host name for the ntp server showing up in the remote stuff. (My devices point to this name for time sync rather than directly to the gateway address - long story.)

For the next test, I went into configuration and switched back to Passive Mode and added DMZ (vmx2).  It seemed to take a few minutes for the impact to show, but it looks like everything is moving back to being the wrong way around as described in the OP.  So it looks like I took too many steps in this test, I don't know which broke it.

I changed the config back to Routed Mode (native map), and left DMZ (vmx2) in the list.  So far it is looking okay.  Both LAN and DMZ hosts are showing in local hosts (and no stray internet stuff).  The remote hosts lists are mostly remote, but do also show the gateway addresses (so "LAN address" and "DMZ address", if you see what I mean); this may be expected and okay, since the gateway is acting as ntp server.

( One oddity is that it seems to show an internal media server as having some internet connections - which I confirmed by a monitor on the host itself.  This host does not have an exception in the Captive Portal, so it seems that I have more issues to run down yet. )


Summary:  It appears to be Passive Mode that causes the problem.  And to avoid it you MUST install in a Routed Mode, simply switching away from Passive Mode doesn't work unless the initial install was in Routed Mode.


P.S. I still cannot get it to filter by Interface name.

[sy]

Hi,

Thanks for the detailed information. Zenarmor collects the destination address as a Remote host. So, when your client is communicating with a local server it seems in the Remote Hosts graphic.

For the passive mode issue, there is no known issue such as. I'm going to check it and inform again.

[BNaCl]

Not sure if this is helpful as I am running in a somewhat unique config which I will explain below, but my local/remote hosts are also jumbled up with internal and external IPs. When I drill down into session detail for any of the local or remote host charts they are largely blank. The app breakdown charts at the top are populating session detail correctly.

Now, it should be noted that I am running OPNs as a transparent bridge. ZA has acknowledged that their bridge deployment mode is having issues with Netmap and is working on the fix. In the meantime, I was trying ZA in Passive Mode so I can get the analytics, but I am seeing similar to what you are, if not worse. My guess is ZA just wont work with OPNs bridges yet in either Passive or Bridge Modes.   

[GeoffW]

Thanks for your input BNaCl.  It sounds like there maybe overlap between what you see and what I see, but probably more than one issue.  However, your mention of bridges made me think to add (just in case it turns out to be relevant):

The WAN interface in my set up is connected to a 4GB modem set in a transparent bridge mode (its default) rather than router mode.  This has caused me some issues getting the WAN address updated when modem reboots or whatever, mostly solved with a cron script now I hope, but I doubt if this is related to what I'm seeing in ZenArmor.

[BNaCl]

Yeah agree that wouldn't be a factor as long as OPNs and ZA are seeing the WAN IP, which it seems it is. I re-read your setup and associated troubleshooting and think it is sound. The only question is if the virtual layer is adding some complexity which is contributing.

FWIW, I have been using both the free and paid versions off/on for over 3 years and the filters have been particularly problematic. Sometimes you can find one that works, sometimes not, wildcards seem to not be a thing, etc. IMO, it  seems to me they are working through some gremlins. That being said, I find their product to have a lot of promise and still provides a lot of insight/value. The good news is they are continually releasing updates/fixes so I am hopeful the progress continues.

Question, is there a reason you can't use routed mode since it seems to work correctly? You can basically make it perform as passive mode by not enabling any of the blocking/filtering. Like I mentioned, their bridge mode is not functioning correctly and I would use routed mode, but I would have to rework my network which I don't want to do (I also agreed to beta test the bridge fixes they are working on). Just a thought. 

[GeoffW]

I am indeed using Routed Mode (with Cloud Threats turned off) at the moment, at least while I wait to hear back from sy, because the confusion in Passive Mode makes using the product difficult.

I started with Passive Mode because that's all I wanted and I thought it might have less overhead (but don't know if that's true).  I have a very small network, just a few users and occasional guests (they're not encouraged  ;) ), and for now at least I am content with the Unbound DNS blocklist feature for blocking stuff.

What I find myself looking for most regularly is "what is (or was) causing that traffic?"  It seems to be the Sessions Explorer in ZA that most easily answers that question.  The normal reporting options in OPNs and ntopng show parts that you have to join together yourself (unless I'm missing something obvious, it wouldn't be the first time).  What I'd really like was a product that took me directly from "bump in traffic graph" to "sessions active at that moment".  ZA comes close.

[jclendineng]

It doesn't matter, both tie into netmap and as such both will perform similarly as long as you have decent hardware. For context I have 10Gb networking and with ZA enabled passive OR routed bandwidth is cut to ~5gb to about half. SO I don't think it matters and you could just leave it in routed mode.

[sy]

Hi,

Actually, the passive mode is using pcap, not netmap. the disadvantage of it is that you lose blocking features. @GeoffW, I will get back to you on Monday about the passive mode issue.

[Prevok]

Sorry to revive an older thead. I just didn't see value in starting a new one for the same issue.

Has there been any movement on this? I am a new Zenarmor user and have the same issue.

I have a very simple setup, I only monitor the LAN interface in passive mode and the remote and local are inversed. Drilling down is essentially useless. I have not tried the routed approach, as I have i225-v3 interfaces and read there was incompatibility with Netmap.

[jclendineng]

Hi,

Actually, the passive mode is using pcap, not netmap. the disadvantage of it is that you lose blocking features. @GeoffW, I will get back to you on Monday about the passive mode issue.

Interesting I did not know that! Thank you for the clarification.